diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f131747fa..b3bf0e770 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -524,7 +524,7 @@ jobs: if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.linux == 'true' }} name: test (tsgo and tsc-ts give the same IR) runs-on: ubuntu-24.04 - timeout-minutes: 30 + timeout-minutes: 60 steps: - uses: actions/checkout@v4 - uses: pnpm/action-setup@v4 diff --git a/.gitignore b/.gitignore index cd44fe11d..b271af8e1 100644 --- a/.gitignore +++ b/.gitignore @@ -33,6 +33,7 @@ output/ /packages/runtime-*/runtime-pack.json /packages/runtime-*/.runtime-pack-* !tests/fixtures/npm/node_modules/*/dist/ +!tests/fixtures/npm/node_modules/@*/*/dist/ !tests/fixtures/npm/workspace/*/dist/ !tests/fixtures/npm/cases/workspace-copied/members/*/dist/ !tests/fixtures/npm/cases/workspace-copied/node_modules/*/dist/ diff --git a/CHANGELOG.md b/CHANGELOG.md index ec0b27dbf..96f13fa26 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,28 @@ All notable changes to scriptc will be documented in this file. +## 0.4.3 + +### Highlights + +- **Faster checking and smaller builds with tsc-ts 0.0.9:** scriptc checks with tsc-ts 0.0.9, and builds reuse the bundled declaration files' diagnostics instead of checking them again. Small IR builds are 10.6 to 16.0% faster and small executable builds 5.5 to 7.6% faster, with identical output. A native build of tsc-ts with this release checks 0.4 to 6.6% faster than with 0.4.2 and peaks 4 to 16% lower. +- **Integer arrays and loops:** + - Number arrays that only hold int32 values use 4-byte slots in locals, across direct calls and in module globals, and loop-carried locals written from their elements stay int32. A workload that walks arrays of cursors runs 54.5% faster. + - Loops with two counters closing in on each other (`for (let i = lo, j = hi; i < j; i++, j--)`) and element copies through `a[i]!` run without per-element checks, and two-counter `while` loops run on integer counters. + - Loops over typed arrays and charAt scans are versioned on their bounds, and ASCII strings are scanned as bytes. +- **Less work for untyped and dynamic code:** + - String-keyed record reads under `??` and `!` no longer allocate: a dictionary-walking workload makes no union boxes instead of 363,359 and runs 12 to 15% faster. + - Checked-dynamic method calls dispatch on method ids, observed class methods call natively while their slot is unchanged, and calls on bag-tracked class instances skip building their views. +- **Node.js parity:** + - `crypto.getCipherInfo`, AES-CCM, AES key wrap, the legacy `createCipheriv` modes, `checkPrime`/`generatePrime`, and Node's argument checks for `generateKeyPair`. + - Streams: `construct`, `writev`, `decodeStrings` and `defaultEncoding`, including `writev` on `Transform` and `PassThrough`. + - `FinalizationRegistry`, `new.target`, `String.prototype.at` on untyped values, `Error.stackTraceLimit`, `spawnSync`'s `env` option, and class constructors called through `call`/`apply` throw Node's TypeError. + - Records enumerate in the key order their object literal wrote, and objects converting to a union with an empty-record arm keep every field. +- **npm packages:** zod 3 compiles natively and zod 4's core runs, minified packages such as marked compile, chalk 5 and hot-shots 8 build from built-in shims, and `@hono/node-server` 2.x is built in. User code types npm package declarations at a checked, identity-preserving boundary. +- **Builds:** the cached build is reused after whitespace and comment edits, function bodies are emitted on forked workers, and the LLVM emitter reads its analyses from one recorded walk of each module. + + + ## 0.4.2 ### Highlights @@ -159,8 +181,6 @@ All notable changes to scriptc will be documented in this file. - **The same program compiles to the same executable wherever it sits on disk** and whatever order its functions lower in, and edits outside a declaration do not rename the helpers and closures inside it. - **Libraries that use untyped byte arrays** link. - - ## 0.4.1 ### Highlights diff --git a/benchmarks/runtime/README.md b/benchmarks/runtime/README.md index 4f152e4bb..04c90f7bb 100644 --- a/benchmarks/runtime/README.md +++ b/benchmarks/runtime/README.md @@ -13,6 +13,7 @@ Application-shaped workloads for measuring the speed of compiled executables. Ea | `cyclic-graphs` | cpu | object graphs with back references built and dropped: trees with parent and sibling links, rings of records sharing a long-lived registry, closures capturing their owner | | `dictionary-walks` | cpu | `Record` dictionaries walked with for-in and `Object.keys`/`values`/`entries` while entries are added, updated and deleted | | `dynamic-methods` | cpu | untyped JavaScript calling builtin methods on values it does not type: string `slice`/`indexOf`/`toUpperCase`/`padStart`, array `push`/`pop`/`includes`, `Map` and `Set`, and an object's own methods | +| `dynamic-properties` | cpu | untyped JavaScript reading and writing properties of values it does not type: records built by object literals in two key orders, nested records, and instances of a class and its subclass | | `functional-pipeline` | cpu | `filter`/`map`/`reduce`/`flatMap` chains, closures, small records | | `intern-table`, `intern-table-map`, `intern-table-1`, `intern-table-map-1` | cpu | worker threads interning the same type-like objects under string keys and records under number keys, then hits and misses: one shared `InternTable` per kind of key, or a module-level `Map` per thread (the `-map` entries), with 8 threads or 1 (the `-1` entries); under Node.js every thread has its own tables | | `json-records` | cpu | `JSON.stringify`, typed `JSON.parse`, Map aggregation | diff --git a/benchmarks/runtime/workloads/dynamic-properties.json b/benchmarks/runtime/workloads/dynamic-properties.json new file mode 100644 index 000000000..398981ef3 --- /dev/null +++ b/benchmarks/runtime/workloads/dynamic-properties.json @@ -0,0 +1,6 @@ +{ + "entry": "workloads/dynamic-properties.mjs", + "args": [ + "1000000" + ] +} diff --git a/benchmarks/runtime/workloads/dynamic-properties.mjs b/benchmarks/runtime/workloads/dynamic-properties.mjs new file mode 100644 index 000000000..d0ee3802c --- /dev/null +++ b/benchmarks/runtime/workloads/dynamic-properties.mjs @@ -0,0 +1,56 @@ +// Untyped JavaScript reading and writing properties of values it never +// declares types for: records built by object literals in two key orders, +// nested records, property updates, and instances of a class and its +// subclass read and written the same way. Argument: the number of +// iterations. +const rounds = Number(process.argv[2] ?? "1000000"); +class Cursor { + constructor(x, y) { + this.x = x; + this.y = y; + } + step(dx, dy) { + this.x += dx; + this.y += dy; + return this; + } +} +class Marker extends Cursor { + constructor(x, y, label) { + super(x, y); + this.label = label; + } +} +function makeItem(i) { + if (i % 3 === 0) + return { id: i, price: i % 17, qty: i % 5, meta: { tag: i % 7, weight: i % 11 } }; + return { id: i, qty: i % 5, price: i % 13, meta: { weight: i % 9, tag: i % 5 } }; +} +function cost(item) { + return item.price * item.qty + item.meta.weight; +} +function restock(item, round) { + item.qty = (item.qty + round) % 7; + item.meta.tag = (item.meta.tag + 1) % 7; +} +function distance(a, b) { + return Math.abs(a.x - b.x) + Math.abs(a.y - b.y); +} +const items = []; +for (let i = 0; i < 1024; i++) items.push(makeItem(i)); +const cursors = []; +for (let i = 0; i < 64; i++) + cursors.push(i % 2 === 0 ? new Cursor(i - 32, 7 - i) : new Marker(i - 32, 7 - i, "m" + i)); +const origin = { x: 0, y: 0 }; +let checksum = 0; +for (let r = 0; r < rounds; r++) { + const item = items[r & 1023]; + checksum += cost(item); + restock(item, r & 3); + checksum += item.meta.tag; + const cursor = cursors[r & 63]; + checksum += distance(cursor.step((r % 3) - 1, (r % 5) - 2), origin); + if (cursor.x > 100 || cursor.x < -100) cursor.x = 0; + if (cursor.y > 100 || cursor.y < -100) cursor.y = 0; +} +console.log(checksum, items[5].qty, items[6].meta.tag, cursors[3].x, cursors[3].y, cursors[3].label); diff --git a/benchmarks/runtime/workloads/index-cursors.json b/benchmarks/runtime/workloads/index-cursors.json new file mode 100644 index 000000000..24f2a263c --- /dev/null +++ b/benchmarks/runtime/workloads/index-cursors.json @@ -0,0 +1,6 @@ +{ + "entry": "workloads/index-cursors.ts", + "args": [ + "20000" + ] +} diff --git a/benchmarks/runtime/workloads/index-cursors.ts b/benchmarks/runtime/workloads/index-cursors.ts new file mode 100644 index 000000000..b65ded46b --- /dev/null +++ b/benchmarks/runtime/workloads/index-cursors.ts @@ -0,0 +1,25 @@ +// Arrays of indices walked by loop-carried cursors: each step reads the +// next index from the array and relinks the visited entries, so the +// cursor and every stored value are small integers read back from the +// array itself. +const N = 4096; + +function relink(rounds: number): number { + const next: number[] = []; + for (let i = 0; i < N; i++) next.push(i + 1 < N ? i + 1 : 0); + let at = next[0]!; + let total = 0; + for (let r = 0; r < rounds; r++) { + for (let s = 0; s < N; s++) { + const t = next[at]!; + next[at] = next[t]!; + next[t] = at; + at = t; + total = (total + at) | 0; + } + } + return total; +} + +const rounds = Number(process.argv[2] ?? "1"); +console.log("relink", relink(rounds)); diff --git a/docs/content/docs/dependencies.mdx b/docs/content/docs/dependencies.mdx index 409b9cb30..a9c5ae138 100644 --- a/docs/content/docs/dependencies.mdx +++ b/docs/content/docs/dependencies.mdx @@ -53,7 +53,7 @@ Type-only imports follow Node.js's type stripping. `import type` and `export typ scriptc compiles a package automatically when it: - ships its own type declarations, or is typed by its DefinitelyTyped `@types/` package, or is a dependency of another package that has no declarations at all, and -- ships readable JavaScript: unminified, or minified by a bundler whose helpers keep their names. +- ships JavaScript that isn't a webpack bundle. Minified JavaScript compiles like unminified JavaScript. The declarations check your program; the package's JavaScript compiles with the types scriptc infers from it. A program that doesn't typecheck against the inferred types, for example because an `@types/` package describes another release, fails the build at that use. @@ -92,6 +92,13 @@ scriptc implements a few packages itself and compiles that implementation in pla - graceful-fs's retry queue, which fs-extra uses under Node.js, isn't part of the implementation, so a program that runs out of file descriptors fails at the first `EMFILE` error instead of retrying. - An ES module that imports fs-extra can use any of its exports by name. Node.js only exposes the names its CommonJS lexer finds in fs-extra's source, so a name like `readFile` throws a `SyntaxError` there. +`@hono/node-server` 2.x is built in. `serve`, `createAdaptorServer`, `getRequestListener` and `RequestError` serve a fetch handler such as Hono's `app.fetch` over `node:http` with native `Request` and `Response` objects, and write each response the way the package does under Node.js: the same status line, header names, header order and content length, for buffered and streamed bodies, including the package's shortcut for a response that is still exactly what its constructor received. Request bodies stream to the handler as it reads them, and a body the handler leaves unread is drained after the response, up to 64 MiB or 500 ms, as the package does. Known differences: + +- Only HTTP/1.1 servers are declared: the `createServer` and `serverOptions` options for `node:https` and `node:http2`, the `websocket` option and `upgradeWebSocket` aren't declared, so using one is a type error at build time. Subpath modules such as `@hono/node-server/serve-static` compile from the package's own files. +- The fetch handler must return a `Response` or a promise of one; the handler's second argument holds `incoming` and `outgoing`. +- The global `Request` and `Response` stay the built-in classes, so `overrideGlobalObjects` has no effect, and a `Response` reads its default text content type as `text/plain;charset=UTF-8` where the package's replacement reads `text/plain; charset=UTF-8`. Written responses carry the package's spelling. +- A streamed response is written without waiting for `drain`, and a failed handler answers 504 only for an error whose `name` is `TimeoutError`. + Small polyfill packages whose module is exactly a Node.js built-in on Node.js 24 compile as that built-in, at the `require()` that loads them, from your program's CommonJS files, from compiled packages, and from esbuild bundles whose module paths name the package and its version (as pnpm installs do). The package's own files aren't compiled. diff --git a/docs/content/limitations/builtin-apis/crypto.mdx b/docs/content/limitations/builtin-apis/crypto.mdx index 971065408..964ca4ec9 100644 --- a/docs/content/limitations/builtin-apis/crypto.mdx +++ b/docs/content/limitations/builtin-apis/crypto.mdx @@ -8,7 +8,10 @@ `randomFill`, `randomFillSync` (every view kind and ArrayBuffer), `randomInt`, `randomUUID`, and `randomUUIDv7` follow Node 24.22, including callback timing; a `randomInt` callback whose error parameter is typed `Error | null` receives `null` where Node passes `undefined`. Scrypt supports its default parameters, numeric options, aliases, and memory limit. Both KDFs accept strings, Buffers, numeric typed arrays, DataViews, and fixed-length ArrayBuffers. - `createCipheriv` and `createDecipheriv` support aes-128, aes-192, and aes-256 in CBC, CTR, GCM, ECB, CFB, CFB8, CFB1, and OFB modes, des-ede3 in CBC, ECB, CFB, CFB8, CFB1, and OFB modes, des-ede in CBC, ECB, CFB, and OFB modes, chacha20, and chacha20-poly1305, on native targets, through `update()`, `final()`, `setAAD()`, `setAutoPadding()`, `getAuthTag()`, and `setAuthTag()`; other ciphers and wasm32-wasi builds are refused. + `createCipheriv` and `createDecipheriv` support aes-128, aes-192, and aes-256 in CBC, CTR, GCM, CCM, ECB, CFB, CFB8, CFB1, and OFB modes and as key wrap (the wrap and wrap-pad ciphers and their -inv forms), des-ede3 in CBC, ECB, CFB, CFB8, CFB1, and OFB modes, des-ede in CBC, ECB, CFB, and OFB modes, chacha20, and chacha20-poly1305, on native targets, through `update()`, `final()`, `setAAD()`, `setAutoPadding()`, `getAuthTag()`, and `setAuthTag()`; other ciphers and wasm32-wasi builds are refused. + `getCipherInfo` answers Node's info for every cipher OpenSSL names, including the ones without a native implementation. + `getCipherInfo` options whose class defines `keyLength` or `ivLength` throw a coded SC2020 error. + The result is Node's object; the SIV ciphers and the SM4 GCM, CCM, and XTS ciphers have no `nid`, and reading it where the declared `CipherInfo` type gives `number` throws a TypeError. Cipheriv and Decipheriv values are also Transforms: they can be written, piped into, used as `pipeline()` stages, read, iterated with `for await`, and listened to, and their stream properties read as in Node; written chunks go through `update()`, and the end pushes `final()` or reports its error through `'error'`. Through `unknown` and `any` slots and untyped JavaScript, in a program that uses streams, their stream properties and their `on`, `once`, `off`, `pipe`, `unpipe`, `read`, `pause`, `resume`, `setEncoding`, `write`, `end`, `destroy`, `cork`, and `uncork` members work; `write` and `end` take no encoding or callback there, and the other stream members throw a coded SC2020 error. Hash, Hmac, Cipheriv, and Decipheriv streams count a written string's bytes in `writableLength`, where Node counts the string's length. @@ -29,8 +32,12 @@ Candidates and sizes longer than 262144 bits throw a coded SC2020 error, and wasm32-wasi builds are refused. Node calls their callbacks with `undefined` as the error, which a callback whose error parameter is typed `Error | null`, as @types/node declares it, receives as `null`. RSA, EC, and Ed25519 private and public KeyObjects come from `generateKeyPairSync`, `generateKeyPair`, `createPrivateKey`, and `createPublicKey`, on native targets. + `generateKeyPairSync` and `generateKeyPair` check their arguments in Node's order, with Node's errors, including the OpenSSL errors for RSA modulus lengths below 512 or above 2147483647 bits and for public exponents of 0, 1, or an even number, which `generateKeyPair` passes to its callback. + RSA keys generate with even modulus lengths from 1024 to 8192 bits and public exponents up to 2147483647; other lengths and exponents that Node accepts, and `paramEncoding: 'explicit'`, throw a coded SC2020 error. The EC curves are prime256v1 (P-256), secp384r1 (P-384), secp521r1 (P-521), and secp256k1. Keys read and write PEM and DER as PKCS#1, SEC1, PKCS#8, and SubjectPublicKeyInfo, and JWK (Ed25519 keys use PKCS#8, SubjectPublicKeyInfo, and JWK); `createPublicKey` also takes a private key or an X.509 certificate. + EC and Ed25519 keys also read and write the `raw-public` and `raw-private` formats, including compressed EC points. + Raw X25519, X448, Ed448, and post-quantum keys, EC scalars of 0 or at least the curve order, and the EC point at infinity, which Node accepts, throw a coded SC2020 error. They support `type`, `asymmetricKeyType`, `asymmetricKeyDetails`, `export()`, `equals()`, and `instanceof KeyObject`. `crypto.sign`, `crypto.verify`, `createSign`, and `createVerify` support RSA PKCS#1 v1.5 and PSS padding and ECDSA with DER and IEEE P1363 encodings over SHA-1 and SHA-2 digests, and Ed25519 through `crypto.sign` and `crypto.verify` with a `null` or `undefined` algorithm. Ed25519 signing, verification, and private keys link when a string literal in the program names `ed25519` (in any case); without one they throw a coded SC2020 error, while Ed25519 public keys still import and export. diff --git a/docs/content/limitations/builtin-apis/streams-and-readline.mdx b/docs/content/limitations/builtin-apis/streams-and-readline.mdx index fe6bb2dfb..f9d69e3db 100644 --- a/docs/content/limitations/builtin-apis/streams-and-readline.mdx +++ b/docs/content/limitations/builtin-apis/streams-and-readline.mdx @@ -18,7 +18,7 @@ Stages may take the `{ signal }` argument. With a function destination, `pipeline()` returns an object-mode PassThrough that receives an async generator destination's values, or an async function destination's value unless it is `null` or `undefined`. `@types/node` types that result as `NodeJS.WritableStream`, so `for await` over it needs the `PassThrough` type (`pipeline(...) as PassThrough`, or a JSDoc `@type`). - In typed code, the callback of a pipeline with an async function destination receives `null` instead of `undefined` when there is no error. + In typed code, a pipeline callback with an async function or async generator destination receives `null` for no error when its error parameter cannot hold `undefined` (@types/node declares it `NodeJS.ErrnoException | null`), and `undefined` when it can, as in Node. `Readable.from` accepts `objectMode` and `highWaterMark` options. The `Readable` operators (`map`, `filter`, `forEach`, `toArray`, `some`, `find`, `every`, `flatMap`, `drop`, `take`, and `reduce`) accept `concurrency`, `highWaterMark`, and `signal` options. Their callbacks may take the `{ signal }` argument, `map` may return any value, and `flatMap` may return any iterable, async iterable, or stream. @@ -27,14 +27,16 @@ A `Duplex.from` body function may take the `{ signal }` argument, which is aborted with the error the Duplex is destroyed with. `stream.compose` accepts streams and async generator functions in any position, an async function as the last stage, and an array, a string, or a generator as the first stage; one stage is `Duplex.from` of it. A web `ReadableStream` is read as an async iterable by `Readable.from` and as the first stage of `compose` or `pipeline`, and through its reader by `Readable.fromWeb` (with `objectMode`, `highWaterMark`, `encoding`, and `signal` in an options object literal) and `Duplex.from`. - A web stream in a later compose stage, `Readable.toWeb`, and web `WritableStream` and `TransformStream` interop are unsupported. + `new TransformStream()` without arguments builds the identity transform: its readable side is a web `ReadableStream`, its writable side supports `getWriter`, `abort`, `close`, and `locked`, and the writer supports `write`, `close`, `abort`, `releaseLock`, `ready`, `closed`, and `desiredSize`, with Node's backpressure (a write completes when the readable side reads its chunk). + A web stream in a later compose stage, `Readable.toWeb`, a `TransformStream` with a transformer or queuing strategies, the `WritableStream` constructor, `pipeTo`, and `pipeThrough` are unsupported. The `ERR_INVALID_ARG_VALUE` message for a stream that is no longer readable or writable names the stream's class instead of printing the stream. `readable.compose(stage, { signal })` is `stream.compose(readable, stage)` with the signal added; the options are an object literal whose `signal` is a variable or a property read. `pipeline` from `node:stream/promises` accepts a trailing `{ signal, end }` object, with or without function stages, and `finished` accepts `readable`, `writable`, `error`, `cleanup`, and `signal` options, with boolean literals for the flags. Stream constructors accept a `signal` option in an options object literal, and `stream.addAbortSignal` accepts any stream. - The `construct` option and `_construct` method, `decodeStrings`, and `defaultEncoding` work on every stream class, and the `writev` option and `_writev` method on Writable and Duplex; `writev` on a Transform or PassThrough, assigning `_construct` after construction, and a non-literal `decodeStrings` or `defaultEncoding` in an options object literal are unsupported. + The `construct` option and `_construct` method, `decodeStrings`, and `defaultEncoding` work on every stream class, and the `writev` option and `_writev` method on every class with a writable side. + A `decodeStrings` or `defaultEncoding` value known only at run time is unsupported after an option member whose value has effects (a call or an assignment). + In typed code, a `writev` entry's `encoding` declared as a string receives `""` where Node passes `undefined` or `null` (an object-mode write). `setDefaultEncoding()` takes any encoding Node knows. - A `transform` or `flush` option callback whose completion callback types `data` as `any`, as @types/node's `TransformCallback` does, may pass a Buffer, a string, or nothing; any other value throws a TypeError where the callback is called, instead of Node's `ERR_INVALID_ARG_TYPE` stream error. `node:readline` and `node:readline/promises` (also reached as `readline.promises`) interfaces, from `createInterface(options)` or `createInterface(input, output, completer, terminal)`, read `process.stdin` or a readable stream: `question`, `prompt`, `write` (with a key object literal of `name`, `ctrl`, `meta`, and `shift`), `pause`, `resume`, `'line'`/`'close'`/`'pause'`/`'resume'`/`'error'`/`'history'`/`'SIGINT'`/`'SIGTSTP'`/`'SIGCONT'` listeners, and `for await` compile. `on`, `once`, and `addListener` return the interface, so registrations chain. The `history`, `historySize`, `removeHistoryDuplicates`, `completer`, `tabSize`, and `escapeCodeTimeout` options are validated as in Node. @@ -42,7 +44,7 @@ The output's `'resize'` event does not redraw the line. The cursor never stops between the two UTF-16 units of a supplementary character: where Node's Up or Down movement through a multiline line would, the cursor moves to the character's start or end. In JavaScript, a `'history'` listener receives a copy of the history; its changes are written back when the listener returns, but the copy is not `rl.history` itself. - `rl.history`, `historySize`, `historyIndex`, `crlfDelay`, `tabSize`, `escapeCodeTimeout`, `line`, `cursor`, and `terminal` read as in Node, except that `rl.cursor` of an interface that is not a terminal throws instead of answering `undefined`. + `rl.history`, `historySize`, `historyIndex`, `crlfDelay`, `tabSize`, `escapeCodeTimeout`, `line`, `cursor`, and `terminal` read as in Node; `rl.cursor` is `undefined` on an interface that is not a terminal. `rl[Symbol.dispose]()` closes the interface, and `instanceof` with `readline.Interface` or `readlinePromises.Interface` answers whether the interface came from that module, as in Node. In JavaScript, the input and output can be any expression that evaluates to a stream, such as a function's result or an object's member; a value that is not a stream is unsupported. In JavaScript, `question()` and `setPrompt()` accept a value that is not a string: when such a prompt is written, an interface that is not a terminal throws `ERR_INVALID_ARG_TYPE` as Node does (the `readline/promises` question rejects with it) and the question stays pending, and a terminal interface writes `String(value)`. diff --git a/docs/content/limitations/builtin-apis/text-encoding.mdx b/docs/content/limitations/builtin-apis/text-encoding.mdx index bffd24424..e20cfd943 100644 --- a/docs/content/limitations/builtin-apis/text-encoding.mdx +++ b/docs/content/limitations/builtin-apis/text-encoding.mdx @@ -1,6 +1,7 @@ - **Text encoding:** `TextEncoder` and `TextDecoder` can be stored in fields, arguments, return values, arrays, and closures. Encoding supports UTF-8, including `encodeInto` into a Uint8Array or Buffer view. Codec properties expose the canonical encoding and decoder flags. + Codecs held in `any` or `unknown` storage keep their identity and streaming state, and dynamic code can call `encode`, `encodeInto` and `decode`, with options objects, and read their properties. Decoding accepts recognized WHATWG labels, including runtime labels and omitted or undefined encodings. Literal constructor options support `fatal` and `ignoreBOM`; literal decode options support streaming UTF-8, UTF-16, and single-byte encodings. Other legacy streaming encodings, runtime options objects, inspection, JSON conversion, and module-scope declarations using `var` or lacking an initializer are unsupported. diff --git a/docs/package.json b/docs/package.json index 3c2a950fd..51d74d0a1 100644 --- a/docs/package.json +++ b/docs/package.json @@ -10,9 +10,10 @@ "build": "fumadocs-mdx && next build", "start": "next start", "typecheck": "fumadocs-mdx && next typegen && tsc --noEmit", + "test:unit": "node --test test/compatibility-matrix.test.mjs", "test:routes": "node scripts/test-routes.mjs", "compat:check": "pnpm --dir .. node-compat:check", - "check": "pnpm compat:check && pnpm typecheck && pnpm build && pnpm test:routes" + "check": "pnpm compat:check && pnpm typecheck && pnpm test:unit && pnpm build && pnpm test:routes" }, "dependencies": { "@vercel/geistdocs": "2.7.8", diff --git a/docs/src/components/node-compatibility-matrix.tsx b/docs/src/components/node-compatibility-matrix.tsx index 095bcdaf2..af2435668 100644 --- a/docs/src/components/node-compatibility-matrix.tsx +++ b/docs/src/components/node-compatibility-matrix.tsx @@ -4,52 +4,28 @@ // empty after an edit. // @refresh reset -import { useEffect, useMemo, useState } from "react"; +import { memo, startTransition, useCallback, useDeferredValue, useEffect, useId, useMemo, useState, type FocusEvent, type KeyboardEvent, type ReactNode } from "react"; +import { + buildTree, + expandableIds, + filterTree, + flattenTree, + nodeCount, + rowMatcher, + searchTexts, + statusCounts, + statuses, + type NodeStability, + type Row, + type Snapshot, + type Status, + type Verification, +} from "@/lib/compatibility-tree"; // Set by next.config.mjs from the committed artifact. const artifactVersion = process.env.COMPATIBILITY_ARTIFACT_VERSION; const rowCount = Number(process.env.COMPATIBILITY_ROW_COUNT); -type Status = - | "supported" - | "partial" - | "refused" - | "not-implemented" - | "by-design" - | "unreviewed" - | "not-applicable"; -type VerificationBasis = - | "test-backed" - | "explicit-refusal" - | "verified-absence" - | "declared-gap" - | "registry-gap" - | "architectural-policy" - | "unreviewed" - | "not-applicable" - | "derived"; -type Verification = { label: string; detail: string }; -type Tier = { status: Status; detail: string; verification: VerificationBasis }; -type NodeStability = { index: string; level: number; text: string; inherited: boolean } | null; -type Row = { - id: string; - chapter: string; - kind: string; - name: string; - signature: string; - label: string; - apiSymbol: string; - depth: number; - scope: "api" | "documentation" | "metadata" | "configuration"; - anchor: string; - nodeStability: NodeStability; - static: Tier; -}; -type Chapter = { slug: string; title: string; entries: number; apiEntries: number }; -type Snapshot = { nodeVersion: string; verificationBases: Record; chapters: Chapter[]; rows: Row[] }; -type TreeNode = { row: Row; children: TreeNode[] }; -type VisibleNode = { node: TreeNode; level: number }; - const labels: Record = { supported: "Supported", partial: "Partial", @@ -70,7 +46,6 @@ const styles: Record = { "not-applicable": "border-gray-alpha-400 bg-background-200 text-gray-700", }; -const statuses = ["supported", "partial", "not-implemented", "refused", "by-design", "unreviewed", "not-applicable"] as const; const stabilityStyles: Record = { 0: "border-red-500/40 bg-red-100 text-red-900 dark:bg-red-200", 1: "border-amber-500/50 bg-amber-100 text-amber-900 dark:bg-amber-200", @@ -78,114 +53,83 @@ const stabilityStyles: Record = { 3: "border-gray-alpha-500 bg-gray-alpha-100 text-gray-900", }; -function StatusBadge({ status, detail, verification }: { status: Status; detail?: string; verification?: Verification }) { - const tooltip = verification ? `${detail ?? ""}${detail ? "\n\n" : ""}Verification: ${verification.label}. ${verification.detail}` : detail; +/** A badge's tooltip, mounted only while the badge is hovered or has + * keyboard focus (the :focus-visible rule the CSS-only tooltip used), and + * named by aria-describedby while open. Hover and focus are tracked apart, + * so moving the pointer away keeps a keyboard-focused badge's tooltip. + * Escape dismisses it until the next hover or focus. */ +function BadgeTooltip({ tooltipClass, children, badge }: { tooltipClass: string; children: ReactNode; badge: ReactNode }) { + const [hovered, setHovered] = useState(false); + const [focused, setFocused] = useState(false); + const open = hovered || focused; + const id = useId(); + const enter = useCallback(() => setHovered(true), []); + const leave = useCallback(() => setHovered(false), []); + const focus = useCallback((event: FocusEvent) => { + if (event.currentTarget.matches(":focus-visible")) setFocused(true); + }, []); + const blur = useCallback(() => setFocused(false), []); + const key = useCallback((event: KeyboardEvent) => { + if (event.key !== "Escape") return; + setHovered(false); + setFocused(false); + }, []); return ( - - - {labels[status]} - - {tooltip ? ( - - {detail ? {detail} : null} - {verification ? Verification: {verification.label}. {verification.detail} : null} + + {badge} + {open ? ( + + {children} ) : null} ); } -function StabilityBadge({ stability }: { stability: NodeStability }) { +export const StatusBadge = memo(function StatusBadge({ status, detail, verification }: { status: Status; detail?: string; verification?: Verification }) { + const tooltip = verification ? `${detail ?? ""}${detail ? "\n\n" : ""}Verification: ${verification.label}. ${verification.detail}` : detail; + const badge = ( + + {labels[status]} + + ); + if (!tooltip) return {badge}; + return ( + + {detail ? {detail} : null} + {verification ? Verification: {verification.label}. {verification.detail} : null} + + ); +}); + +export const StabilityBadge = memo(function StabilityBadge({ stability }: { stability: NodeStability }) { if (stability === null) return —; const detail = `Node.js Stability ${stability.index}: ${stability.text}${stability.inherited ? " (inherited from the nearest parent section)" : ""}`; return ( - - - {stability.index} - - - {detail} - - - ); -} - -function statusCounts(rows: Row[], tier: "static") { - return rows.reduce>( - (counts, row) => { - if (row.scope !== "api" || row.depth === 0) return counts; - counts[row[tier].status] += 1; - return counts; - }, - { - supported: 0, - partial: 0, - refused: 0, - "not-implemented": 0, - "by-design": 0, - unreviewed: 0, - "not-applicable": 0, - }, - ); -} - -/** Node's all.json is preorder with an explicit heading depth. */ -function buildTree(rows: Row[]): TreeNode[] { - const roots: TreeNode[] = []; - const stack: TreeNode[] = []; - for (const row of rows) { - const node: TreeNode = { row, children: [] }; - while (stack.length > row.depth) stack.pop(); - const parent = stack[row.depth - 1]; - if (row.depth === 0 || parent === undefined) roots.push(node); - else parent.children.push(node); - stack[row.depth] = node; - stack.length = row.depth + 1; - } - return roots; -} - -function filterTree(nodes: TreeNode[], matches: (row: Row) => boolean): TreeNode[] { - const filtered: TreeNode[] = []; - for (const node of nodes) { - const children = filterTree(node.children, matches); - if (matches(node.row) || children.length > 0) filtered.push({ row: node.row, children }); - } - return filtered; -} - -function flattenTree(nodes: TreeNode[], expanded: Set, forceExpanded: boolean): VisibleNode[] { - const visible: VisibleNode[] = []; - const visit = (items: TreeNode[], level: number) => { - for (const node of items) { - visible.push({ node, level }); - if (node.children.length > 0 && (forceExpanded || expanded.has(node.row.id))) { - visit(node.children, level + 1); + + {stability.index} + } - } - }; - visit(nodes, 1); - return visible; -} - -function nodeCount(nodes: TreeNode[]): number { - return nodes.reduce((count, node) => count + 1 + nodeCount(node.children), 0); -} - -function expandableIds(nodes: TreeNode[]): Set { - const ids = new Set(); - const visit = (items: TreeNode[]) => { - for (const node of items) { - if (node.children.length > 0) ids.add(node.row.id); - visit(node.children); - } - }; - visit(nodes); - return ids; -} + > + {detail} + + ); +}); function Chevron({ open }: { open: boolean }) { return ( @@ -195,6 +139,47 @@ function Chevron({ open }: { open: boolean }) { ); } +type MatrixRowProps = { + row: Row; + level: number; + hasChildren: boolean; + open: boolean; + filtering: boolean; + displayName: string; + href: string; + verification: Verification | undefined; + onToggle: (id: string) => void; +}; + +/** One table row. Memoized on its own values, so a filter or expansion + * change re-renders only the rows whose values changed. */ +const MatrixRow = memo(function MatrixRow({ row, level, hasChildren, open, filtering, displayName, href, verification, onToggle }: MatrixRowProps) { + const isRoot = level === 1; + return ( + + + + + + ); +}); + export function NodeCompatibilityMatrix() { const [snapshot, setSnapshot] = useState(null); const [loadError, setLoadError] = useState(false); @@ -202,15 +187,16 @@ export function NodeCompatibilityMatrix() { const [staticFilter, setStaticFilter] = useState<"all" | Status>("all"); const [expanded, setExpanded] = useState>(() => new Set()); - const rows = snapshot?.rows ?? []; - const chapters = snapshot?.chapters ?? []; + const rows = useMemo(() => snapshot?.rows ?? [], [snapshot]); + const chapters = useMemo(() => snapshot?.chapters ?? [], [snapshot]); const verificationBases = snapshot?.verificationBases; const apiRowCount = useMemo(() => rows.filter((row) => row.scope === "api").length, [rows]); const chapterTitles = useMemo(() => new Map(chapters.map((item) => [item.slug, item.title])), [chapters]); const base = `https://nodejs.org/docs/v${snapshot?.nodeVersion ?? "24.22.0"}/api/`; - const staticCounts = useMemo(() => statusCounts(rows, "static"), [rows]); + const staticCounts = useMemo(() => statusCounts(rows), [rows]); const visibleStatuses = useMemo(() => statuses.filter((status) => staticCounts[status] > 0), [staticCounts]); const tree = useMemo(() => buildTree(rows), [rows]); + const texts = useMemo(() => searchTexts(rows, chapterTitles), [rows, chapterTitles]); useEffect(() => { const controller = new AbortController(); @@ -231,36 +217,23 @@ export function NodeCompatibilityMatrix() { return () => controller.abort(); }, []); - const needle = query.trim().toLocaleLowerCase(); - const hasResultFilter = needle !== "" || staticFilter !== "all"; - const rowMatches = (row: Row) => { - if (staticFilter !== "all" && row.static.status !== staticFilter) return false; - if (!needle) return true; - return [chapterTitles.get(row.chapter), row.kind, row.name, row.signature, row.apiSymbol, row.static.detail] - .join(" ") - .toLocaleLowerCase() - .includes(needle); - }; + // The search field and the status select update at once; the table + // follows the deferred values, so input stays responsive while the rows + // re-render. + const deferredQuery = useDeferredValue(query); + const deferredStatic = useDeferredValue(staticFilter); + const needle = deferredQuery.trim().toLocaleLowerCase(); + const hasResultFilter = needle !== "" || deferredStatic !== "all"; - const resultTree = useMemo( - () => hasResultFilter ? filterTree(tree, rowMatches) : tree, - // rowMatches closes over these filter values. - // eslint-disable-next-line react-hooks/exhaustive-deps - [hasResultFilter, needle, staticFilter, tree], + const result = useMemo( + () => hasResultFilter ? filterTree(tree, rowMatcher(texts, needle, deferredStatic)) : { nodes: tree, matched: nodeCount(tree) }, + [hasResultFilter, needle, deferredStatic, texts, tree], ); const visible = useMemo( - () => flattenTree(resultTree, expanded, hasResultFilter), - [expanded, hasResultFilter, resultTree], - ); - const matchedCount = useMemo( - () => hasResultFilter ? tree.reduce((count, node) => { - const visit = (item: TreeNode): number => (rowMatches(item.row) ? 1 : 0) + item.children.reduce((sum, child) => sum + visit(child), 0); - return count + visit(node); - }, 0) : nodeCount(tree), - // rowMatches closes over these filter values. - // eslint-disable-next-line react-hooks/exhaustive-deps - [hasResultFilter, needle, staticFilter, tree], + () => flattenTree(result.nodes, expanded, hasResultFilter), + [expanded, hasResultFilter, result], ); + const matchedCount = result.matched; const changeQuery = (value: string) => setQuery(value); const resetFilters = () => { @@ -268,12 +241,12 @@ export function NodeCompatibilityMatrix() { setStaticFilter("all"); setExpanded(new Set()); }; - const toggle = (id: string) => setExpanded((current) => { + const toggle = useCallback((id: string) => setExpanded((current) => { const next = new Set(current); if (next.has(id)) next.delete(id); else next.add(id); return next; - }); + }), []); useEffect(() => { if (!snapshot) return; @@ -316,8 +289,8 @@ export function NodeCompatibilityMatrix() { {visible.length.toLocaleString()} visible · {matchedCount.toLocaleString()} {hasResultFilter ? "matching" : "API"} rows {!hasResultFilter ? ( - - + + ) : null} @@ -335,33 +308,19 @@ export function NodeCompatibilityMatrix() { {visible.map(({ node, level }) => { const row = node.row; - const hasChildren = node.children.length > 0; - const open = hasResultFilter || expanded.has(row.id); - const href = `${base}${row.chapter}.html${row.anchor ? `#${row.anchor}` : ""}`; - const isRoot = level === 1; - const displayName = isRoot ? chapterTitles.get(row.chapter) ?? row.label : row.label; return ( - - - - - + 0} + open={hasResultFilter || expanded.has(row.id)} + filtering={hasResultFilter} + displayName={level === 1 ? chapterTitles.get(row.chapter) ?? row.label : row.label} + href={`${base}${row.chapter}.html${row.anchor ? `#${row.anchor}` : ""}`} + verification={verificationBases?.[row.static.verification]} + onToggle={toggle} + /> ); })} diff --git a/docs/src/generated/node-v24-compatibility.json b/docs/src/generated/node-v24-compatibility.json index c341da852..fc5826666 100644 --- a/docs/src/generated/node-v24-compatibility.json +++ b/docs/src/generated/node-v24-compatibility.json @@ -11660,9 +11660,9 @@ "inherited": true }, "static": { - "status": "refused", - "detail": "Explicitly rejected by a named compiler diagnostic.", - "verification": "explicit-refusal" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { @@ -35971,9 +35971,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the documented scriptc module-loader subset.", + "verification": "test-backed" } }, { @@ -81657,9 +81657,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the documented static async-iteration subset.", + "verification": "test-backed" } }, { diff --git a/docs/src/lib/compatibility-tree.ts b/docs/src/lib/compatibility-tree.ts new file mode 100644 index 000000000..4044bbc8d --- /dev/null +++ b/docs/src/lib/compatibility-tree.ts @@ -0,0 +1,157 @@ +// The compatibility matrix's data model: the ledger rows as a tree, and the +// search and filter passes over it. Kept free of React so the passes can be +// tested directly. + +export type Status = + | "supported" + | "partial" + | "refused" + | "not-implemented" + | "by-design" + | "unreviewed" + | "not-applicable"; +export type VerificationBasis = + | "test-backed" + | "explicit-refusal" + | "verified-absence" + | "declared-gap" + | "registry-gap" + | "architectural-policy" + | "unreviewed" + | "not-applicable" + | "derived"; +export type Verification = { label: string; detail: string }; +export type Tier = { status: Status; detail: string; verification: VerificationBasis }; +export type NodeStability = { index: string; level: number; text: string; inherited: boolean } | null; +export type Row = { + id: string; + chapter: string; + kind: string; + name: string; + signature: string; + label: string; + apiSymbol: string; + depth: number; + scope: "api" | "documentation" | "metadata" | "configuration"; + anchor: string; + nodeStability: NodeStability; + static: Tier; +}; +export type Chapter = { slug: string; title: string; entries: number; apiEntries: number }; +export type Snapshot = { + nodeVersion: string; + verificationBases: Record; + chapters: Chapter[]; + rows: Row[]; +}; +export type TreeNode = { row: Row; children: TreeNode[] }; +export type VisibleNode = { node: TreeNode; level: number }; + +export const statuses = ["supported", "partial", "not-implemented", "refused", "by-design", "unreviewed", "not-applicable"] as const; + +export function statusCounts(rows: Row[]): Record { + return rows.reduce>( + (counts, row) => { + if (row.scope !== "api" || row.depth === 0) return counts; + counts[row.static.status] += 1; + return counts; + }, + { + supported: 0, + partial: 0, + refused: 0, + "not-implemented": 0, + "by-design": 0, + unreviewed: 0, + "not-applicable": 0, + }, + ); +} + +/** Node's all.json is preorder with an explicit heading depth. */ +export function buildTree(rows: Row[]): TreeNode[] { + const roots: TreeNode[] = []; + const stack: TreeNode[] = []; + for (const row of rows) { + const node: TreeNode = { row, children: [] }; + while (stack.length > row.depth) stack.pop(); + const parent = stack[row.depth - 1]; + if (row.depth === 0 || parent === undefined) roots.push(node); + else parent.children.push(node); + stack[row.depth] = node; + stack.length = row.depth + 1; + } + return roots; +} + +/** The lowercase text a search matches against, built once per row: the + * chapter title, kind, name, signature, API symbol and support detail. */ +export function searchTexts(rows: Row[], chapterTitles: ReadonlyMap): Map { + return new Map( + rows.map((row) => [ + row.id, + [chapterTitles.get(row.chapter), row.kind, row.name, row.signature, row.apiSymbol, row.static.detail] + .join(" ") + .toLocaleLowerCase(), + ]), + ); +} + +/** The rows a status filter and a lowercase search needle keep. */ +export function rowMatcher( + texts: ReadonlyMap, + needle: string, + status: "all" | Status, +): (row: Row) => boolean { + return (row) => { + if (status !== "all" && row.static.status !== status) return false; + return needle === "" || (texts.get(row.id) ?? "").includes(needle); + }; +} + +/** The matching rows with their ancestors, and the number of matching + * rows, in one pass. */ +export function filterTree(nodes: TreeNode[], matches: (row: Row) => boolean): { nodes: TreeNode[]; matched: number } { + let matched = 0; + const visit = (items: TreeNode[]): TreeNode[] => { + const filtered: TreeNode[] = []; + for (const node of items) { + const children = visit(node.children); + const own = matches(node.row); + if (own) matched += 1; + if (own || children.length > 0) filtered.push({ row: node.row, children }); + } + return filtered; + }; + return { nodes: visit(nodes), matched }; +} + +export function flattenTree(nodes: TreeNode[], expanded: ReadonlySet, forceExpanded: boolean): VisibleNode[] { + const visible: VisibleNode[] = []; + const visit = (items: TreeNode[], level: number) => { + for (const node of items) { + visible.push({ node, level }); + if (node.children.length > 0 && (forceExpanded || expanded.has(node.row.id))) { + visit(node.children, level + 1); + } + } + }; + visit(nodes, 1); + return visible; +} + +export function nodeCount(nodes: TreeNode[]): number { + return nodes.reduce((count, node) => count + 1 + nodeCount(node.children), 0); +} + +export function expandableIds(nodes: TreeNode[]): Set { + const ids = new Set(); + const visit = (items: TreeNode[]) => { + for (const node of items) { + if (node.children.length > 0) ids.add(node.row.id); + visit(node.children); + } + }; + visit(nodes); + return ids; +} diff --git a/docs/test/compatibility-matrix.test.mjs b/docs/test/compatibility-matrix.test.mjs new file mode 100644 index 000000000..abd3df945 --- /dev/null +++ b/docs/test/compatibility-matrix.test.mjs @@ -0,0 +1,115 @@ +import assert from "node:assert/strict"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { test } from "node:test"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { createElement } from "react"; +import { renderToStaticMarkup } from "react-dom/server"; +import ts from "typescript"; +import { + buildTree, + expandableIds, + filterTree, + flattenTree, + nodeCount, + rowMatcher, + searchTexts, + statusCounts, +} from "../src/lib/compatibility-tree.ts"; + +const row = (id, depth, label, status, detail = "") => ({ + id, + chapter: "zlib", + kind: depth === 0 ? "module" : "method", + name: label, + signature: label, + label, + apiSymbol: label, + depth, + scope: "api", + anchor: "", + nodeStability: null, + static: { status, detail, verification: "test-backed" }, +}); +const rows = [ + row("root", 0, "zlib", "partial"), + row("a", 1, "zlib.gzipSync()", "supported", "Lowered with Node's options"), + row("b", 1, "zlib.Gzip", "partial"), + row("b1", 2, "gzip.flush()", "supported"), + row("c", 1, "zlib.zstdCompress()", "not-implemented"), +]; +const titles = new Map([["zlib", "Zlib"]]); + +test("the tree follows the rows' heading depths", () => { + const tree = buildTree(rows); + assert.equal(tree.length, 1); + assert.deepEqual(tree[0].children.map((node) => node.row.id), ["a", "b", "c"]); + assert.deepEqual(tree[0].children[1].children.map((node) => node.row.id), ["b1"]); + assert.equal(nodeCount(tree), 5); + assert.deepEqual([...expandableIds(tree)].sort(), ["b", "root"]); +}); + +test("filtering keeps matching rows with their ancestors and counts the matches once", () => { + const tree = buildTree(rows); + const texts = searchTexts(rows, titles); + const byText = filterTree(tree, rowMatcher(texts, "flush", "all")); + assert.equal(byText.matched, 1); + assert.deepEqual(flattenTree(byText.nodes, new Set(), true).map(({ node, level }) => `${node.row.id}@${level}`), ["root@1", "b@2", "b1@3"]); + const byDetail = filterTree(tree, rowMatcher(texts, "node's options", "all")); + assert.deepEqual(flattenTree(byDetail.nodes, new Set(), true).map(({ node }) => node.row.id), ["root", "a"]); + const byStatus = filterTree(tree, rowMatcher(texts, "", "supported")); + assert.equal(byStatus.matched, 2); + const both = filterTree(tree, rowMatcher(texts, "gzip", "partial")); + assert.equal(both.matched, 1); + assert.equal(filterTree(tree, rowMatcher(texts, "brotli", "all")).nodes.length, 0); +}); + +test("collapsed rows hide their children until expanded", () => { + const tree = buildTree(rows); + assert.deepEqual(flattenTree(tree, new Set(), false).map(({ node }) => node.row.id), ["root"]); + assert.deepEqual(flattenTree(tree, new Set(["root"]), false).map(({ node }) => node.row.id), ["root", "a", "b", "c"]); + assert.deepEqual(flattenTree(tree, new Set(["root", "b"]), false).map(({ node }) => node.row.id), ["root", "a", "b", "b1", "c"]); +}); + +test("status counts cover API rows below the chapters", () => { + const counts = statusCounts(rows); + assert.equal(counts.supported, 2); + assert.equal(counts.partial, 1); + assert.equal(counts["not-implemented"], 1); +}); + +/** The matrix component, compiled for Node from its TSX source. */ +async function loadMatrix() { + const docs = new URL("../", import.meta.url); + const source = await readFile(new URL("src/components/node-compatibility-matrix.tsx", docs), "utf8"); + const { outputText } = ts.transpileModule(source, { + compilerOptions: { jsx: ts.JsxEmit.ReactJSX, module: ts.ModuleKind.ESNext, target: ts.ScriptTarget.ES2022 }, + }); + const lib = pathToFileURL(fileURLToPath(new URL("src/lib/compatibility-tree.ts", docs))).href; + const dir = new URL("node_modules/.cache/compatibility-matrix-test/", docs); + await mkdir(dir, { recursive: true }); + const file = new URL("node-compatibility-matrix.mjs", dir); + await writeFile(file, outputText.replace('"@/lib/compatibility-tree"', JSON.stringify(lib))); + return import(file.href); +} + +test("badges render without their tooltips until hovered or focused", async () => { + const { StatusBadge, StabilityBadge } = await loadMatrix(); + const status = renderToStaticMarkup( + createElement(StatusBadge, { + status: "partial", + detail: "Lowered with Node's options", + verification: { label: "Test-backed", detail: "Corpus programs cover it." }, + }), + ); + assert.ok(status.includes('tabindex="0"'), status); + assert.ok(status.includes('title="Lowered with Node's options'), status); + assert.ok(!status.includes('role="tooltip"'), status); + assert.ok(!status.includes("aria-describedby"), status); + const stability = renderToStaticMarkup( + createElement(StabilityBadge, { stability: { index: "2", level: 2, text: "Stable", inherited: false } }), + ); + assert.ok(stability.includes('tabindex="0"'), stability); + assert.ok(!stability.includes('role="tooltip"'), stability); + const legend = renderToStaticMarkup(createElement(StatusBadge, { status: "supported" })); + assert.ok(!legend.includes("tabindex"), legend); +}); diff --git a/internal/compatibility/generated/node-v24-internal.json b/internal/compatibility/generated/node-v24-internal.json index 9393fcaa8..ccf2c219a 100644 --- a/internal/compatibility/generated/node-v24-internal.json +++ b/internal/compatibility/generated/node-v24-internal.json @@ -19332,6 +19332,10 @@ "tests/corpus/7151-cipher-legacy-errors.cjs", "tests/corpus/7152-crypto-prime.ts", "tests/corpus/7153-crypto-prime-errors.cjs", + "tests/corpus/7154-crypto-cipher-info.cjs", + "tests/corpus/7155-cipher-aes-ccm.cjs", + "tests/corpus/7156-cipher-key-wrap.cjs", + "tests/corpus/7157-crypto-cipher-info-typed.ts", "tests/corpus/node24-crypto-kdf-errors.ts", "tests/corpus/node24-crypto-kdf-options.ts", "tests/corpus/node24-crypto-kdf.ts", @@ -19719,6 +19723,8 @@ "status": "partial", "evidence": "surface-manifest:node-builtin.crypto.createCipheriv", "tests": [ + "tests/corpus/7155-cipher-aes-ccm.cjs", + "tests/corpus/7156-cipher-key-wrap.cjs", "tests/corpus/7150-cipher-legacy-modes.ts", "tests/corpus/7151-cipher-legacy-errors.cjs", "tests/corpus/5250-cipher-aes-gcm.ts", @@ -19760,6 +19766,8 @@ "status": "partial", "evidence": "surface-manifest:node-builtin.crypto.createDecipheriv", "tests": [ + "tests/corpus/7155-cipher-aes-ccm.cjs", + "tests/corpus/7156-cipher-key-wrap.cjs", "tests/corpus/7150-cipher-legacy-modes.ts", "tests/corpus/7151-cipher-legacy-errors.cjs", "tests/corpus/5250-cipher-aes-gcm.ts", @@ -20434,8 +20442,12 @@ "inherited": true }, "static": { - "status": "refused", - "evidence": "surface-manifest:unsupported:node-builtin.crypto.getCipherInfo" + "status": "partial", + "evidence": "surface-manifest:node-builtin.crypto.getCipherInfo", + "tests": [ + "tests/corpus/7154-crypto-cipher-info.cjs", + "tests/corpus/7157-crypto-cipher-info-typed.ts" + ] }, "anchorSource": "exact" }, @@ -52182,6 +52194,8 @@ "tests/corpus/6294-webcrypto-js-values.mjs", "tests/corpus/6295-jose-hs256/main.mjs", "tests/corpus/6331-jose-root-entry/main.ts", + "tests/corpus/7743-transform-stream-identity.ts", + "tests/corpus/7747-text-codec-untyped-slot.js", "tests/corpus/text-codec-values/main.ts", "tests/corpus/webassembly-api.mjs", "tests/corpus/webassembly-dylink/main.mjs", @@ -53334,7 +53348,8 @@ "tests": [ "tests/corpus/1423-text-codec.ts", "tests/corpus/text-codec-values/main.ts", - "tests/corpus/3133-text-decoder-streaming.ts" + "tests/corpus/3133-text-decoder-streaming.ts", + "tests/corpus/7747-text-codec-untyped-slot.js" ] }, "anchorSource": "exact" @@ -53385,7 +53400,8 @@ "tests": [ "tests/corpus/1423-text-codec.ts", "tests/corpus/text-codec-values/main.ts", - "tests/corpus/3133-text-decoder-streaming.ts" + "tests/corpus/3133-text-decoder-streaming.ts", + "tests/corpus/7747-text-codec-untyped-slot.js" ] }, "anchorSource": "exact" @@ -53431,8 +53447,11 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:globals" + "status": "partial", + "evidence": "compiler-feature:globals.TransformStream", + "tests": [ + "tests/corpus/7743-transform-stream-identity.ts" + ] }, "anchorSource": "exact" }, @@ -101265,6 +101284,7 @@ "tests/corpus/5835-util-format-integer-specifiers.ts", "tests/corpus/5884-inspect-layout-options.ts", "tests/corpus/6428-promisify-callback-functions.ts", + "tests/corpus/7747-text-codec-untyped-slot.js", "tests/corpus/node24-deep-equality-properties.js", "tests/corpus/node24-deep-equality.cjs", "tests/corpus/node24-deep-equality.ts", @@ -101793,7 +101813,8 @@ "tests/corpus/text-codec-values/main.ts", "tests/fixtures/node-types/text-codecs.mts", "tests/harness/project-config.test.ts", - "tests/corpus/3133-text-decoder-streaming.ts" + "tests/corpus/3133-text-decoder-streaming.ts", + "tests/corpus/7747-text-codec-untyped-slot.js" ] }, "anchorSource": "exact" @@ -101914,7 +101935,8 @@ "tests/corpus/text-codec-values/main.ts", "tests/fixtures/node-types/text-codecs.mts", "tests/harness/project-config.test.ts", - "tests/corpus/3133-text-decoder-streaming.ts" + "tests/corpus/3133-text-decoder-streaming.ts", + "tests/corpus/7747-text-codec-untyped-slot.js" ] }, "anchorSource": "exact" @@ -102024,7 +102046,8 @@ "tests/corpus/text-codec-values/main.ts", "tests/fixtures/node-types/text-codecs.mts", "tests/harness/project-config.test.ts", - "tests/corpus/3133-text-decoder-streaming.ts" + "tests/corpus/3133-text-decoder-streaming.ts", + "tests/corpus/7747-text-codec-untyped-slot.js" ] }, "anchorSource": "exact" @@ -102053,7 +102076,8 @@ "tests/corpus/text-codec-values/main.ts", "tests/fixtures/node-types/text-codecs.mts", "tests/harness/project-config.test.ts", - "tests/corpus/3133-text-decoder-streaming.ts" + "tests/corpus/3133-text-decoder-streaming.ts", + "tests/corpus/7747-text-codec-untyped-slot.js" ] }, "anchorSource": "exact" @@ -111471,6 +111495,7 @@ "tests/corpus/2630-stream-consumers-errors.ts", "tests/corpus/2862-web-readable-async-iteration.ts", "tests/corpus/4875-stream-consumers-array-buffer.ts", + "tests/corpus/7743-transform-stream-identity.ts", "tests/corpus/stream-web-readable.ts", "tests/corpus/stream-web-require.cjs", "tests/harness/fetch-conformance.test.ts" @@ -112945,8 +112970,11 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:webstreams" + "status": "partial", + "evidence": "compiler-feature:webstreams.TransformStream", + "tests": [ + "tests/corpus/7743-transform-stream-identity.ts" + ] }, "anchorSource": "exact" }, diff --git a/internal/compatibility/static-support.json b/internal/compatibility/static-support.json index 06aad26cf..2c6956cad 100644 --- a/internal/compatibility/static-support.json +++ b/internal/compatibility/static-support.json @@ -44,8 +44,8 @@ "node-builtin.console": ["tests/corpus/console-imports.cjs", "tests/corpus/2440-console-inspect-args.ts", "tests/corpus/console-native-value.js", "tests/harness/npm-static.test.ts"], "node-builtin.crypto.checkPrime": ["tests/corpus/7152-crypto-prime.ts", "tests/corpus/7153-crypto-prime-errors.cjs"], "node-builtin.crypto.checkPrimeSync": ["tests/corpus/7152-crypto-prime.ts", "tests/corpus/7153-crypto-prime-errors.cjs"], - "node-builtin.crypto.createCipheriv": ["tests/corpus/7150-cipher-legacy-modes.ts", "tests/corpus/7151-cipher-legacy-errors.cjs", "tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/corpus/7055-cipher-streams.ts", "tests/corpus/7056-cipher-stream-errors.ts", "tests/corpus/7057-cipher-streams.mjs", "tests/fixtures/node-types/cipher.ts"], - "node-builtin.crypto.createDecipheriv": ["tests/corpus/7150-cipher-legacy-modes.ts", "tests/corpus/7151-cipher-legacy-errors.cjs", "tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/corpus/7055-cipher-streams.ts", "tests/corpus/7056-cipher-stream-errors.ts", "tests/corpus/7057-cipher-streams.mjs", "tests/fixtures/node-types/cipher.ts"], + "node-builtin.crypto.createCipheriv": ["tests/corpus/7155-cipher-aes-ccm.cjs", "tests/corpus/7156-cipher-key-wrap.cjs", "tests/corpus/7150-cipher-legacy-modes.ts", "tests/corpus/7151-cipher-legacy-errors.cjs", "tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/corpus/7055-cipher-streams.ts", "tests/corpus/7056-cipher-stream-errors.ts", "tests/corpus/7057-cipher-streams.mjs", "tests/fixtures/node-types/cipher.ts"], + "node-builtin.crypto.createDecipheriv": ["tests/corpus/7155-cipher-aes-ccm.cjs", "tests/corpus/7156-cipher-key-wrap.cjs", "tests/corpus/7150-cipher-legacy-modes.ts", "tests/corpus/7151-cipher-legacy-errors.cjs", "tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/corpus/7055-cipher-streams.ts", "tests/corpus/7056-cipher-stream-errors.ts", "tests/corpus/7057-cipher-streams.mjs", "tests/fixtures/node-types/cipher.ts"], "node-builtin.crypto.createDiffieHellman": ["tests/corpus/6706-dh-groups.ts", "tests/corpus/6707-dh-explicit-params.ts", "tests/corpus/6708-dh-errors.js", "tests/corpus/6709-dh-generate.ts", "tests/corpus/6710-dh-encodings.cjs", "tests/corpus/6711-dh-handles.ts", "tests/fixtures/node-types/dh.ts"], "node-builtin.crypto.createDiffieHellmanGroup": ["tests/corpus/6706-dh-groups.ts", "tests/corpus/6707-dh-explicit-params.ts", "tests/corpus/6708-dh-errors.js", "tests/corpus/6709-dh-generate.ts", "tests/corpus/6710-dh-encodings.cjs", "tests/corpus/6711-dh-handles.ts", "tests/fixtures/node-types/dh.ts"], "node-builtin.crypto.createECDH": ["tests/corpus/6700-ecdh-shared-secret.ts", "tests/corpus/6701-ecdh-formats.ts", "tests/corpus/6702-ecdh-errors.js", "tests/corpus/6703-ecdh-private-keys.ts", "tests/corpus/6704-ecdh-set-public-key.cjs", "tests/corpus/6705-ecdh-handles.ts", "tests/fixtures/node-types/ecdh.ts"], @@ -124,6 +124,7 @@ "node-builtin.fs.constants": ["tests/corpus/6207-fs-constants-object.ts", "tests/corpus/6208-fs-constants-require.cjs"], "node-builtin.crypto.generatePrime": ["tests/corpus/7152-crypto-prime.ts", "tests/corpus/7153-crypto-prime-errors.cjs"], "node-builtin.crypto.generatePrimeSync": ["tests/corpus/7152-crypto-prime.ts", "tests/corpus/7153-crypto-prime-errors.cjs"], + "node-builtin.crypto.getCipherInfo": ["tests/corpus/7154-crypto-cipher-info.cjs", "tests/corpus/7157-crypto-cipher-info-typed.ts"], "node-builtin.crypto.getDiffieHellman": ["tests/corpus/6706-dh-groups.ts", "tests/corpus/6707-dh-explicit-params.ts", "tests/corpus/6708-dh-errors.js", "tests/corpus/6709-dh-generate.ts", "tests/corpus/6710-dh-encodings.cjs", "tests/corpus/6711-dh-handles.ts", "tests/fixtures/node-types/dh.ts"], "node-builtin.crypto.getRandomValues": ["tests/corpus/6204-crypto-get-random-values.ts", "tests/corpus/6205-crypto-get-random-values.js"], "node-builtin.url.parse": ["tests/corpus/5220-url-parse-warnings.ts", "tests/corpus/5221-url-parse-shapes.ts", "tests/corpus/5222-url-parse-hosts.ts", "tests/corpus/5223-url-parse-query.ts", "tests/corpus/5227-url-legacy-dependency/main.ts", "tests/corpus/5229-url-legacy-errors.cjs", "packages/runtime/test/url-legacy.test.ts", "tests/fixtures/node-types/url-legacy.ts"], @@ -331,11 +332,11 @@ { "chapter": "buffer", "signatures": ["`buf.readBigInt64BE([offset])`", "`buf.readBigInt64LE([offset])`", "`buf.readBigUInt64BE([offset])`", "`buf.readBigUInt64LE([offset])`", "`buf.writeBigInt64BE(value[, offset])`", "`buf.writeBigInt64LE(value[, offset])`", "`buf.writeBigUInt64BE(value[, offset])`", "`buf.writeBigUInt64LE(value[, offset])`"], "status": "supported", "evidence": ["tests/corpus/2902-bigint-bytes.ts", "tests/corpus/2904-bigint-errors.ts"] }, { "chapter": "environment_variables", "symbols": ["export"], "status": "not-applicable" }, { "chapter": "globals", "symbols": ["fetch"], "status": "partial", "evidence": ["tests/harness/fetch-conformance.test.ts"] }, - { "chapter": "globals", "signatures": ["Class: `TextEncoder`", "Class: `TextDecoder`"], "status": "partial", "evidence": ["tests/corpus/1423-text-codec.ts", "tests/corpus/text-codec-values/main.ts", "tests/corpus/3133-text-decoder-streaming.ts"] }, + { "chapter": "globals", "signatures": ["Class: `TextEncoder`", "Class: `TextDecoder`"], "status": "partial", "evidence": ["tests/corpus/1423-text-codec.ts", "tests/corpus/text-codec-values/main.ts", "tests/corpus/3133-text-decoder-streaming.ts", "tests/corpus/7747-text-codec-untyped-slot.js"] }, { "chapter": "globals", "signatures": ["Class: `Blob`", "Class: `File`"], "status": "partial", "evidence": ["tests/corpus/5285-blob-construct.ts", "tests/corpus/5286-blob-readers.ts", "tests/corpus/5287-blob-inspect.ts", "tests/corpus/5288-blob-validation.js", "tests/corpus/5290-blob-dynamic.js"] }, { "chapter": "buffer", "signatures": ["Class: `Blob`", "Class: `File`"], "status": "partial", "evidence": ["tests/corpus/5285-blob-construct.ts", "tests/corpus/5288-blob-validation.js", "tests/fixtures/node-types/blob-file.ts"] }, { "chapter": "buffer", "signatures": ["`blob.arrayBuffer()`", "`blob.bytes()`", "`blob.text()`", "`blob.slice([start[, end[, type]]])`", "`blob.size`", "`type` Type: {string}", "`name` Type: {string}", "`lastModified` Type: {number}"], "status": "supported", "evidence": ["tests/corpus/5285-blob-construct.ts", "tests/corpus/5286-blob-readers.ts", "tests/corpus/5288-blob-validation.js", "tests/corpus/5289-blob-read-order.ts", "tests/corpus/5290-blob-dynamic.js", "tests/fixtures/node-types/blob-file.ts"] }, - { "chapter": "util", "signatures": ["Class: `util.TextEncoder`", "Class: `util.TextDecoder`", "`textEncoder.encode([input])`", "`textDecoder.decode([input[, options]])`"], "status": "partial", "evidence": ["tests/corpus/text-codec-values/main.ts", "tests/fixtures/node-types/text-codecs.mts", "tests/harness/project-config.test.ts", "tests/corpus/3133-text-decoder-streaming.ts"] }, + { "chapter": "util", "signatures": ["Class: `util.TextEncoder`", "Class: `util.TextDecoder`", "`textEncoder.encode([input])`", "`textDecoder.decode([input[, options]])`"], "status": "partial", "evidence": ["tests/corpus/text-codec-values/main.ts", "tests/fixtures/node-types/text-codecs.mts", "tests/harness/project-config.test.ts", "tests/corpus/3133-text-decoder-streaming.ts", "tests/corpus/7747-text-codec-untyped-slot.js"] }, { "chapter": "modules", "symbols": ["__dirname", "__filename", "exports", "require", "module.require", "main", "filename"], "status": "partial", "evidence": ["tests/corpus/1612-cjs-module-globals.cjs", "tests/corpus/1596-cjs-modules/main.js", "tests/corpus/1629-require-main-filename.cjs", "tests/corpus/2830-cjs-require-self-name/main.cjs", "tests/corpus/2880-cjs-module-graph/main.cjs", "tests/corpus/2923-cjs-factory-export/main.cjs"] }, { "chapter": "modules", "symbols": ["require.resolve", "require.resolve.paths"], "status": "partial", "evidence": ["tests/corpus/2858-import-meta-resolve/main.mjs", "tests/corpus/2860-require-resolve/main.cjs"] }, { "chapter": "modules", "symbols": ["cache", "children", "id", "isPreloading", "loaded", "parent", "path", "paths", "Module"], "status": "partial", "evidence": ["tests/corpus/2880-cjs-module-graph/main.cjs"] }, @@ -406,7 +407,9 @@ { "chapter": "zlib", "signatures": ["`zlib.constants`"], "status": "partial", "evidence": ["tests/corpus/5376-zlib-constants.cjs", "tests/corpus/5374-zlib-stream-flush-params.ts"] }, { "chapter": "dns", "symbols": ["dns.lookup", "dns.lookupService", "dns.getServers", "dns.getDefaultResultOrder", "dns.setDefaultResultOrder"], "status": "partial", "evidence": ["tests/corpus/5648-dns-lookup-forms.ts", "tests/corpus/5649-dns-service-servers.ts", "tests/corpus/5650-dns-lookup-errors-untyped.cjs"] }, { "chapter": "dns", "symbols": ["dnsPromises.lookup", "dnsPromises.lookupService", "dnsPromises.getDefaultResultOrder"], "status": "partial", "evidence": ["tests/corpus/dns-promises-lookup.ts", "tests/corpus/dns-promises-untyped.cjs"] }, - { "chapter": "webstreams", "symbols": ["ReadableStream"], "status": "partial", "evidence": ["tests/corpus/stream-web-readable.ts", "tests/corpus/stream-web-require.cjs", "tests/corpus/2862-web-readable-async-iteration.ts"] } + { "chapter": "webstreams", "symbols": ["ReadableStream"], "status": "partial", "evidence": ["tests/corpus/stream-web-readable.ts", "tests/corpus/stream-web-require.cjs", "tests/corpus/2862-web-readable-async-iteration.ts"] }, + { "chapter": "webstreams", "symbols": ["TransformStream"], "status": "partial", "evidence": ["tests/corpus/7743-transform-stream-identity.ts"] }, + { "chapter": "globals", "signatures": ["Class: `TransformStream`"], "status": "partial", "evidence": ["tests/corpus/7743-transform-stream-identity.ts"] } ], "dedicated": [ { "symbols": ["assert.doesNotThrow", "assert.doesNotReject"], "status": "partial", "evidence": ["tests/corpus/node24-no-error-assertions.ts", "tests/corpus/node24-no-error-assertions.cjs", "tests/corpus/node24-no-error-assertions-coercion.js", "tests/fixtures/node-types/no-error-assertions.ts", "tests/harness/project-config.test.ts"] }, diff --git a/packages/cli-darwin-arm64/package.json b/packages/cli-darwin-arm64/package.json index 23584b60a..eee0ec3c8 100644 --- a/packages/cli-darwin-arm64/package.json +++ b/packages/cli-darwin-arm64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/cli-darwin-arm64", - "version": "0.4.2", + "version": "0.4.3", "description": "Native scriptc compiler for darwin-arm64", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli-darwin-x64/package.json b/packages/cli-darwin-x64/package.json index d9e6dafc6..b527a3cd7 100644 --- a/packages/cli-darwin-x64/package.json +++ b/packages/cli-darwin-x64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/cli-darwin-x64", - "version": "0.4.2", + "version": "0.4.3", "description": "Native scriptc compiler for darwin-x64", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli-linux-arm64-gnu/package.json b/packages/cli-linux-arm64-gnu/package.json index 5b8cb4db2..512a7cd13 100644 --- a/packages/cli-linux-arm64-gnu/package.json +++ b/packages/cli-linux-arm64-gnu/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/cli-linux-arm64-gnu", - "version": "0.4.2", + "version": "0.4.3", "description": "Native scriptc compiler for linux-arm64-gnu", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli-linux-arm64-musl/package.json b/packages/cli-linux-arm64-musl/package.json index 0901602ec..8d0fdfb0f 100644 --- a/packages/cli-linux-arm64-musl/package.json +++ b/packages/cli-linux-arm64-musl/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/cli-linux-arm64-musl", - "version": "0.4.2", + "version": "0.4.3", "description": "Native scriptc compiler for linux-arm64-musl", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli-linux-x64-gnu/package.json b/packages/cli-linux-x64-gnu/package.json index f01478aad..b71205614 100644 --- a/packages/cli-linux-x64-gnu/package.json +++ b/packages/cli-linux-x64-gnu/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/cli-linux-x64-gnu", - "version": "0.4.2", + "version": "0.4.3", "description": "Native scriptc compiler for linux-x64-gnu", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli-linux-x64-musl/package.json b/packages/cli-linux-x64-musl/package.json index a6986a66a..eaeb6f15a 100644 --- a/packages/cli-linux-x64-musl/package.json +++ b/packages/cli-linux-x64-musl/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/cli-linux-x64-musl", - "version": "0.4.2", + "version": "0.4.3", "description": "Native scriptc compiler for linux-x64-musl", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli-win32-x64-msvc/package.json b/packages/cli-win32-x64-msvc/package.json index 6b03811b8..3e3b8259f 100644 --- a/packages/cli-win32-x64-msvc/package.json +++ b/packages/cli-win32-x64-msvc/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/cli-win32-x64-msvc", - "version": "0.4.2", + "version": "0.4.3", "description": "Native scriptc compiler for win32-x64-msvc", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli/package.json b/packages/cli/package.json index 034747c29..264a7f5f1 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "scriptc", - "version": "0.4.2", + "version": "0.4.3", "description": "Compile ordinary TypeScript and JavaScript to small, fast native executables and WebAssembly modules that run without Node.js", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/cli/test/native-link-info.test.ts b/packages/cli/test/native-link-info.test.ts index a5edb11bb..8ac579449 100644 --- a/packages/cli/test/native-link-info.test.ts +++ b/packages/cli/test/native-link-info.test.ts @@ -72,7 +72,7 @@ describe.runIf(supported)("macOS arm64 native link info", () => { minimum_os: "14.0", }, program: { object, entry_symbol: "main" }, - runtime_abi: { version: 36, marker: "scr_runtime_abi_v36" }, + runtime_abi: { version: 37, marker: "scr_runtime_abi_v37" }, runtime_pack: { kind: "precompiled", package: "@scriptc/runtime-darwin-arm64", @@ -165,6 +165,6 @@ describe.runIf(supported)("macOS arm64 native link info", () => { () => null, (failure: { stderr?: string }) => failure, ); - expect(error?.stderr).toContain("scr_runtime_abi_v36"); + expect(error?.stderr).toContain("scr_runtime_abi_v37"); }); }); diff --git a/packages/compiler/ambient/scriptc-node-fallback.d.ts b/packages/compiler/ambient/scriptc-node-fallback.d.ts index b195c3fbb..09587aa72 100644 --- a/packages/compiler/ambient/scriptc-node-fallback.d.ts +++ b/packages/compiler/ambient/scriptc-node-fallback.d.ts @@ -1106,6 +1106,34 @@ declare var ReadableStream: { from(iterable: string): ReadableStream; readonly prototype: ReadableStream; }; +/* The writable side of a TransformStream and its writer. The + * WritableStream constructor has no native lowering; writable streams come + * from `new TransformStream()`. */ +interface WritableStreamDefaultWriter { + readonly closed: Promise; + readonly desiredSize: number | null; + readonly ready: Promise; + abort(reason?: unknown): Promise; + close(): Promise; + releaseLock(): void; + write(chunk?: W): Promise; +} +interface WritableStream { + readonly locked: boolean; + abort(reason?: unknown): Promise; + close(): Promise; + getWriter(): WritableStreamDefaultWriter; +} +/* The identity transform: chunks written to `writable` are read from + * `readable`, and a write completes when its chunk is read. */ +interface TransformStream { + readonly readable: ReadableStream; + readonly writable: WritableStream; +} +declare var TransformStream: { + new (): TransformStream; + readonly prototype: TransformStream; +}; /* fetch, typed as the native JSON + readable-stream slice: one URL, an * optional init with the members CLI requests carry, and a Response @@ -3171,13 +3199,17 @@ declare module "crypto" { callback: (error: Error | null, buffer: Buffer) => void, ): void; export type BinaryLike = string | ArrayBuffer | ArrayBufferView; - /* Cipheriv/Decipheriv: aes-128/192/256 in cbc, ctr, gcm, ecb, cfb, cfb8, - * cfb1, and ofb modes, des-ede3 and des-ede, chacha20, and - * chacha20-poly1305. Both are Transforms: written chunks go through - * update(), and the end pushes final(). */ + /* Cipheriv/Decipheriv: aes-128/192/256 in cbc, ctr, gcm, ccm, ecb, cfb, + * cfb8, cfb1, and ofb modes and their key wraps, des-ede3 and des-ede, + * chacha20, and chacha20-poly1305. Both are Transforms: written chunks go + * through update(), and the end pushes final(). */ + export type CipherCCMTypes = "aes-128-ccm" | "aes-192-ccm" | "aes-256-ccm"; export type CipherGCMTypes = "aes-128-gcm" | "aes-192-gcm" | "aes-256-gcm"; export type CipherChaCha20Poly1305Types = "chacha20-poly1305"; export type CipherKey = BinaryLike | KeyObject; + export interface CipherCCMOptions { + authTagLength: number; + } export interface CipherGCMOptions { authTagLength?: number; } @@ -3197,6 +3229,10 @@ declare module "crypto" { final(outputEncoding: BufferEncoding): string; setAutoPadding(autoPadding?: boolean): this; } + export interface CipherCCM extends Cipheriv { + setAAD(buffer: ArrayBufferView, options: { plaintextLength: number }): this; + getAuthTag(): Buffer; + } export interface CipherGCM extends Cipheriv { setAAD(buffer: ArrayBufferView, options?: { plaintextLength: number }): this; getAuthTag(): Buffer; @@ -3218,6 +3254,10 @@ declare module "crypto" { final(outputEncoding: BufferEncoding): string; setAutoPadding(autoPadding?: boolean): this; } + export interface DecipherCCM extends Decipheriv { + setAuthTag(buffer: ArrayBufferView): this; + setAAD(buffer: ArrayBufferView, options: { plaintextLength: number }): this; + } export interface DecipherGCM extends Decipheriv { setAuthTag(buffer: ArrayBufferView): this; setAAD(buffer: ArrayBufferView, options?: { plaintextLength: number }): this; @@ -3226,6 +3266,12 @@ declare module "crypto" { setAuthTag(buffer: ArrayBufferView): this; setAAD(buffer: ArrayBufferView, options?: { plaintextLength: number }): this; } + export function createCipheriv( + algorithm: CipherCCMTypes, + key: CipherKey, + iv: BinaryLike, + options: CipherCCMOptions, + ): CipherCCM; export function createCipheriv( algorithm: CipherGCMTypes, key: CipherKey, @@ -3244,6 +3290,12 @@ declare module "crypto" { iv: BinaryLike | null, options?: { authTagLength?: number }, ): Cipheriv; + export function createDecipheriv( + algorithm: CipherCCMTypes, + key: CipherKey, + iv: BinaryLike, + options: CipherCCMOptions, + ): DecipherCCM; export function createDecipheriv( algorithm: CipherGCMTypes, key: CipherKey, @@ -3468,6 +3520,34 @@ declare module "crypto" { * per member like http2.constants. */ export function getFips(): 1 | 0; export function getCiphers(): string[]; + export type CipherMode = + | "cbc" + | "ccm" + | "cfb" + | "ctr" + | "ecb" + | "gcm" + | "ocb" + | "ofb" + | "stream" + | "wrap" + | "xts"; + export interface CipherInfoOptions { + keyLength?: number | undefined; + ivLength?: number | undefined; + } + export interface CipherInfo { + name: string; + nid: number; + blockSize?: number | undefined; + ivLength?: number | undefined; + keyLength: number; + mode: CipherMode; + } + export function getCipherInfo( + nameOrNid: string | number, + options?: CipherInfoOptions, + ): CipherInfo | undefined; export function getHashes(): string[]; export function getCurves(): string[]; /* Node v24 crypto.constants, literal-typed (the Http2Constants @@ -5561,6 +5641,7 @@ declare module "http" { readonly trailersDistinct: { [name: string]: string[] | undefined }; readonly rawTrailers: string[]; resume(): void; + pause(): void; destroy(): void; setTimeout(msecs: number, callback?: () => void): this; /* setEncoding('utf8'): 'data' delivers strings (other real encodings @@ -5986,6 +6067,7 @@ declare module "http2" { export interface Http2Stream { on(event: "error" | "close", listener: (err: Error) => void): void; once(event: "error" | "close", listener: (err: Error) => void): void; + addListener(event: "error" | "close", listener: (err: Error) => void): void; destroy(): void; } export interface Http2ServerRequest { @@ -6086,11 +6168,6 @@ declare module "http2" { listener: (stream: ServerHttp2Stream, headers: IncomingHttpHeaders, flags: number) => void, ): void; on(event: "session", listener: (session: ServerHttp2Session) => void): void; - once( - event: "stream", - listener: (stream: ServerHttp2Stream, headers: IncomingHttpHeaders, flags: number) => void, - ): void; - once(event: "session", listener: (session: ServerHttp2Session) => void): void; /* HTTP CONNECT (and, in Node, the h2 extended CONNECT — which the * allowHTTP1 lowering never sees): the second argument is the raw * socket on HTTP/1.1 connections; narrow with instanceof. */ @@ -6108,6 +6185,42 @@ declare module "http2" { ): void; once(event: "error", listener: (err: Error) => void): void; once(event: "close", listener: () => void): void; + once(event: "sessionError", listener: (err: Error, session: ServerHttp2Session) => void): void; + once( + event: "stream", + listener: (stream: ServerHttp2Stream, headers: IncomingHttpHeaders, flags: number) => void, + ): void; + once(event: "session", listener: (session: ServerHttp2Session) => void): void; + once( + event: "connect", + listener: ( + req: Http2ServerRequest | import("http").IncomingMessage, + resOrSocket: Http2ServerResponse | Socket, + ) => void, + ): void; + addListener(event: "connection", listener: (socket: Socket) => void): void; + addListener( + event: "request", + listener: (req: Http2ServerRequest, res: Http2ServerResponse) => void, + ): void; + addListener(event: "error", listener: (err: Error) => void): void; + addListener(event: "close", listener: () => void): void; + addListener( + event: "sessionError", + listener: (err: Error, session: ServerHttp2Session) => void, + ): void; + addListener( + event: "stream", + listener: (stream: ServerHttp2Stream, headers: IncomingHttpHeaders, flags: number) => void, + ): void; + addListener(event: "session", listener: (session: ServerHttp2Session) => void): void; + addListener( + event: "connect", + listener: ( + req: Http2ServerRequest | import("http").IncomingMessage, + resOrSocket: Http2ServerResponse | Socket, + ) => void, + ): void; } export interface SecureServerOptions { allowHTTP1?: boolean; @@ -6449,6 +6562,28 @@ declare module "http2" { listener: (errorCode: number, lastStreamID: number, opaqueData?: Uint8Array) => void, ): void; once(event: "localSettings" | "remoteSettings", listener: (settings: any) => void): void; + once(event: "frameError", listener: (type: number, code: number, id: number) => void): void; + once(event: "ping", listener: (payload: Uint8Array) => void): void; + addListener(event: "close" | "timeout", listener: () => void): void; + addListener(event: "error", listener: (err: Error) => void): void; + addListener( + event: "connect", + listener: (session: Http2Session, socket: import("net").Socket) => void, + ): void; + addListener( + event: "stream", + listener: (stream: ServerHttp2Stream, headers: IncomingHttpHeaders, flags: number) => void, + ): void; + addListener( + event: "goaway", + listener: (errorCode: number, lastStreamID: number, opaqueData?: Uint8Array) => void, + ): void; + addListener(event: "localSettings" | "remoteSettings", listener: (settings: any) => void): void; + addListener( + event: "frameError", + listener: (type: number, code: number, id: number) => void, + ): void; + addListener(event: "ping", listener: (payload: Uint8Array) => void): void; } export interface ClientHttp2Session extends Http2Session { @@ -6526,7 +6661,30 @@ declare module "http2" { listener: () => void, ): void; once(event: "error", listener: (err: Error) => void): void; + once(event: "frameError", listener: (type: number, code: number, id: number) => void): void; once(event: "trailers", listener: (trailers: IncomingHttpHeaders, flags: number) => void): void; + addListener(event: "data", listener: (chunk: any) => void): void; + addListener( + event: + | "end" + | "close" + | "aborted" + | "ready" + | "timeout" + | "drain" + | "finish" + | "wantTrailers", + listener: () => void, + ): void; + addListener(event: "error", listener: (err: Error) => void): void; + addListener( + event: "frameError", + listener: (type: number, code: number, id: number) => void, + ): void; + addListener( + event: "trailers", + listener: (trailers: IncomingHttpHeaders, flags: number) => void, + ): void; respond( headers?: OutgoingHttpHeaders, options?: { endStream?: boolean; waitForTrailers?: boolean }, @@ -6576,6 +6734,7 @@ declare module "http2" { on(event: "trailers", listener: (trailers: IncomingHttpHeaders, flags: number) => void): void; once(event: "response", listener: (headers: IncomingHttpHeaders, flags: number) => void): void; once(event: "push", listener: (headers: IncomingHttpHeaders, flags: number) => void): void; + once(event: "headers", listener: (headers: IncomingHttpHeaders, flags: number) => void): void; once(event: "data", listener: (chunk: any) => void): void; once( event: @@ -6590,6 +6749,42 @@ declare module "http2" { listener: () => void, ): void; once(event: "error", listener: (err: Error) => void): void; + once(event: "frameError", listener: (type: number, code: number, id: number) => void): void; + once(event: "trailers", listener: (trailers: IncomingHttpHeaders, flags: number) => void): void; + addListener( + event: "response", + listener: (headers: IncomingHttpHeaders, flags: number) => void, + ): void; + addListener( + event: "push", + listener: (headers: IncomingHttpHeaders, flags: number) => void, + ): void; + addListener( + event: "headers", + listener: (headers: IncomingHttpHeaders, flags: number) => void, + ): void; + addListener(event: "data", listener: (chunk: any) => void): void; + addListener( + event: + | "end" + | "close" + | "aborted" + | "ready" + | "timeout" + | "drain" + | "finish" + | "wantTrailers", + listener: () => void, + ): void; + addListener(event: "error", listener: (err: Error) => void): void; + addListener( + event: "frameError", + listener: (type: number, code: number, id: number) => void, + ): void; + addListener( + event: "trailers", + listener: (trailers: IncomingHttpHeaders, flags: number) => void, + ): void; } export interface Http2Server { @@ -6624,6 +6819,24 @@ declare module "http2" { ): void; once(event: "error", listener: (err: Error) => void): void; once(event: "close" | "listening" | "timeout", listener: () => void): void; + once(event: "connection", listener: (socket: import("net").Socket) => void): void; + once(event: "sessionError", listener: (err: Error, session: ServerHttp2Session) => void): void; + addListener( + event: "stream", + listener: (stream: ServerHttp2Stream, headers: IncomingHttpHeaders, flags: number) => void, + ): void; + addListener(event: "session", listener: (session: ServerHttp2Session) => void): void; + addListener( + event: "request", + listener: (req: Http2ServerRequest, res: Http2ServerResponse) => void, + ): void; + addListener(event: "error", listener: (err: Error) => void): void; + addListener(event: "close" | "listening" | "timeout", listener: () => void): void; + addListener(event: "connection", listener: (socket: import("net").Socket) => void): void; + addListener( + event: "sessionError", + listener: (err: Error, session: ServerHttp2Session) => void, + ): void; } export interface ServerOptions { diff --git a/packages/compiler/native/cli.profdata.gz b/packages/compiler/native/cli.profdata.gz index aa0e9c36b..b5f1e079f 100644 Binary files a/packages/compiler/native/cli.profdata.gz and b/packages/compiler/native/cli.profdata.gz differ diff --git a/packages/compiler/package.json b/packages/compiler/package.json index a358ae545..60f6bd0f0 100644 --- a/packages/compiler/package.json +++ b/packages/compiler/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/compiler", - "version": "0.4.2", + "version": "0.4.3", "description": "The scriptc compiler — TypeScript/JavaScript frontend, typed IR, LLVM backend", "license": "Apache-2.0", "homepage": "https://scriptc.dev", @@ -40,7 +40,7 @@ }, "dependencies": { "@scriptc/runtime": "workspace:*", - "tsc-ts": "0.0.8", + "tsc-ts": "0.0.9", "typescript": "7.0.2", "typescript5": "npm:typescript@5.9.3" }, diff --git a/packages/compiler/shims/hono-node-server/v2.ts b/packages/compiler/shims/hono-node-server/v2.ts new file mode 100644 index 000000000..a4dcc6805 --- /dev/null +++ b/packages/compiler/shims/hono-node-server/v2.ts @@ -0,0 +1,645 @@ +// scriptc's built-in implementation of `@hono/node-server` 2.x: the Node.js +// adapter that serves a fetch-style handler (Hono's `app.fetch`) over +// node:http. The published package swaps the global Request and Response +// for lightweight objects and reparents prototypes, which only pays off +// under V8; this port builds native Request objects from the incoming +// message, with a body streamed from it as the app reads it, and writes +// the handler's Response the way the package's listener does (header +// record, content length for bodies that complete within the first reads, +// streamed bodies, error responses), so a client sees the same status +// lines, headers and bodies. A body the app leaves unread is drained after +// the response, within the package's limits. +// +// HTTP/1.1 only: the http2/https server options and WebSocket upgrades are +// not declared. A Response that is still exactly its constructor arguments +// takes the package's cached-response path (its lightweight Response), read +// from the runtime through scriptcResponseLight; every other Response takes +// the general path. +import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; +import type { AddressInfo } from "node:net"; + +export type HttpBindings = { + incoming: IncomingMessage; + outgoing: ServerResponse; +}; + +export type ServerType = Server; + +export type FetchCallback = (request: Request, env: HttpBindings) => Response | Promise; + +export type CustomErrorHandler = (err: unknown) => void | Response | Promise; + +export type Options = { + fetch: FetchCallback; + overrideGlobalObjects?: boolean; + autoCleanupIncoming?: boolean; + port?: number; + hostname?: string; +}; + +export interface RequestListenerOptions { + hostname?: string; + errorHandler?: CustomErrorHandler; + overrideGlobalObjects?: boolean; + autoCleanupIncoming?: boolean; +} + +export class RequestError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "RequestError"; + } +} + +const DEFAULT_CONTENT_TYPE = "text/plain; charset=UTF-8"; +const X_ALREADY_SENT = "x-hono-already-sent"; + +const reValidRequestUrl = /^\/[!#$&-;=?-[\]_a-z~]*$/; +const reDotSegment = /\/\.\.?(?:[/?#]|$)/; +const reValidHost = /^[a-z0-9._-]+(?::(?:[1-5]\d{3,4}|[6-9]\d{3}))?$/; + +function buildUrl(scheme: string, host: string, incomingUrl: string): string { + const url = `${scheme}://${host}${incomingUrl}`; + if (!reValidHost.test(host)) { + const urlObj = new URL(url); + if ( + urlObj.hostname.length !== host.length && + urlObj.hostname !== (host.includes(":") ? host.replace(/:\d+$/, "") : host).toLowerCase() + ) + throw new RequestError("Invalid host header"); + return urlObj.href; + } + if (incomingUrl.length === 0) return url + "/"; + if (incomingUrl.charCodeAt(0) !== 47) throw new RequestError("Invalid URL"); + if (!reValidRequestUrl.test(incomingUrl) || reDotSegment.test(incomingUrl)) + return new URL(url).href; + return url; +} + +function normalizeIncomingMethod(method: string | undefined): string { + if (method === undefined || method.length === 0) return "GET"; + switch (method) { + case "DELETE": + case "GET": + case "HEAD": + case "OPTIONS": + case "PATCH": + case "POST": + case "PUT": + case "QUERY": + return method; + } + const upper = method.toUpperCase(); + switch (upper) { + case "DELETE": + case "GET": + case "HEAD": + case "OPTIONS": + case "POST": + case "PUT": + return upper; + default: + return method; + } +} + +/** The request URL the package builds: an absolute request target as is, + * otherwise the Host header (or the configured hostname) over http. */ +function requestUrl(incoming: IncomingMessage, defaultHostname: string | undefined): string { + const incomingUrl = incoming.url ?? ""; + if ( + incomingUrl[0] !== "/" && + (incomingUrl.startsWith("http://") || incomingUrl.startsWith("https://")) + ) { + try { + return new URL(incomingUrl).href; + } catch (e) { + throw new RequestError("Invalid absolute URL", { cause: e }); + } + } + const hostHeader = incoming.headers.host; + const host = hostHeader !== undefined && hostHeader !== "" ? hostHeader : (defaultHostname ?? ""); + if (host === "") throw new RequestError("Missing host header"); + try { + return buildUrl("http", host, incomingUrl); + } catch (e) { + if (e instanceof RequestError) throw e; + throw new RequestError("Invalid URL", { cause: e }); + } +} + +function requestHeaders(incoming: IncomingMessage): Headers { + const headers = new Headers(); + const raw = incoming.rawHeaders; + for (let i = 0; i < raw.length; i += 2) { + const name = raw[i]!; + if (!name.startsWith(":")) headers.append(name, raw[i + 1]!); + } + return headers; +} + +/** What the listener knows about the incoming body: whether it ended + * (the package's `readableEnded`) and whether a drain started. */ +type IncomingState = { ended: boolean; draining: boolean }; + +/** The incoming message's readableHighWaterMark: Readable.toWeb queues + * this many bytes before it pauses the message. */ +const BODY_HIGH_WATER_MARK = 64 * 1024; + +/** The request body as the package streams it: Readable.toWeb over the + * incoming message, read one chunk per pull. Chunks queue until + * BODY_HIGH_WATER_MARK bytes wait unread, then the message pauses until a + * pull takes them, so the socket holds the rest. The stream pulls once on + * its own when it is constructed; the package builds its stream only when + * the app reads the body, so until the app's first read a drain discards + * the chunks instead of queueing them. */ +function incomingBody(incoming: IncomingMessage, state: IncomingState): ReadableStream { + let pulls = 0; + let listening = false; + let ended = false; + let failure: Error | undefined = undefined; + const queue: Buffer[] = []; + let queued = 0; + let wake: (() => void) | undefined = undefined; + const woken = (): void => { + const resolve = wake; + wake = undefined; + if (resolve !== undefined) resolve(); + }; + const listen = (): void => { + listening = true; + incoming.on("data", (chunk: Buffer) => { + if (state.draining && pulls === 1) return; + queue.push(chunk); + queued += chunk.length; + if (queued >= BODY_HIGH_WATER_MARK) incoming.pause(); + woken(); + }); + incoming.on("end", () => { + ended = true; + woken(); + }); + incoming.on("error", (err: Error) => { + failure = err; + woken(); + }); + }; + return new ReadableStream({ + async pull(controller) { + pulls++; + if (!listening) listen(); + for (;;) { + const chunk = queue.shift(); + if (chunk !== undefined) { + queued -= chunk.length; + if (queued < BODY_HIGH_WATER_MARK) incoming.resume(); + controller.enqueue(chunk); + return; + } + if (failure !== undefined) { + controller.error(failure); + return; + } + if (ended) { + controller.close(); + return; + } + await new Promise((resolve) => { + wake = resolve; + }); + } + }, + }); +} + +function newRequest( + incoming: IncomingMessage, + defaultHostname: string | undefined, + state: IncomingState, +): Request { + const method = normalizeIncomingMethod(incoming.method); + const url = requestUrl(incoming, defaultHostname); + const headers = requestHeaders(incoming); + if (method === "GET" || method === "HEAD") return new Request(url, { method, headers }); + return new Request(url, { + method, + headers, + body: incomingBody(incoming, state), + duplex: "half", + }); +} + +const DRAIN_TIMEOUT_MS = 500; +const MAX_DRAIN_BYTES = 64 * 1024 * 1024; + +/** The package's drain of a body the app left unread: read and discard + * up to MAX_DRAIN_BYTES or DRAIN_TIMEOUT_MS, then close the connection + * once its writes are out. */ +function drainIncoming(incoming: IncomingMessage, state: IncomingState): void { + const socket = incoming.socket; + if (socket.destroyed || state.draining) return; + state.draining = true; + let bytesRead = 0; + let done = false; + const cleanup = (): void => { + if (done) return; + done = true; + clearTimeout(timer); + }; + const forceClose = (): void => { + if (done) return; + cleanup(); + if (!socket.destroyed) socket.destroySoon(); + }; + const timer = setTimeout(forceClose, DRAIN_TIMEOUT_MS); + timer.unref(); + incoming.on("data", (chunk: Buffer) => { + if (done) return; + bytesRead += chunk.length; + if (bytesRead > MAX_DRAIN_BYTES) forceClose(); + }); + incoming.on("end", cleanup); + incoming.on("error", cleanup); + incoming.resume(); +} + +/** The package's close handler cleanup: an unread body drains two timer + * turns after the response closes. */ +function drainAfterClose(incoming: IncomingMessage, state: IncomingState): void { + if (state.ended) return; + setTimeout(() => { + if (!state.ended) + setTimeout(() => { + drainIncoming(incoming, state); + }); + }); +} + +/** The package's outgoing header record: Headers order (lowercase, + * sorted), Set-Cookie values gathered into one array entry at the end, + * and the default content type for a response with a body. */ +function buildOutgoingHttpHeaders( + headers: Headers, + hasBody: boolean, +): Record { + const res: Record = {}; + if (headers.has("set-cookie")) { + const cookies: string[] = []; + headers.forEach((v, k) => { + if (k === "set-cookie") cookies.push(v); + else res[k] = v; + }); + // Headers.forEach joins nothing but set-cookie, which getSetCookie + // lists value by value, as the package's iteration does. + if (cookies.length > 0) res["set-cookie"] = headers.getSetCookie(); + } else { + headers.forEach((v, k) => { + res[k] = v; + }); + } + if (hasBody && res["content-type"] === undefined) res["content-type"] = DEFAULT_CONTENT_TYPE; + return res; +} + +/* Spelled out rather than ReadableStreamReadResult, whose + * declaration depends on the program's lib and @types/node settings: the + * shim compiles inside every program that imports the package. */ +type ReadResult = { done: boolean; value?: Uint8Array }; + +function readWithoutBlocking(read: Promise): Promise { + return Promise.race([read, Promise.resolve().then(() => Promise.resolve(undefined))]); +} + +function nextTick(): Promise { + return new Promise((resolve) => { + setTimeout(resolve); + }); +} + +/** Writes the rest of a body stream. As in the package, a response that + * closes before the stream ends (the client disconnected) cancels the + * stream, so its producer stops; the pending read then answers done. A + * response that fails closes too, and its stream is cancelled the same way + * (the package passes the error as the reason). The listener stays attached + * and does nothing once the stream has ended. */ +async function pump( + reader: ReadableStreamDefaultReader, + outgoing: ServerResponse, + pending: Promise | undefined, +): Promise { + let finished = false; + outgoing.on("close", () => { + if (!finished) reader.cancel().catch(() => {}); + }); + try { + let read = pending ?? reader.read(); + for (;;) { + const { done, value } = await read; + if (done) { + outgoing.end(); + return; + } + outgoing.write(value!); + read = reader.read(); + } + } finally { + finished = true; + } +} + +function handleResponseError(e: unknown, outgoing: ServerResponse): void { + const err = e instanceof Error ? e : new Error("unknown error", { cause: e }); + console.error(e); + if (!outgoing.headersSent) outgoing.writeHead(500, { "Content-Type": "text/plain" }); + outgoing.end(`Error: ${err.message}`); + outgoing.destroy(); +} + +function emptyResponse(status: number): Response { + return new Response(null, { status }); +} + +/** The lightweight state of a constructed Response (the runtime's + * scr_fetch_response_light): [form, flags, byte length, ...record pairs]. */ +declare function scriptcResponseLight(res: Response): string[]; + +const UNDICI_TEXT_TYPE = "text/plain;charset=UTF-8"; + +/** Content-Length as Node's implicit header: set the headers one by one and + * let end(body) compute the length, which Node then writes after its own + * Date/Connection headers, as the package's preset length does. */ +function endWithImplicitLength( + outgoing: ServerResponse, + status: number, + header: Record, + body: Uint8Array, +): void { + outgoing.statusCode = status; + for (const name of Object.keys(header)) outgoing.setHeader(name, header[name]!); + outgoing.end(body); +} + +/** The package's cached-response path: a Response that is still exactly + * its constructor arguments (no body or metadata read since) is written + * from them. Form 1 has no init headers, form 2 Headers, form 3 a plain + * record kept verbatim. */ +async function writeLightResponse( + res: Response, + outgoing: ServerResponse, + light: string[], + head: boolean, +): Promise { + const form = light[0]!; + const flags = light[1]!; + const length = Number(light[2]!); + const status = res.status; + const sized = length >= 0; + if (form === "1") { + if (length === -1) { + outgoing.writeHead(status); + outgoing.end(); + } else if (sized) { + const body = await res.bytes(); + if (head) { + outgoing.writeHead(status, { + "Content-Type": DEFAULT_CONTENT_TYPE, + "Content-Length": String(length), + }); + outgoing.end(body); + } else + endWithImplicitLength(outgoing, status, { "Content-Type": DEFAULT_CONTENT_TYPE }, body); + } else { + outgoing.writeHead(status, { "Content-Type": DEFAULT_CONTENT_TYPE }); + outgoing.flushHeaders(); + await pump(res.body!.getReader(), outgoing, undefined); + } + return; + } + let header: Record; + let hasContentLength = false; + let plain = false; + let canAutoLength = true; + if (form === "2") { + const headers = new Headers(res.headers); + if (flags.includes("a") && headers.get("content-type") === UNDICI_TEXT_TYPE) { + if (flags.includes("d")) headers.set("content-type", DEFAULT_CONTENT_TYPE); + else headers.delete("content-type"); + } else if (flags.includes("d") && !headers.has("content-type")) { + headers.set("content-type", DEFAULT_CONTENT_TYPE); + } + hasContentLength = headers.has("content-length"); + header = buildOutgoingHttpHeaders(headers, length !== -1); + } else { + plain = true; + header = {}; + for (let i = 3; i + 1 < light.length; i += 2) { + const name = light[i]!; + const value = light[i + 1]!; + const previous = header[name]; + if (previous === undefined) header[name] = value; + else if (typeof previous === "string") header[name] = [previous, value]; + else previous.push(value); + const lower = name.toLowerCase(); + if (lower === "content-length") hasContentLength = true; + else if (lower === "transfer-encoding" || lower === "trailer") canAutoLength = false; + } + } + const body = sized ? await res.bytes() : undefined; + if (!hasContentLength && body !== undefined) { + if (plain && canAutoLength && !head && status !== 204 && status !== 304) { + endWithImplicitLength(outgoing, status, header, body); + return; + } + header["Content-Length"] = String(length); + } + outgoing.writeHead(status, header); + if (length === -1) outgoing.end(); + else if (body !== undefined) outgoing.end(body); + else { + outgoing.flushHeaders(); + await pump(res.body!.getReader(), outgoing, undefined); + } +} + +async function writeResponse( + res: Response, + outgoing: ServerResponse, + head: boolean, +): Promise { + const light = scriptcResponseLight(res); + if (light[0] !== "0") return writeLightResponse(res, outgoing, light, head); + const body = res.body; + const record = buildOutgoingHttpHeaders(res.headers, body !== null); + if (body !== null) { + const reader: ReadableStreamDefaultReader = body.getReader(); + const values: Uint8Array[] = []; + let done = false; + let pending: Promise | undefined = undefined; + if (record["transfer-encoding"] !== "chunked") { + let maxReadCount = 2; + for (let i = 0; i < maxReadCount; i++) { + pending ??= reader.read(); + let chunk: ReadResult | undefined = undefined; + try { + chunk = await readWithoutBlocking(pending); + } catch (e) { + console.error(e); + done = true; + } + if (chunk === undefined) { + if (i === 1 && !done) { + await nextTick(); + maxReadCount = 3; + continue; + } + break; + } + pending = undefined; + if (chunk.value !== undefined) values.push(chunk.value); + if (chunk.done) { + done = true; + break; + } + } + if (done && record["content-length"] === undefined) { + let length = 0; + for (const value of values) length += value.length; + record["content-length"] = String(length); + } + } + outgoing.writeHead(res.status, record); + for (const value of values) outgoing.write(value); + if (done) outgoing.end(); + else { + if (values.length === 0) outgoing.flushHeaders(); + await pump(reader, outgoing, pending); + } + } else if (record[X_ALREADY_SENT] === undefined) { + outgoing.writeHead(res.status, record); + outgoing.end(); + } +} + +async function respond( + pending: Response | Promise, + outgoing: ServerResponse, + errorHandler: CustomErrorHandler | undefined, + head: boolean, +): Promise { + let res: Response; + if (pending instanceof Promise) { + try { + res = await pending; + } catch (err) { + if (errorHandler === undefined) { + res = handleFetchError(err); + } else { + const errRes = await errorHandler(err); + if (errRes === undefined) return; + res = errRes; + } + } + } else res = pending; + await writeResponse(res, outgoing, head); +} + +/** A Response the package writes from its cache without waiting: still + * exactly its constructor arguments, with no body or a string or bytes + * body. */ +function immediateCacheable(res: Response): boolean { + const light = scriptcResponseLight(res); + return light[0] !== "0" && Number(light[2]!) >= -1; +} + +function handleFetchError(e: unknown): Response { + return emptyResponse(e instanceof Error && e.name === "TimeoutError" ? 504 : 500); +} + +async function handleRequest( + fetchCallback: FetchCallback, + options: RequestListenerOptions, + incoming: IncomingMessage, + outgoing: ServerResponse, +): Promise { + let req: Request | undefined = undefined; + let res: Response | Promise | undefined = undefined; + const state: IncomingState = { ended: false, draining: false }; + let needsBodyCleanup = false; + let closeHandlerAttached = false; + const ensureCloseHandler = (): void => { + if (req === undefined || closeHandlerAttached) return; + closeHandlerAttached = true; + if (needsBodyCleanup) + outgoing.on("close", () => { + drainAfterClose(incoming, state); + }); + }; + try { + req = newRequest(incoming, options.hostname, state); + needsBodyCleanup = + options.autoCleanupIncoming !== false && + !(incoming.method === "GET" || incoming.method === "HEAD"); + if (needsBodyCleanup) + incoming.on("end", () => { + state.ended = true; + }); + res = fetchCallback(req, { incoming, outgoing }); + if (!(res instanceof Promise) && immediateCacheable(res)) { + if (needsBodyCleanup) + outgoing.once("finish", () => { + if (!state.ended) drainIncoming(incoming, state); + }); + } else ensureCloseHandler(); + } catch (e) { + if (res !== undefined) return handleResponseError(e, outgoing); + const errorHandler = options.errorHandler; + if (errorHandler !== undefined) { + ensureCloseHandler(); + const handled = await errorHandler( + req !== undefined ? e : e instanceof RequestError ? e : toRequestError(e), + ); + if (handled === undefined) return; + res = handled; + } else if (req === undefined) res = emptyResponse(400); + else res = handleFetchError(e); + } + try { + await respond(res, outgoing, options.errorHandler, incoming.method === "HEAD"); + } catch (e) { + handleResponseError(e, outgoing); + } +} + +function toRequestError(e: unknown): RequestError { + return new RequestError(e instanceof Error ? e.message : String(e), { cause: e }); +} + +export function getRequestListener( + fetchCallback: FetchCallback, + options: RequestListenerOptions = {}, +): (incoming: IncomingMessage, outgoing: ServerResponse) => Promise { + return (incoming, outgoing) => handleRequest(fetchCallback, options, incoming, outgoing); +} + +export function createAdaptorServer(options: Options): ServerType { + const listener = getRequestListener(options.fetch, { + hostname: options.hostname, + overrideGlobalObjects: options.overrideGlobalObjects, + autoCleanupIncoming: options.autoCleanupIncoming, + }); + return createServer((incoming, outgoing) => { + void listener(incoming, outgoing); + }); +} + +export function serve( + options: Options, + listeningListener?: (info: AddressInfo) => void, +): ServerType { + const server = createAdaptorServer(options); + server.listen(options.port ?? 3000, options.hostname, () => { + const info = server.address() as AddressInfo; + if (listeningListener !== undefined) listeningListener(info); + }); + return server; +} diff --git a/packages/compiler/src/backend/executable-features.ts b/packages/compiler/src/backend/executable-features.ts index 32318d53e..a9076af22 100644 --- a/packages/compiler/src/backend/executable-features.ts +++ b/packages/compiler/src/backend/executable-features.ts @@ -50,6 +50,7 @@ export function executableLinkFeatures(mod: IrModule): NativeLinkFeatures { dgram: features.dgram, dnsPromises: features.dnsPromises, dgramDyn: features.dgramDyn, + textCodecDyn: features.textCodecDyn, watch: features.fsWatch, foreignFfi: hasForeignFfiCallback(mod.ffiImports ?? []), nodeTest: features.nodeTest, diff --git a/packages/compiler/src/backend/immutability.ts b/packages/compiler/src/backend/immutability.ts index 21380de8f..379ad60ee 100644 --- a/packages/compiler/src/backend/immutability.ts +++ b/packages/compiler/src/backend/immutability.ts @@ -3,6 +3,7 @@ import type { DynamicFieldStores } from "../ir/int32-slots.js"; import type { IrExpr, IrModule, IrStmt } from "../ir/ir.js"; import { everyExprChild, everyStmtChild } from "../ir/traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "../ir/node-index.js"; import { computeCycleMutability, constructorEscapePoint, @@ -97,7 +98,10 @@ export interface ConstructionFacts { privateStores: Set; } -export function computeConstructionFacts(mod: IrModule): ConstructionFacts { +/** The nodes that can reach a constructor with a reused receiver. */ +const CONSTRUCTOR_USE_KINDS: readonly IrNodeKind[] = ["call", "closure"]; + +export function computeConstructionFacts(mod: IrModule, nodes?: IrNodeIndex): ConstructionFacts { const facts: ConstructionFacts = { privateStores: new Set() }; if (mod.lib !== undefined) return facts; const classes = new Map((mod.classes ?? []).map((c) => [c.name, c] as const)); @@ -122,6 +126,7 @@ export function computeConstructionFacts(mod: IrModule): ConstructionFacts { c = classes.get(c)!.base; } }; + const scan = nodes?.scan(CONSTRUCTOR_USE_KINDS); for (const fn of mod.functions) { const own = ctorClass.get(fn.name); const thisId = own !== undefined ? thisOf(own) : null; @@ -129,22 +134,19 @@ export function computeConstructionFacts(mod: IrModule): ConstructionFacts { own !== undefined && classes.get(own)!.base !== undefined ? `%${classes.get(own)!.base}.constructor` : null; - const expr = (e: IrExpr): boolean => { - if (e.kind === "call") { - const target = ctorClass.get(e.callee); + eachNodeOfKinds(fn, scan, (node) => { + if (node.kind === "call") { + const target = ctorClass.get(node.callee); if ( target !== undefined && - !(e.callee === baseCtor && e.args.length > 0 && isThisRef(e.args[0]!, thisId)) + !(node.callee === baseCtor && node.args.length > 0 && isThisRef(node.args[0]!, thisId)) ) markReused(target); - } else if (e.kind === "closure") { - const target = ctorClass.get(e.fnName); + } else if (node.kind === "closure") { + const target = ctorClass.get(node.fnName); if (target !== undefined) markReused(target); } - return everyExprChild(e, expr, stmt); - }; - const stmt = (s: IrStmt): boolean => everyStmtChild(s, expr, stmt); - fn.body.every(stmt); + }); } // The first escaping statement of each constructor (body length: none). @@ -192,32 +194,32 @@ export function computeConstructionFacts(mod: IrModule): ConstructionFacts { return facts; } +/** The nodes that write a binding: statement and expression assignments. */ +const ASSIGNMENT_KINDS: readonly IrNodeKind[] = ["assign", "assignExpr"]; + /** Module constants whose value never changes once initialized: immutable, * not TDZ-checked (every read follows the initialization), no import-cycle * init flag, and at most one assignment in the whole module. Such a global * owns its value for the rest of the program, so a read of it can be * borrowed across any later operand or call. */ -export function stableGlobals(mod: IrModule): Set { +export function stableGlobals(mod: IrModule, nodes?: IrNodeIndex): Set { const candidates = new Set( (mod.globals ?? []) .filter((g) => !g.mutable && g.tdz !== true && g.initFlag === undefined) .map((g) => g.id), ); const assigned = new Set(); - const expr = (e: IrExpr): boolean => { - if (e.kind === "assignExpr" && candidates.has(e.localId)) candidates.delete(e.localId); - return everyExprChild(e, expr, stmt); - }; - const stmt = (s: IrStmt): boolean => { - if (s.kind === "assign" && candidates.has(s.localId)) { - if (assigned.has(s.localId)) candidates.delete(s.localId); - assigned.add(s.localId); - } - return everyStmtChild(s, expr, stmt); - }; + const scan = nodes?.scan(ASSIGNMENT_KINDS); for (const fn of mod.functions) { if (candidates.size === 0) break; - fn.body.every(stmt); + eachNodeOfKinds(fn, scan, (node) => { + if (node.kind === "assignExpr" && candidates.has(node.localId)) + candidates.delete(node.localId); + else if (node.kind === "assign" && candidates.has(node.localId)) { + if (assigned.has(node.localId)) candidates.delete(node.localId); + assigned.add(node.localId); + } + }); } return candidates; } diff --git a/packages/compiler/src/backend/library-pack.ts b/packages/compiler/src/backend/library-pack.ts index e9f52f7a0..79be316db 100644 --- a/packages/compiler/src/backend/library-pack.ts +++ b/packages/compiler/src/backend/library-pack.ts @@ -54,6 +54,7 @@ export async function compilePackedLibrary( dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, diff --git a/packages/compiler/src/backend/llvm/alloc.ts b/packages/compiler/src/backend/llvm/alloc.ts index aa6345dab..2efeb96ac 100644 --- a/packages/compiler/src/backend/llvm/alloc.ts +++ b/packages/compiler/src/backend/llvm/alloc.ts @@ -459,18 +459,72 @@ export type FreeObservers = "weak" | "weak-or-source"; * type's disposals), and the block — `scr_cyc_free` for collector-headered * objects, `scr_rt_free` otherwise. Labels use the `sf.` prefix; the lines * end in an open block. + * + * With `keep` (an alloca of ptr holding NULL; inline allocation paths only), + * a block that would go onto the calling thread's free list is stored there + * instead, for a teardown segment (emitSegmentFree); a block the system or + * another arena owns is still freed. */ export function emitObjectFree( host: AllocHost, cyc: boolean, observers: FreeObservers = "weak", + keep: string | null = null, ): string[] { - const lines = threadLocalState(host, "sf", freeLines(host, cyc, observers)); + const lines = threadLocalState(host, "sf", freeLines(host, cyc, observers, keep)); if (host.objectAudit === false) return lines.filter((line) => line !== FREE_NOTE); host.declare(`declare void @scr_obj_free_note()`); return lines; } +/** Lines that put the blocks `first` to `last` (linked through their first + * words; one block when they are equal) of the class at reservation offset + * `off` on the calling thread's free list. */ +function freeListPushLines(prefix: string, off: string, first: string, last: string): string[] { + return [ + ` %${prefix}.shp = getelementptr inbounds i8, ptr @scr_sa, i64 ${SA_SHIFT}`, + ` %${prefix}.sh32 = load i32, ptr %${prefix}.shp`, + ` %${prefix}.sh = zext i32 %${prefix}.sh32 to i64`, + ` %${prefix}.c = lshr i64 ${off}, %${prefix}.sh`, + ` %${prefix}.c8 = shl i64 %${prefix}.c, 3`, + ` %${prefix}.fo = add i64 %${prefix}.c8, ${SA_FREE}`, + ` %${prefix}.fp = getelementptr inbounds i8, ptr @scr_sa, i64 %${prefix}.fo`, + ` %${prefix}.head = load ptr, ptr %${prefix}.fp`, + ` store ptr %${prefix}.head, ptr ${last}`, + ` store ptr ${first}, ptr %${prefix}.fp`, + ]; +} + +/** + * Lines that free a teardown segment: the blocks `first` to `last` (ptr + * values, both NULL for an empty segment), linked through their first words, + * which emitObjectFree kept. Every block of a segment is in the calling + * thread's arena and of one size class, so the segment goes onto that + * class's free list in one step, and the next allocations of the class take + * its blocks in segment order. Labels use `prefix`; the lines end in the + * open block `${prefix}.done`. + */ +export function emitSegmentFree( + host: AllocHost, + prefix: string, + first: string, + last: string, +): string[] { + declareState(host); + const lines = [ + ` %${prefix}.none = icmp eq ptr ${first}, null`, + ` br i1 %${prefix}.none, label %${prefix}.done, label %${prefix}.push`, + `${prefix}.push:`, + ` %${prefix}.base = load i64, ptr @scr_sa`, + ` %${prefix}.bi = ptrtoint ptr ${first} to i64`, + ` %${prefix}.off = sub i64 %${prefix}.bi, %${prefix}.base`, + ...freeListPushLines(prefix, `%${prefix}.off`, first, last), + ` br label %${prefix}.done`, + `${prefix}.done:`, + ]; + return threadLocalState(host, prefix, lines); +} + /** Lines that leave in `%.obs` whether the program can observe a * disposal of this type (folds to a constant). */ function observedLines(prefix: string, observers: FreeObservers): string[] { @@ -482,8 +536,14 @@ function observedLines(prefix: string, observers: FreeObservers): string[] { ]; } -function freeLines(host: AllocHost, cyc: boolean, observers: FreeObservers): string[] { +function freeLines( + host: AllocHost, + cyc: boolean, + observers: FreeObservers, + keep: string | null, +): string[] { if (host.inlineAlloc !== true) { + if (keep !== null) throw new Error("a kept block needs the inline allocation paths"); if (cyc) { host.declare(`declare void @scr_cyc_free(ptr)`); return [` call void @scr_obj_free_note()`, ` call void @scr_cyc_free(ptr %o)`]; @@ -548,16 +608,9 @@ function freeLines(host: AllocHost, cyc: boolean, observers: FreeObservers): str ` %sf.inx = call i1 @llvm.expect.i1(i1 %sf.in, i1 true)`, ` br i1 %sf.inx, label %sf.push, label %sf.sys`, `sf.push:`, - ` %sf.shp = getelementptr inbounds i8, ptr @scr_sa, i64 ${SA_SHIFT}`, - ` %sf.sh32 = load i32, ptr %sf.shp`, - ` %sf.sh = zext i32 %sf.sh32 to i64`, - ` %sf.c = lshr i64 %sf.off, %sf.sh`, - ` %sf.c8 = shl i64 %sf.c, 3`, - ` %sf.fo = add i64 %sf.c8, ${SA_FREE}`, - ` %sf.fp = getelementptr inbounds i8, ptr @scr_sa, i64 %sf.fo`, - ` %sf.head = load ptr, ptr %sf.fp`, - ` store ptr %sf.head, ptr ${blk}`, - ` store ptr ${blk}, ptr %sf.fp`, + ...(keep !== null + ? [` store ptr ${blk}, ptr ${keep}`] + : freeListPushLines("sf", "%sf.off", blk, blk)), ` br label %sf.done`, `sf.sys: ; outside the reservation, or the allocator is compiled out`, ` call void @scr_obj_free_note()`, diff --git a/packages/compiler/src/backend/llvm/appended-array-literals.ts b/packages/compiler/src/backend/llvm/appended-array-literals.ts index 09a2976de..3c09904e3 100644 --- a/packages/compiler/src/backend/llvm/appended-array-literals.ts +++ b/packages/compiler/src/backend/llvm/appended-array-literals.ts @@ -9,8 +9,8 @@ * one directly or through a binding read from it. The choice never changes * behavior. */ -import type { IrExpr, IrModule, IrStmt } from "../../ir/ir.js"; -import { everyStmtList } from "../../ir/traverse.js"; +import type { IrExpr, IrModule } from "../../ir/ir.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "../../ir/node-index.js"; const APPENDING_METHODS: ReadonlySet = new Set([ "push", @@ -30,7 +30,27 @@ function isEmptyLiteral(e: IrExpr): boolean { ); } -export function findAppendedArrayLiterals(module: IrModule): ReadonlySet { +/** The nodes that name an appended binding or field, and the stores that + * initialize one. */ +const TARGET_KINDS: readonly IrNodeKind[] = [ + "arrIntrinsic", + "assignExpr", + "arraySet", + "varDecl", + "assign", +]; +const STORE_KINDS: readonly IrNodeKind[] = [ + "assignExpr", + "varDecl", + "assign", + "fieldSet", + "recordSet", +]; + +export function findAppendedArrayLiterals( + module: IrModule, + index?: IrNodeIndex, +): ReadonlySet { // Bindings are function-scoped by id; module globals are shared. const localsOf = new Map>(); for (const fn of module.functions) { @@ -65,20 +85,15 @@ export function findAppendedArrayLiterals(module: IrModule): ReadonlySet const field = fieldOf(target); if (field !== null) appendedFields.add(field); }; + const targets = index?.scan(TARGET_KINDS); for (const fn of module.functions) { - everyStmtList(fn.body, { - expr: (e) => { - if (e.kind === "arrIntrinsic" && APPENDING_METHODS.has(e.method)) - noteTarget(fn.name, e.receiver); - else if (e.kind === "assignExpr") noteAlias(fn.name, e.localId, e.value); - return true; - }, - stmt: (s) => { - if (s.kind === "arraySet") noteTarget(fn.name, s.arr); - else if (s.kind === "varDecl") noteAlias(fn.name, s.localId, s.init); - else if (s.kind === "assign") noteAlias(fn.name, s.localId, s.value); - return true; - }, + eachNodeOfKinds(fn, targets, (node) => { + if (node.kind === "arrIntrinsic" && APPENDING_METHODS.has(node.method)) + noteTarget(fn.name, node.receiver); + else if (node.kind === "assignExpr") noteAlias(fn.name, node.localId, node.value); + else if (node.kind === "arraySet") noteTarget(fn.name, node.arr); + else if (node.kind === "varDecl") noteAlias(fn.name, node.localId, node.init); + else if (node.kind === "assign") noteAlias(fn.name, node.localId, node.value); }); } for (const binding of appendedBindings) @@ -89,23 +104,18 @@ export function findAppendedArrayLiterals(module: IrModule): ReadonlySet const consider = (value: IrExpr | null, appended: boolean): void => { if (value && appended && isEmptyLiteral(value)) result.add(value); }; + const stores = index?.scan(STORE_KINDS); for (const fn of module.functions) { const binding = (localId: string): boolean => appendedBindings.has(bindingKey(fn.name, localId)); - everyStmtList(fn.body, { - expr: (e) => { - if (e.kind === "assignExpr") consider(e.value, binding(e.localId)); - return true; - }, - stmt: (s: IrStmt) => { - if (s.kind === "varDecl") consider(s.init, binding(s.localId)); - else if (s.kind === "assign") consider(s.value, binding(s.localId)); - else if (s.kind === "fieldSet") - consider(s.value, appendedFields.has(fieldKey(s.className, s.field))); - else if (s.kind === "recordSet") - consider(s.value, appendedFields.has(fieldKey(s.shapeId, s.field))); - return true; - }, + eachNodeOfKinds(fn, stores, (node) => { + if (node.kind === "assignExpr") consider(node.value, binding(node.localId)); + else if (node.kind === "varDecl") consider(node.init, binding(node.localId)); + else if (node.kind === "assign") consider(node.value, binding(node.localId)); + else if (node.kind === "fieldSet") + consider(node.value, appendedFields.has(fieldKey(node.className, node.field))); + else if (node.kind === "recordSet") + consider(node.value, appendedFields.has(fieldKey(node.shapeId, node.field))); }); } return result; diff --git a/packages/compiler/src/backend/llvm/bytes-alias.ts b/packages/compiler/src/backend/llvm/bytes-alias.ts index 9aaeaa6c7..d43620fd9 100644 --- a/packages/compiler/src/backend/llvm/bytes-alias.ts +++ b/packages/compiler/src/backend/llvm/bytes-alias.ts @@ -7,9 +7,15 @@ import type { LlvmEmitterContext } from "./expr-context.js"; * buffer. Neither overlaps the other, a native field, array storage or a * module global's slot, so element stores leave header loads and global * receiver slots loop-invariant. Untagged accesses and calls still alias - * every category. */ + * every category. + * + * Header loads are also `!invariant.load`: no store or call between the + * construction of a view and its release changes them. Without it, a data + * pointer load ahead of a bounds branch is sunk into the in-bounds block, + * and LICM keeps a conditionally executed load inside the loop. */ export function bytesHeaderAlias(host: LlvmEmitterContext): string { - return host.memoryAliasAttachment("bytes:header"); + const tag = host.memoryAliasAttachment("bytes:header"); + return tag === "" ? "" : `${tag}, !invariant.load !{}`; } export function bytesElementAlias(host: LlvmEmitterContext): string { diff --git a/packages/compiler/src/backend/llvm/bytes-loop-versions.test.ts b/packages/compiler/src/backend/llvm/bytes-loop-versions.test.ts index cee91c66d..de71a6ef0 100644 --- a/packages/compiler/src/backend/llvm/bytes-loop-versions.test.ts +++ b/packages/compiler/src/backend/llvm/bytes-loop-versions.test.ts @@ -11,7 +11,7 @@ import { type IrType, } from "../../ir/ir.js"; import { analyzeIntegerRanges } from "../../ir/integer-ranges.js"; -import { matchIntegerCountedForLoop } from "../../ir/integer-loops.js"; +import { matchIntegerCountedForLoop, matchIntegerPairedForLoop } from "../../ir/integer-loops.js"; import { findInitializedByteLoopBindings } from "./byte-windows.js"; import { matchBytesLoopVersion, type BytesLoopVersion } from "./bytes-loop-versions.js"; import { emitLlvmModule } from "./emitter.js"; @@ -249,6 +249,11 @@ test("emitted loops version the body and tag typed-array memory", () => { const elements = /store double [^\n]*!tbaa !(\d+)/.exec(body)![1]; const header = /load ptr, ptr [^\n]*!tbaa !(\d+)/.exec(body)![1]; expect(elements).not.toBe(header); + // Header loads are invariant for the view's lifetime; element accesses + // are not. + expect(body).toMatch(/load ptr, ptr [^\n]*, !tbaa !\d+, !invariant\.load !\{\}$/m); + expect(body).toMatch(/load i64, ptr [^\n]*, !tbaa !\d+, !invariant\.load !\{\}$/m); + expect(body).not.toMatch(/(load double|store double)[^\n]*invariant\.load/); }); test("loops to a typed array's length version the other typed arrays they index", () => { @@ -618,3 +623,78 @@ test("while loop counters below a string's length version its reads", () => { expect(version.bounded.map((c) => c.slack)).toEqual([0]); expect(version.fixed).toEqual([]); }); + +test("a two-counter loop versions the typed arrays it indexes at either counter", () => { + // for (let i = 0, j = k; i < j; i++, j--) { const t = out[i]; out[i] = out[j]; out[j] = t; } + const loop: IrStmt & { kind: "for" } = { + kind: "for", + init: { + kind: "block", + body: [ + { kind: "varDecl", localId: "i", init: num(0), loc }, + { kind: "varDecl", localId: "j", init: ref("k"), loc }, + ], + loc, + }, + cond: { kind: "bin", op: "<", left: ref("i"), right: ref("j"), type: BOOL, loc }, + update: { + kind: "block", + body: [ + { kind: "assign", localId: "i", value: bin("+", ref("i"), num(1)), loc }, + { kind: "assign", localId: "j", value: bin("-", ref("j"), num(1)), loc }, + ], + loc, + }, + body: [ + { kind: "varDecl", localId: "t", init: get(ref("out", f64s), ref("i")), loc }, + set(ref("out", f64s), ref("i"), get(ref("out", f64s), ref("j"))), + set(ref("out", f64s), ref("j"), ref("t")), + // Another index stays checked. + set(ref("src", f64s), bin("+", ref("i"), num(1)), num(0)), + ], + loc, + }; + const kernel = fn([loop], [local("t", F64)]); + const ranges = analyzeIntegerRanges(kernel); + const locals = new Map(kernel.locals.map((l) => [l.id, l])); + const paired = matchIntegerPairedForLoop(loop, locals, ranges)!; + expect(paired).not.toBeNull(); + const version = matchBytesLoopVersion( + loop, + null, + { + locals, + invariantGlobals: new Set(), + captured: new Set(), + ranges, + initialized: findInitializedByteLoopBindings(kernel).get(loop) ?? new Set(), + proven: new Set(), + workers: false, + owned: new Set(), + scalarStrings: new Set(), + }, + paired, + )!; + expect(version.paired?.loop).toBe(paired); + expect(version.paired?.receivers.map((r) => version.receivers[r]!.ref)).toEqual([ + ref("out", f64s), + ]); + expect(version.bounds.size).toBe(4); + const mod: IrModule = { + irVersion: 17, + sourceFile: loc.file, + entry: "__main", + functions: [ + { name: "__main", params: [], returnType: VOID, locals: [], body: [], loc }, + kernel, + ], + }; + const body = /^define internal [^\n]*@sc_bf_kernel[^]*?^}/m.exec(emitLlvmModule(mod))![0]; + const guard = body.slice(0, body.search(/^bytes\.version\.(fast|slow)\d+:$/m)); + expect(guard).toMatch(/icmp sge i64 %\S+, 0/); + expect(guard).toMatch(/icmp slt i64 %\S+, %\S+/); + const fast = /^bytes\.version\.fast\d+:$[^]*?^bytes\.version\.done\d+:$/m.exec(body)![0]; + // The fast copy keeps only the `src[i + 1]` check, and no second guard. + expect(fast.match(/bytes\.index\.invalid\d+:/g)).toHaveLength(1); + expect(fast).not.toContain("counted.fast"); +}); diff --git a/packages/compiler/src/backend/llvm/bytes-loop-versions.ts b/packages/compiler/src/backend/llvm/bytes-loop-versions.ts index 704b6723a..2013d03e4 100644 --- a/packages/compiler/src/backend/llvm/bytes-loop-versions.ts +++ b/packages/compiler/src/backend/llvm/bytes-loop-versions.ts @@ -1,11 +1,15 @@ import { F64, type IrExpr, type IrLocal, type IrStmt } from "../../ir/ir.js"; import type { IntegerRange, IntegerRanges } from "../../ir/integer-ranges.js"; -import type { IntegerCountedForLoop } from "../../ir/integer-loops.js"; +import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js"; import type { LlvmEmitterContext } from "./expr-context.js"; import { byteNumberAccess } from "../../ir/byte-numbers.js"; import { everyExprChild, everyStmtChild } from "../../ir/traverse.js"; import { exactInteger, widenInteger } from "./integer-values.js"; -import { emitCountedLoopLimit, emitCountedLoopSafety } from "./counted-loops.js"; +import { + emitCountedLoopLimit, + emitCountedLoopSafety, + emitExactIntegerTest, +} from "./counted-loops.js"; import { emitBytesHeaderLength, emitBytesSharedStorage } from "./expr-bytes.js"; import { f64Lit } from "./common.js"; import { emitStringAsciiFlag, emitStringByteLength } from "./string-reads.js"; @@ -79,6 +83,10 @@ export interface BytesLoopVersion { guarded: ReadonlySet; /** The counted loop whose iteration space the affine checks cover. */ counted: IntegerCountedForLoop | null; + /** A two-counter loop (`for (let i = lo, j = hi; i < j; i++, j--)`) and + * the receivers it indexes at either counter. Both counters stay between + * the two starts, so the guard requires `lo >= 0` and `hi < length`. */ + paired: { loop: IntegerPairedForLoop; receivers: number[] } | null; /** The guard also requires every counter value within [-bound, bound]. */ counterBound: number | null; /** Integer ranges of affine index operands that hold in the unchecked @@ -153,6 +161,7 @@ export function matchBytesLoopVersion( loop: IrStmt & { kind: "for" | "while" }, counted: IntegerCountedForLoop | null, facts: BytesLoopFacts, + paired: IntegerPairedForLoop | null = null, ): BytesLoopVersion | null { const written = new Set(); const temps = new Map(); @@ -438,6 +447,9 @@ export function matchBytesLoopVersion( string, { check: BoundedCheck; nodes: { node: Node; access: WhileAccess }[] } >(); + const pairedCounters = + paired !== null && counted === null ? new Set([paired.low.localId, paired.high.localId]) : null; + const pairedNodes = new Map(); const access = (node: Node, array: IrExpr, index: IrExpr, width: number): void => { const ref = root(array); if (!ref) return; @@ -448,6 +460,20 @@ export function matchBytesLoopVersion( else owned.set(receiver, [node]); } if (facts.proven.has(node)) return; + // An element at either counter of a two-counter loop. + if ( + pairedCounters !== null && + width === 1 && + ref.type.kind === "bytes" && + index.kind === "varRef" && + pairedCounters.has(index.localId) + ) { + const receiver = receiverOf(ref); + const nodes = pairedNodes.get(receiver); + if (nodes) nodes.push(node); + else pairedNodes.set(receiver, [node]); + return; + } const shape = counter !== null ? affine(index) : null; if (shape && shape.counter !== 0) { const operands = new Set(); @@ -543,11 +569,16 @@ export function matchBytesLoopVersion( Math.max(0, MAX_CHECKS - fixedKept.length - affineKept.length), ); const ownedKept = [...owned.entries()].slice(0, MAX_CHECKS); + const pairedKept = [...pairedNodes.entries()].slice( + 0, + Math.max(0, MAX_CHECKS - fixedKept.length - affineKept.length - boundedKept.length), + ); if ( fixedKept.length === 0 && affineKept.length === 0 && boundedKept.length === 0 && - ownedKept.length === 0 + ownedKept.length === 0 && + pairedKept.length === 0 ) return null; const bounds = new Set(); @@ -616,6 +647,10 @@ export function matchBytesLoopVersion( if (access.index !== access.ref) narrow(access.index, { min: 0, max: SAFE }); } } + for (const [receiver, nodes] of pairedKept) { + used.add(receiver); + for (const node of nodes) bounds.add(node); + } const counterChecks: CounterCheck[] = []; for (const [id, min] of counterMins) { const info = whileAccesses!.counters.get(id)!; @@ -658,6 +693,10 @@ export function matchBytesLoopVersion( counters: counterChecks, guarded, counted: affineKept.length > 0 ? counted : null, + paired: + paired !== null && pairedKept.length > 0 + ? { loop: paired, receivers: pairedKept.map(([receiver]) => renumber.get(receiver)!) } + : null, counterBound: needsBound ? bound : null, ranges, }; @@ -1143,6 +1182,25 @@ export function emitBytesLoopVersionGuard( conditions.push(both); } } + if (version.paired !== null) { + // Both counters stay within [lo, hi] (exact integers below 2^53): every + // access at either one is in bounds when `lo >= 0` and `hi < length`. + const { loop } = version.paired; + const [low, high] = [loop.low.start, loop.high.start].map((expr) => { + const tested = emitExactIntegerTest(host, host.emitExpr(expr).name, -SAFE, SAFE); + conditions.push(tested.ok); + return tested.integer; + }); + const nonNegative = B.tmp(); + B.line(`${nonNegative} = icmp sge i64 ${low}, 0`); + conditions.push(nonNegative); + for (const receiver of version.paired.receivers) { + // Lengths are at most 2^53 elements: a signed comparison is exact. + const below = B.tmp(); + B.line(`${below} = icmp slt i64 ${high}, ${lengths[receiver]}`); + conditions.push(below); + } + } let valid = "true"; for (const cond of conditions) { if (valid === "true") valid = cond; @@ -1156,39 +1214,6 @@ export function emitBytesLoopVersionGuard( else B.condBr(valid, fast, slow); } -/** Tests that a double is an exact integer within [min, max] (not -0; NaN - * fails the ordered comparisons) and gives its i64. The integer is frozen, - * so it is a fixed value even when the test fails. */ -function emitExactIntegerTest( - host: LlvmEmitterContext, - value: string, - min: number, - max: number, -): { ok: string; integer: string } { - const B = host.B; - const low = B.tmp(), - high = B.tmp(), - inRange = B.tmp(), - raw = B.tmp(), - integer = B.tmp(), - back = B.tmp(), - bits = B.tmp(), - backBits = B.tmp(), - exact = B.tmp(), - ok = B.tmp(); - B.line(`${low} = fcmp oge double ${value}, ${f64Lit(min)}`); - B.line(`${high} = fcmp ole double ${value}, ${f64Lit(max)}`); - B.line(`${inRange} = and i1 ${low}, ${high}`); - B.line(`${raw} = fptosi double ${value} to i64`); - B.line(`${integer} = freeze i64 ${raw}`); - B.line(`${back} = sitofp i64 ${integer} to double`); - B.line(`${bits} = bitcast double ${value} to i64`); - B.line(`${backBits} = bitcast double ${back} to i64`); - B.line(`${exact} = icmp eq i64 ${bits}, ${backBits}`); - B.line(`${ok} = and i1 ${inRange}, ${exact}`); - return { ok, integer }; -} - function scaleCounter(host: LlvmEmitterContext, value: string, scale: number): string { if (scale === 1) return value; const scaled = host.B.tmp(); diff --git a/packages/compiler/src/backend/llvm/classes.test.ts b/packages/compiler/src/backend/llvm/classes.test.ts index bee954afb..f472eec60 100644 --- a/packages/compiler/src/backend/llvm/classes.test.ts +++ b/packages/compiler/src/backend/llvm/classes.test.ts @@ -77,6 +77,65 @@ test("leaf layouts keep direct teardown while recursive layouts share the depth } }); +test("teardowns of types with children of their own type free their blocks as one segment", () => { + const linked = { kind: "object" as const, className: "Linked" }; + const mod: IrModule = { + irVersion: 17, + sourceFile: loc.file, + entry: "main", + functions: [{ name: "main", params: [], returnType: VOID, locals: [], body: [], loc }], + classes: [ + { + name: "Linked", + fields: [ + { name: "next", type: linked }, + { name: "other", type: linked }, + ], + methods: [], + loc, + }, + ], + records: [{ id: "branch", fields: [{ name: "next", type: linked }] }], + }; + const define = (llvm: string, name: string): string => + new RegExp(`^define internal [^@]*@${name}\\([^]*?^}`, "m").exec(llvm)?.[0] ?? ""; + const linkedDestroy = `${mangleClassRelease("Linked")}_destroy`; + const branchDestroy = `${mangleRecordRelease("branch")}_destroy`; + for (const workers of [false, true]) { + const llvm = emitLlvmModule({ ...mod, workers }, { pointerBits: 64 }); + // Linked's teardown keeps its block and its children's in a segment: it + // releases its own block first and places it by address against its + // first child, a child torn down directly appends to the segment, one + // past the direct chain appends through the runtime, and the chain's + // first teardown frees the segment onto the free list in one step. + const seg = define(llvm, `${linkedDestroy}_s`); + expect(seg.startsWith("define internal ptr"), "segment form").toBe(true); + expect(seg).toMatch( + /store ptr %(?:sf\.blk|o), ptr %sg\.own[^]*call ptr @sc_release_Linked_destroy_s\(/, + ); + // The children's releases have a copy for each place of the own block. + for (const copy of ["ta", "tb"]) + expect(seg).toContain( + `call ptr @scr_rc_destroy_seg(ptr %v0, ptr @${linkedDestroy}_sc, ptr @${linkedDestroy}, ptr %${copy}0.bt)`, + ); + expect(seg).not.toContain("call void @scr_rc_destroy("); + expect(seg).toContain("icmp ult ptr %sg.o, %sg.r1"); + const first = define(llvm, `${linkedDestroy}_d`); + expect(first).toContain(`call ptr @${linkedDestroy}_s(ptr %o, i32 %d, ptr %head)`); + expect(first).toContain("store ptr %first, ptr %sgf.fp"); + expect(first.includes("llvm.threadlocal.address"), "thread-local state").toBe(workers); + expect(define(llvm, linkedDestroy)).toContain(`call void @${linkedDestroy}_d(ptr %o, i32 0)`); + // A record whose child is another type frees its own block as before. + expect(define(llvm, `${branchDestroy}_s`)).toBe(""); + expect(define(llvm, branchDestroy)).toContain(`call void @${mangleClassRelease("Linked")}(`); + } + // Without the inline allocation paths (32-bit targets) a teardown frees + // through the runtime, block by block. + const llvm32 = emitLlvmModule(mod, { pointerBits: 32 }); + expect(define(llvm32, `${linkedDestroy}_s`)).toBe(""); + expect(define(llvm32, `${linkedDestroy}_d`)).toContain(`call void @${linkedDestroy}_d(`); +}); + test("releases split into an inlinable fast path and an out-of-line remainder", () => { // An array of the class itself makes the hierarchy cycle-capable. const kids = { kind: "array" as const, elem: { kind: "object" as const, className: "Node" } }; diff --git a/packages/compiler/src/backend/llvm/classes.ts b/packages/compiler/src/backend/llvm/classes.ts index 8f179828d..82e34de90 100644 --- a/packages/compiler/src/backend/llvm/classes.ts +++ b/packages/compiler/src/backend/llvm/classes.ts @@ -43,6 +43,9 @@ import { destroyDefinitions, directChildReleaseLines, directDestroyTarget, + segmentPrologueLines, + segmentTeardownLines, + segmentedTeardown, usedDirectTargets, type DirectDestroy, type RegionHooks, @@ -1082,25 +1085,62 @@ export function emitClassShapes( bounded && (directTargets.has(`object:${cls.name}`) || refFields.some((f) => directDestroyTarget(host, f.type, directTargets) !== null)); + // The teardown's own symbol, and whether it keeps a segment (shapes.ts + // segmentedTeardown). + const destroy = `${meta.hierarchy ? mangleClassReleaseDirect(cls.name) : mangleClassRelease(cls.name)}_destroy`; + const segmented = + chained && + !meta.compact && + runtimePrefix === null && + !isStreamRooted && + !meta.threadSlot && + segmentedTeardown( + host, + destroy, + refFields.map((f) => f.type), + directTargets, + ); // The field-releasing teardown body shared by both release shapes // (the public one on standalone classes, the DIRECT one on hierarchy // members). Runs at rc == 0. const teardown = (lines: string[]): void => { - refFields.forEach((f, t) => { - lines.push( - ` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${f.index}`, - ` %v${t} = load ptr, ptr %f${t}`, - ); + const release = (prefix: string, f: (typeof refFields)[number], t: number): string[] => { const direct = bounded ? directDestroyTarget(host, f.type, directTargets) : null; - if (direct) - lines.push( - ...directChildReleaseLines(host, `td${t}`, `%v${t}`, direct, llvmCommentText(f.name)), - ); - else - lines.push( + if (direct === null) + return [ ` call void ${releaseSym(host, f.type)}(ptr %v${t}) ; ${llvmCommentText(f.name)}`, - ); - }); + ]; + const own = segmented && direct.destroy === destroy; + return directChildReleaseLines( + host, + `${prefix}${t}`, + `%v${t}`, + direct, + llvmCommentText(f.name), + own, + ); + }; + const loads = refFields.map((f, t) => [ + ` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${f.index}`, + ` %v${t} = load ptr, ptr %f${t}`, + ]); + if (segmented) { + // A segmented teardown loads its fields, releases its own block, and + // then releases the fields (shapes.ts segmentTeardownLines). + const ownType = refFields.flatMap((f, t) => + directDestroyTarget(host, f.type, directTargets)?.destroy === destroy ? [`%v${t}`] : [], + ); + lines.push( + ...segmentPrologueLines(), + ...loads.flat(), + ...emitObjectFree(host, traced, "weak", "%sg.own"), + ...segmentTeardownLines(ownType, (prefix) => + refFields.flatMap((f, t) => release(prefix, f, t)), + ), + ); + return; + } + refFields.forEach((f, t) => lines.push(...loads[t]!, ...release("td", f, t))); if (runtimePrefix) lines.push(...regCall("td", runtimePrefix.drop, "")); if (isStreamRooted) lines.push(...stCall("tds", "scr_stream_st_release", "")); if (meta.threadSlot) lines.push(...slotForgetLines(host, struct, meta, "tdw")); @@ -1182,10 +1222,12 @@ export function emitClassShapes( ]; if (traced) reld.push(...cycleDeadLines(host, "teardown")); if (bounded) { - const destroy = `${mangleClassReleaseDirect(cls.name)}_destroy`; const destroyBody: string[] = []; teardown(destroyBody); - defs.push(...destroyDefinitions(destroy, destroyBody, chained), ``); + defs.push( + ...destroyDefinitions(destroy, destroyBody, chained, segmented ? host : null), + ``, + ); host.declare(`declare void @scr_rc_destroy(ptr, ptr)`); reld.push(` call void @scr_rc_destroy(ptr %o, ptr @${destroy})`); } else teardown(reld); @@ -1207,6 +1249,7 @@ export function emitClassShapes( bounded, lifetime, chained, + segmented, ), ``, ); diff --git a/packages/compiler/src/backend/llvm/common.ts b/packages/compiler/src/backend/llvm/common.ts index ac4e28708..cca1b603b 100644 --- a/packages/compiler/src/backend/llvm/common.ts +++ b/packages/compiler/src/backend/llvm/common.ts @@ -29,6 +29,9 @@ export const SCR_ARR_FIELD = { * (SCR_ARR_STATES in scr_array.h). */ export const SCR_ARR_STATES = 16; +/** The `spare` flag of a frozen %ScrArr (SCR_ARR_FROZEN in scr_array.h). */ +export const SCR_ARR_FROZEN = 32; + /** Loads the i32 `len` or `cap` of a %ScrArr at `ptr` into `dest` as a * size-type count. */ export function loadArrCount( diff --git a/packages/compiler/src/backend/llvm/constant-callbacks.ts b/packages/compiler/src/backend/llvm/constant-callbacks.ts index cf5c1b062..4b277dab7 100644 --- a/packages/compiler/src/backend/llvm/constant-callbacks.ts +++ b/packages/compiler/src/backend/llvm/constant-callbacks.ts @@ -1,9 +1,28 @@ import type { IrExpr, IrFunction, IrModule } from "../../ir/ir.js"; -import { exprFunctionReference, moduleFunctionReferences } from "../../ir/function-references.js"; +import { + exprFunctionReference, + FUNCTION_REFERENCE_KINDS, + moduleFunctionReferences, +} from "../../ir/function-references.js"; import { writtenLocals } from "../../ir/callback-specialization.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "../../ir/node-index.js"; import { everyStmtList } from "../../ir/traverse.js"; import type { CallLifetimes } from "./call-lifetimes.js"; +/** The nodes the body census below reads. */ +const CENSUS_KINDS: readonly IrNodeKind[] = [ + "varDecl", + "assign", + "forOf", + "rethrow", + "tryCatch", + "call", + "callValue", + "assignExpr", + "incDec", + ...FUNCTION_REFERENCE_KINDS, +]; + /** Closed direct-call consumers can devirtualize a callback when every * incoming call supplies the same function. Keep its evaluated environment, * arguments and ordinary ownership; only the loaded code pointer changes. @@ -15,6 +34,7 @@ import type { CallLifetimes } from "./call-lifetimes.js"; export function findConstantCallbacks( mod: IrModule, lifetimes: CallLifetimes, + index?: IrNodeIndex, ): Map> { const functions = new Map(mod.functions.map((fn) => [fn.name, fn])); const definitions = new Map>(); @@ -50,7 +70,7 @@ export function findConstantCallbacks( const writes = new Set(); // Parameters and captures each function (or a closure sharing the // binding) writes after entry. - const rebinds = writtenLocals(mod); + const rebinds = writtenLocals(mod, index); // A function named anywhere but as a direct call's callee can be entered // by a generated adapter or the runtime, which a direct-call census cannot // specialize: the entry, exports, class helpers and methods here, and @@ -66,6 +86,7 @@ export function findConstantCallbacks( // The initializer of each immutable function-typed local declared once: // its only write, even when a capture box (not a stable slot) holds it. const constDefinitions = new Map>(); + const scan = index?.scan(CENSUS_KINDS); for (const fn of mod.functions) { const defs = new Map(); const consts = new Map(); @@ -85,51 +106,64 @@ export function findConstantCallbacks( const emptyDecls = new Map(); const assigned = new Map(); const rebound = new Set(); - everyStmtList(fn.body, { - stmt: (stmt) => { - if (stmt.kind === "varDecl" && stmt.init?.type.kind === "func" && stable?.has(stmt.localId)) - defs.set(stmt.localId, stmt.init); - if (stmt.kind === "varDecl" && immutable.has(stmt.localId)) - consts.set(stmt.localId, consts.has(stmt.localId) ? null : stmt.init); - if (stmt.kind === "varDecl" && hoisted.has(stmt.localId)) { - if (stmt.init !== null) rebound.add(stmt.localId); - emptyDecls.set(stmt.localId, (emptyDecls.get(stmt.localId) ?? 0) + 1); - } - if (stmt.kind === "assign" && hoisted.has(stmt.localId)) { - const values = assigned.get(stmt.localId) ?? []; - values.push(stmt.value); - assigned.set(stmt.localId, values); - } - if (stmt.kind === "forOf" || stmt.kind === "rethrow") rebound.add(stmt.localId); - if (stmt.kind === "tryCatch" && stmt.catchLocalId !== null) rebound.add(stmt.catchLocalId); - if (stmt.kind === "assign") { - if (globalIds.has(stmt.localId)) { - if (globals.has(stmt.localId)) writes.add(stmt.localId); - else globals.set(stmt.localId, stmt.value); + eachNodeOfKinds(fn, scan, (node) => { + switch (node.kind) { + case "varDecl": + if (node.init?.type.kind === "func" && stable?.has(node.localId)) + defs.set(node.localId, node.init); + if (immutable.has(node.localId)) + consts.set(node.localId, consts.has(node.localId) ? null : node.init); + if (hoisted.has(node.localId)) { + if (node.init !== null) rebound.add(node.localId); + emptyDecls.set(node.localId, (emptyDecls.get(node.localId) ?? 0) + 1); } + return; + case "assign": + if (hoisted.has(node.localId)) { + const values = assigned.get(node.localId) ?? []; + values.push(node.value); + assigned.set(node.localId, values); + } + if (globalIds.has(node.localId)) { + if (globals.has(node.localId)) writes.add(node.localId); + else globals.set(node.localId, node.value); + } + return; + case "forOf": + case "rethrow": + rebound.add(node.localId); + return; + case "tryCatch": + if (node.catchLocalId !== null) rebound.add(node.catchLocalId); + return; + case "call": + if (consumers.has(node.callee)) { + const calls = incoming.get(node.callee) ?? []; + calls.push({ owner: fn.name, args: node.args }); + incoming.set(node.callee, calls); + } + return; + case "callValue": + if (node.callee.kind === "varRef") callees.add(node.callee.localId); + return; + case "closure": { + escaped.add(node.fnName); + const sites = closureSites.get(node.fnName) ?? []; + sites.push({ owner: fn.name, captures: node.captures }); + closureSites.set(node.fnName, sites); + return; } - return true; - }, - expr: (expr) => { - if (expr.kind === "call" && consumers.has(expr.callee)) { - const calls = incoming.get(expr.callee) ?? []; - calls.push({ owner: fn.name, args: expr.args }); - incoming.set(expr.callee, calls); + case "orDefault": { + const referenced = exprFunctionReference(node); + if (referenced !== undefined) escaped.add(referenced); + return; } - if (expr.kind === "callValue" && expr.callee.kind === "varRef") - callees.add(expr.callee.localId); - const referenced = exprFunctionReference(expr); - if (referenced !== undefined) escaped.add(referenced); - if (expr.kind === "closure") { - const sites = closureSites.get(expr.fnName) ?? []; - sites.push({ owner: fn.name, captures: expr.captures }); - closureSites.set(expr.fnName, sites); - } - if ((expr.kind === "assignExpr" || expr.kind === "incDec") && globalIds.has(expr.localId)) - writes.add(expr.localId); - if (expr.kind === "assignExpr" || expr.kind === "incDec") rebound.add(expr.localId); - return true; - }, + case "assignExpr": + case "incDec": + if (globalIds.has(node.localId)) writes.add(node.localId); + rebound.add(node.localId); + return; + } }); for (const [id, values] of assigned) if (emptyDecls.get(id) === 1 && values.length === 1 && !rebound.has(id)) diff --git a/packages/compiler/src/backend/llvm/counted-loops.ts b/packages/compiler/src/backend/llvm/counted-loops.ts index ca2dd6b92..1ff4c6580 100644 --- a/packages/compiler/src/backend/llvm/counted-loops.ts +++ b/packages/compiler/src/backend/llvm/counted-loops.ts @@ -151,3 +151,36 @@ function emitExactStart(host: LlvmEmitterContext, expr: IrExpr): string { B.line(`${ok} = and i1 ${inRange}, ${exact}`); return ok; } + +/** Tests that a double is an exact integer within [min, max] (not -0; NaN + * fails the ordered comparisons) and gives its i64. The integer is frozen, + * so it is a fixed value even when the test fails. */ +export function emitExactIntegerTest( + host: LlvmEmitterContext, + value: string, + min: number, + max: number, +): { ok: string; integer: string } { + const B = host.B; + const low = B.tmp(), + high = B.tmp(), + inRange = B.tmp(), + raw = B.tmp(), + integer = B.tmp(), + back = B.tmp(), + bits = B.tmp(), + backBits = B.tmp(), + exact = B.tmp(), + ok = B.tmp(); + B.line(`${low} = fcmp oge double ${value}, ${f64Lit(min)}`); + B.line(`${high} = fcmp ole double ${value}, ${f64Lit(max)}`); + B.line(`${inRange} = and i1 ${low}, ${high}`); + B.line(`${raw} = fptosi double ${value} to i64`); + B.line(`${integer} = freeze i64 ${raw}`); + B.line(`${back} = sitofp i64 ${integer} to double`); + B.line(`${bits} = bitcast double ${value} to i64`); + B.line(`${backBits} = bitcast double ${back} to i64`); + B.line(`${exact} = icmp eq i64 ${bits}, ${backBits}`); + B.line(`${ok} = and i1 ${inRange}, ${exact}`); + return { ok, integer }; +} diff --git a/packages/compiler/src/backend/llvm/dense-array-access.ts b/packages/compiler/src/backend/llvm/dense-array-access.ts index 95612e57c..df37d9c58 100644 --- a/packages/compiler/src/backend/llvm/dense-array-access.ts +++ b/packages/compiler/src/backend/llvm/dense-array-access.ts @@ -505,8 +505,12 @@ export function emitDenseArrayState( indexExpr: IrExpr | undefined, kind: "state" | "has", width: number, + /** The index is a dense array loop counter its entry test proved below + * the hole-free prefix (provenDenseIndex): the slot is a present value. */ + proven = false, ): string { const B = host.B; + if (proven) return kind === "state" ? f64Lit(SCR_ARR_VALUE) : "true"; const fn = kind === "state" ? "scr_arr_state" : "scr_arr_has"; const ty = kind === "state" ? "double" : "i1"; host.declare(`declare ${kind === "state" ? "double" : "zeroext i1"} @${fn}(ptr, double)`); diff --git a/packages/compiler/src/backend/llvm/dense-array-loops.test.ts b/packages/compiler/src/backend/llvm/dense-array-loops.test.ts index f7a4bc892..a88c38575 100644 --- a/packages/compiler/src/backend/llvm/dense-array-loops.test.ts +++ b/packages/compiler/src/backend/llvm/dense-array-loops.test.ts @@ -1,9 +1,10 @@ import { expect, test } from "vitest"; import { BOOL, F64, arrayOf, type IrExpr, type IrLocal, type IrStmt } from "../../ir/ir.js"; -import type { IntegerCountedForLoop } from "../../ir/integer-loops.js"; +import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js"; import { countedDenseShape, matchDenseArrayLoop, + pairedDenseShape, type DenseLoopShape, } from "./dense-array-loops.js"; @@ -42,6 +43,8 @@ const locals = new Map([ ["a", { id: "a", name: "a", type: ARR, mutable: false }], ["b", { id: "b", name: "b", type: ARR, mutable: true }], ["c", { id: "c", name: "c", type: ARR, mutable: false }], + ["j", { id: "j", name: "j", type: F64, mutable: true }], + ["t", { id: "t", name: "t", type: F64, mutable: false }], ]); /** for (let i = 0; i < a.length; i++) body */ @@ -193,3 +196,100 @@ test("a loop to another invariant limit covers each array whose length it keeps" expect([...matchLimited(resizing, new Set(["b"]))!.arrays.keys()]).toEqual(["a"]); expect(matchLimited(resizing)).toBeNull(); }); + +test("a two-counter loop covers the arrays indexed at either counter", () => { + // for (let i = 1, j = k - 1; i < j; i++, j--) { t = a[i]; a[i] = a[j]; a[j] = t; } + const paired: IntegerPairedForLoop = { + low: { localId: "i", start: num(1), step: 1, startGuarded: false }, + high: { localId: "j", start: ref("k"), step: -1, startGuarded: true }, + inclusive: false, + guarded: true, + range: { min: -Number.MAX_SAFE_INTEGER, max: Number.MAX_SAFE_INTEGER }, + }; + const pairedShape = pairedDenseShape(paired); + const matchPaired = (body: IrStmt[]) => + matchDenseArrayLoop(loop(body), pairedShape, locals, new Set(), new Set(["a", "b"])); + const swap: IrStmt[] = [ + { kind: "varDecl", localId: "t", init: read("a"), loc }, + write("a", read("a", ref("j"))), + write("a", ref("t"), ref("j")), + ]; + const dense = matchPaired(swap); + expect([...dense!.arrays.keys()]).toEqual(["a"]); + expect(dense!.pair).toEqual({ high: "j", lowStart: num(1), highStart: ref("k") }); + expect(dense!.range).toEqual({ min: 0, max: 2147483646 }); + // Stores at any other index may extend the array. + expect(matchPaired([...swap, write("a", num(0), ref("k"))])).toBeNull(); +}); + +test("a present-undefined store is covered only where it stores a covered read's missing arm", () => { + // const v = b[i] (as number | undefined); + // if (v is undefined) c[i] = undefined; else c[i] = v is number ? v : throw; + const unionType = { kind: "union" as const, unionId: "u" }; + const v: IrExpr = { kind: "varRef", localId: "v", type: unionType, loc }; + const isTag = (tag: number): IrExpr => ({ + kind: "unionIsTag", + unionId: "u", + tag, + negated: false, + value: v, + type: BOOL, + loc, + }); + const reader = (array: string, index = ref("i")): IrExpr => ({ + kind: "call", + callee: "%arr.idxOr.f64", + args: [arr(array), index], + type: unionType, + loc, + }); + const undefinedStore: IrStmt = { kind: "arraySetUndefined", arr: arr("c"), index: ref("i"), loc }; + const guarded = (array: string): IrStmt[] => [ + { kind: "varDecl", localId: "v", init: reader(array), loc }, + { + kind: "if", + cond: isTag(1), + then: [undefinedStore], + else_: [ + write("c", { + kind: "ternary", + cond: isTag(0), + then: { kind: "unionNarrow", unionId: "u", tag: 0, value: v, type: F64, loc }, + else_: { + kind: "libCall", + fn: "error.nodeThrow", + args: [num(1), { kind: "strLit", value: "", type: { kind: "string" }, loc }], + type: F64, + loc, + }, + type: F64, + loc, + }), + ], + loc, + }, + ]; + const withV = new Map([ + ...locals, + ["v", { id: "v", name: "v", type: unionType, mutable: false }], + ]); + const matchStores = (body: IrStmt[], initialized: string[]) => + matchDenseArrayLoop(loop(body), shape, withV, new Set(), new Set(initialized)); + const dense = matchStores(guarded("b"), ["a", "b", "c"]); + expect([...dense!.arrays.keys()].sort()).toEqual(["b", "c"]); + // A read of an array the entry test does not cover may miss in the fast + // copy, and its undefined store would run there. + expect(matchStores(guarded("b"), ["a", "c"])).toBeNull(); + // A present undefined stored on its own leaves out its array and every + // array the other name may hold. + const alone: IrStmt[] = [{ kind: "exprStmt", expr: read("b"), loc }, undefinedStore]; + expect(matchStores(alone, ["a", "b", "c"])).toBeNull(); + const separate = matchDenseArrayLoop( + loop(alone), + shape, + withV, + new Set(["c"]), + new Set(["a", "b", "c"]), + ); + expect([...separate!.arrays.keys()]).toEqual(["b"]); +}); diff --git a/packages/compiler/src/backend/llvm/dense-array-loops.ts b/packages/compiler/src/backend/llvm/dense-array-loops.ts index 184ff1353..b806939b5 100644 --- a/packages/compiler/src/backend/llvm/dense-array-loops.ts +++ b/packages/compiler/src/backend/llvm/dense-array-loops.ts @@ -20,16 +20,23 @@ * A counted loop with another invariant limit, such as `j < n` after * `const n = a.length` or `j < 5`, is covered the same way: the test bounds * the loop's exclusive limit by 2^31 - 1 and by each indexed array's - * `filled`, and the loop must provably never change any of their lengths. */ + * `filled`, and the loop must provably never change any of their lengths. + * + * A two-counter loop closing in on itself, such as + * `for (let i = 1, j = k - 1; i < j; i++, j--)`, indexes only between its + * low counter's start and its high counter's: the low counter only rises + * and stays below the high one, which only falls. Its test bounds the low + * start below by 0 and the high start by 2^31 - 2 and below each indexed + * array's `filled`, both starts being exact integers. */ import type { IrExpr, IrLocal, IrStmt } from "../../ir/ir.js"; import type { IntegerRange } from "../../ir/integer-ranges.js"; import { preservesArrayLength } from "../../ir/array-length-stability.js"; import { everyExprChild, everyStmtChild } from "../../ir/traverse.js"; -import type { IntegerCountedForLoop } from "../../ir/integer-loops.js"; +import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js"; import type { LlvmEmitterContext } from "./expr-context.js"; import { loadArrCount, SCR_ARR_FIELD } from "./common.js"; -import { emitCountedLoopLimit } from "./counted-loops.js"; +import { emitCountedLoopLimit, emitExactIntegerTest } from "./counted-loops.js"; export const INT32_MAX = 2147483647; @@ -47,6 +54,17 @@ export interface DenseArrayLoop { arrays: ReadonlyMap; /** The counter's range on every pass of the fast copy. */ range: IntegerRange; + /** A two-counter loop's high counter and both starts, when the counter + * is its low one (`length` and `limit` are then null). */ + pair: DensePair | null; +} + +/** The high counter of a two-counter dense loop, and the starts its entry + * test bounds. */ +export interface DensePair { + high: string; + lowStart: IrExpr; + highStart: IrExpr; } /** The fast copy duplicates the loop; keep the duplicated code small. */ @@ -62,6 +80,8 @@ export interface DenseLoopShape { /** The counter's proven range on every pass of the ordinary loop; the * fast copy keeps its minimum, which must be at least 0. */ range: IntegerRange; + /** A two-counter loop's high counter and starts. */ + pair?: DensePair; } /** `length.length - c` (c an integer literal, 0 when absent). */ @@ -107,6 +127,124 @@ export function countedDenseShape( return { counter: counted.localId, length: start.receiver, range: counted.range }; } +/** The shape of a two-counter loop whose low counter rises and high + * counter falls: both stay between the two starts, which the entry test + * bounds, so the fast copy's counters lie in [0, 2^31 - 2]. */ +export function pairedDenseShape(loop: IntegerPairedForLoop): DenseLoopShape { + return { + counter: loop.low.localId, + length: null, + range: { min: 0, max: INT32_MAX - 1 }, + pair: { high: loop.high.localId, lowStart: loop.low.start, highStart: loop.high.start }, + }; +} + +interface ElementRead { + arr: IrExpr; + index: IrExpr; +} + +/** The array and index of a call of an element-reader helper + * (`%arr.idxOr.*`, ir/int32-arrays.ts isElementReader): `a[i]` as + * `number | undefined`. */ +function elementReaderCall(e: IrExpr): ElementRead | null { + if (e.kind !== "call" || !e.callee.startsWith("%arr.idxOr.") || e.args.length !== 2) return null; + return { arr: e.args[0]!, index: e.args[1]! }; +} + +/** The array and index of an element read as `T | undefined`: an + * element-reader call, or its body inlined for an int32 array + * (`arrayState(a, i) === 1 ? wrap(a[i]) : wrap(undefined)`). */ +function elementRead(e: IrExpr): ElementRead | null { + const call = elementReaderCall(e); + if (call) return call; + if (e.kind !== "ternary") return null; + const cond = e.cond; + if (cond.kind !== "bin" || cond.op !== "===") return null; + const state = cond.left, + present = cond.right, + wrapped = e.then, + missing = e.else_; + if (state.kind !== "arrayState" || present.kind !== "numLit" || present.value !== 1) return null; + if (wrapped.kind !== "unionWrap" || missing.kind !== "unionWrap") return null; + const value = wrapped.value; + if (value.kind !== "arrayGet" || missing.value.kind !== "unitLit") return null; + const same = (x: IrExpr, y: IrExpr): boolean => + x.kind === "varRef" && y.kind === "varRef" && x.localId === y.localId; + return same(state.arr, value.arr) && same(state.index, value.index) + ? { arr: state.arr, index: state.index } + : null; +} + +/** The present-undefined stores of `a[i] = b[j]!` as lowering spells it, + * with the element read each one stores the missing arm of: + * + * const v = %arr.idxOr(b, j); + * if (v is undefined) a[i] = undefined; else a[i] = v is number ? v : throw; + * + * The read's local is declared once in the same list as the `if`, and only + * read after it. */ +function missingElementStores(body: IrStmt[]): Map { + const result = new Map(); + const writes = new Map(); + const count = (id: string): void => void writes.set(id, (writes.get(id) ?? 0) + 1); + const expr = (e: IrExpr): boolean => { + if (e.kind === "assignExpr" || e.kind === "incDec") count(e.localId); + return everyExprChild(e, expr, stmt); + }; + const stmt = (s: IrStmt): boolean => { + if (s.kind === "varDecl" || s.kind === "assign") count(s.localId); + return everyStmtChild(s, expr, stmt); + }; + body.every(stmt); + const list = (stmts: IrStmt[]): void => { + const reads = new Map(); + for (const s of stmts) { + if (s.kind === "varDecl" && s.init && writes.get(s.localId) === 1) { + const read = elementRead(s.init); + if (read) reads.set(s.localId, read); + } + if (s.kind === "if") { + const store = missingArmStore(s, reads); + if (store) result.set(store.stmt, store.read); + } + // Nested loops keep a loop out of the fast copy (matchDenseArrayLoop), + // and other statement lists leave their stores unmatched. + if (s.kind === "block") list(s.body); + if (s.kind === "if") { + list(s.then); + if (s.else_) list(s.else_); + } + } + }; + list(body); + return result; +} + +/** The present-undefined store of `if (v is undefined) a[i] = undefined; + * else a[i] = v is number ? v : throw;`, with the read `v` holds. */ +function missingArmStore( + s: IrStmt & { kind: "if" }, + reads: ReadonlyMap, +): { stmt: IrStmt; read: ElementRead } | null { + const cond = s.cond; + if (cond.kind !== "unionIsTag" || cond.negated || cond.value.kind !== "varRef") return null; + const local = cond.value.localId; + const read = reads.get(local); + if (!read || s.then.length !== 1 || s.else_?.length !== 1) return null; + const store = s.then[0]!, + other = s.else_[0]!; + if (store.kind !== "arraySetUndefined" || other.kind !== "arraySet") return null; + const value = other.value; + if (value.kind !== "ternary") return null; + const test = value.cond, + narrowed = value.then; + if (test.kind !== "unionIsTag" || test.negated || test.tag === cond.tag) return null; + if (test.value.kind !== "varRef" || test.value.localId !== local) return null; + if (narrowed.kind !== "unionNarrow" || narrowed.tag !== test.tag) return null; + return narrowed.type.kind === "f64" ? { stmt: store, read } : null; +} + /** Match the fast copy of a loop of the given shape. `initialized` holds the * bindings definitely assigned when the loop starts; `unaliased` the local * arrays no other binding holds. */ @@ -118,6 +256,8 @@ export function matchDenseArrayLoop( initialized: ReadonlySet, ): DenseArrayLoop | null { const { counter, length } = shape; + const high = shape.pair?.high ?? null; + const isCounter = (id: string): boolean => id === counter || id === high; if (shape.range.min < 0 || !loop.cond) return null; let nodes = 0, nested = false; @@ -126,13 +266,21 @@ export function matchDenseArrayLoop( // element write. const disturbed = new Set(); const access = (arr: IrExpr, index: IrExpr): void => { - if (arr.kind === "varRef" && index.kind === "varRef" && index.localId === counter) + if (arr.kind === "varRef" && index.kind === "varRef" && isCounter(index.localId)) candidates.set(arr.localId, arr); }; + const missing = missingElementStores(loop.body); + // The element reads behind the guarded present-undefined stores. + const guards: ElementRead[] = []; + // The arrays of the other present-undefined stores. The fast copy runs + // them, and the slot each one writes leaves the hole-free prefix. + const undefinedStores: IrExpr[] = []; const expr = (e: IrExpr): boolean => { if (++nodes > MAX_LOOP_NODES) return false; if (e.kind === "assignExpr" || e.kind === "incDec") disturbed.add(e.localId); if (e.kind === "arrayGet") access(e.arr, e.index); + const reader = elementReaderCall(e); + if (reader) access(reader.arr, reader.index); if (e.kind === "arrIntrinsic") { if (e.method === "getNumber" && e.args[0]) access(e.receiver, e.args[0]); else if (e.method !== "length" && e.receiver.kind === "varRef") @@ -147,9 +295,21 @@ export function matchDenseArrayLoop( if (s.kind === "assign" || s.kind === "varDecl") disturbed.add(s.localId); if (s.kind === "arraySet") { access(s.arr, s.index); - if (s.arr.kind === "varRef" && (s.index.kind !== "varRef" || s.index.localId !== counter)) + if (s.arr.kind === "varRef" && (s.index.kind !== "varRef" || !isCounter(s.index.localId))) disturbed.add(s.arr.localId); } + if (s.kind === "arraySetUndefined") { + // A present undefined at a counter only where it stores a missing + // element read (`a[i] = b[j]!`): the fast copy reads a present value + // there, so the store never runs in it. + const read = missing.get(s); + if (read === undefined || s.index.kind !== "varRef" || !isCounter(s.index.localId)) { + undefinedStores.push(s.arr); + } else { + access(s.arr, s.index); + guards.push(read); + } + } return everyStmtChild(s, expr, stmt); }; if (!loop.body.every(stmt) || !expr(loop.cond) || nested) return null; @@ -164,6 +324,15 @@ export function matchDenseArrayLoop( ) ) return null; + // A store through another name, such as `const b = a` or a field, lands + // in `a` unless one of the two bindings is a local array no other binding + // holds. + const mayStoreUndefined = (id: string): boolean => + undefinedStores.some((arr) => + arr.kind === "varRef" + ? arr.localId === id || (!unaliased.has(id) && !unaliased.has(arr.localId)) + : !unaliased.has(id), + ); const arrays = new Map(); for (const [id, ref] of candidates) { const local = locals.get(id); @@ -173,6 +342,7 @@ export function matchDenseArrayLoop( local.boxed || local.tdz || disturbed.has(id) || + mayStoreUndefined(id) || (id !== length?.localId && !initialized.has(id)) ) continue; @@ -180,18 +350,33 @@ export function matchDenseArrayLoop( // once: the loop must not change any of their lengths. if ( length === null && - !preservesArrayLength(nodesOfLoop, { receiver: id, inRange: counter }, locals, unaliased) + !preservesArrayLength( + nodesOfLoop, + { receiver: id, inRange: counter, inRangeHigh: high }, + locals, + unaliased, + ) ) continue; arrays.set(id, ref); } if (arrays.size === 0) return null; + // Each guarded store's read must be one the fast copy proves present. + for (const { arr, index } of guards) + if ( + arr.kind !== "varRef" || + !arrays.has(arr.localId) || + index.kind !== "varRef" || + !isCounter(index.localId) + ) + return null; return { counter, length, limit: length === null ? (shape.limit ?? null) : null, arrays, range: { min: shape.range.min, max: Math.min(shape.range.max, INT32_MAX - 1) }, + pair: shape.pair ?? null, }; } @@ -207,6 +392,7 @@ export function emitDenseLoopGuard(host: LlvmEmitterContext, loop: DenseArrayLoo host.markMemoryPointer(ptr, "array:header"); return ptr; }; + if (loop.pair !== null) return emitPairedDenseGuard(host, loop, loop.pair, header); if (loop.length === null) { // A limit at or below the start runs no pass, and passes the test. const limit = emitCountedLoopLimit(host, loop.limit!); @@ -254,3 +440,36 @@ export function emitDenseLoopGuard(host: LlvmEmitterContext, loop: DenseArrayLoo } return valid; } + +/** A two-counter loop's entry test: both starts are exact integers, the + * low one at least 0 and the high one at most 2^31 - 2 and below every + * indexed array's hole-free prefix bound. The starts are stable values + * (matchIntegerPairedForLoop), so evaluating them here again is unobservable. */ +function emitPairedDenseGuard( + host: LlvmEmitterContext, + loop: DenseArrayLoop, + pair: DensePair, + header: (array: string, field: number) => string, +): string { + const B = host.B; + const start = (expr: IrExpr, min: number, max: number) => + emitExactIntegerTest(host, host.emitExpr(expr).name, min, max); + const low = start(pair.lowStart, 0, INT32_MAX - 1); + const high = start(pair.highStart, -1, INT32_MAX - 1); + let valid = B.tmp(); + B.line(`${valid} = and i1 ${low.ok}, ${high.ok}`); + for (const ref of loop.arrays.values()) { + const array = host.emitStableReceiver(ref, []); + const filledPtr = header(array.name, SCR_ARR_FIELD.filled), + filled = B.tmp(), + wide = B.tmp(), + covered = B.tmp(), + both = B.tmp(); + B.line(`${filled} = load i32, ptr ${filledPtr}${host.fieldAliasAttachment(filledPtr)}`); + B.line(`${wide} = zext i32 ${filled} to i64`); + B.line(`${covered} = icmp slt i64 ${high.integer}, ${wide}`); + B.line(`${both} = and i1 ${valid}, ${covered}`); + valid = both; + } + return valid; +} diff --git a/packages/compiler/src/backend/llvm/dyn-handle-tags.test.ts b/packages/compiler/src/backend/llvm/dyn-handle-tags.test.ts index c015b85d0..8c3e5de80 100644 --- a/packages/compiler/src/backend/llvm/dyn-handle-tags.test.ts +++ b/packages/compiler/src/backend/llvm/dyn-handle-tags.test.ts @@ -1,7 +1,7 @@ import { readFileSync } from "node:fs"; import { expect, test } from "vitest"; import { DYN_HANDLE_KINDS } from "../../ir/ir.js"; -import { DYN_HANDLE_TAG_NUM } from "./dyn.js"; +import { DYN_HANDLE_TAG_NUM, TEXT_CODEC_TAG_NUM } from "./dyn.js"; // SCR_DYNH_TAGS in scr_dyn.h lists the ScrDynHandleTag values by name, each // with an explicit value, so changes that add tags edit different lines. The @@ -39,3 +39,9 @@ test("the backend's handle tag numbers match scr_dyn.h", () => { expect(values.get(tag!), `${kind} (${tag})`).toBe(num); } }); + +test("the backend's codec handle tag numbers match scr_dyn.h", () => { + const values = new Map(tags.map((tag) => [tag.name, tag.value])); + expect(values.get("SCR_DYNH_TEXT_DECODER")).toBe(TEXT_CODEC_TAG_NUM.TextDecoder); + expect(values.get("SCR_DYNH_TEXT_ENCODER")).toBe(TEXT_CODEC_TAG_NUM.TextEncoder); +}); diff --git a/packages/compiler/src/backend/llvm/dyn.ts b/packages/compiler/src/backend/llvm/dyn.ts index 2234b8db2..136798796 100644 --- a/packages/compiler/src/backend/llvm/dyn.ts +++ b/packages/compiler/src/backend/llvm/dyn.ts @@ -43,6 +43,8 @@ import { isDynTypedRefType, isRefCounted, isUnitType, + recordTextCodecClass, + textCodecRecordClass, typeKey, } from "../../ir/ir.js"; import { dynDesc, streamTypedRefEligible } from "../../ir/analysis.js"; @@ -94,6 +96,17 @@ export const DYN_HANDLE_TAG_NUM: Record = { cryptoHmac: 53, }; +/** The handle tags of TextEncoder and TextDecoder records boxed as dyn + * (scr_dyn.h; scr_text_codec_dyn.c boxes and unboxes them). */ +export const TEXT_CODEC_TAG_NUM: Record<"TextEncoder" | "TextDecoder", number> = { + TextDecoder: 76, + TextEncoder: 77, +}; + +/** The decoder record fields scr_text_decoder_to_dyn reads, in its + * parameter order. */ +const TEXT_DECODER_FIELDS = ["%TextDecoder", "%fatal", "%ignoreBOM", "%state"] as const; + /** Handle kinds whose runtime unit boxes them through its own function, * which installs the tag's ops on first use (scr_crypto.h). Programs that * never box one of these values do not link its dynamic members. */ @@ -370,6 +383,36 @@ export class LlDyn { B.line(`call void @scr_dyn_build_add(ptr %d, ptr @${name}, ptr ${value})`); } + /** The order a dyn value tries a union's arms in: the canonical order, + * except that among the record arms, those with more required fields go + * first, then those with more fields. Records are width-tolerant, so an + * arm with fewer fields (`{}`, or one whose fields are all optional, + * matches every object) would otherwise take an object that a wider arm + * also matches and drop the fields only the wider arm has. */ + private unionArmOrder(def: IrUnionDef): number[] { + const width = (i: number): [number, number] | null => { + const arm = def.arms[i]!; + if (arm.kind !== "record") return null; + const shape = this.host.recordsById.get(arm.shapeId); + if (!shape || shape.tuple) return null; + const required = shape.fields.filter( + (field) => + field.type.kind !== "union" || + !this.host.unionsById.get(field.type.unionId)?.arms.some((x) => x.kind === "undefinedT"), + ).length; + return [required, shape.fields.length]; + }; + const order = def.arms.map((_, i) => i); + const records = order.filter((i) => width(i) !== null); + const widest = [...records].sort((x, y) => { + const [rx, fx] = width(x)!; + const [ry, fy] = width(y)!; + return ry - rx || fy - fx || x - y; + }); + records.forEach((slot, k) => (order[slot] = widest[k]!)); + return order; + } + /** Branch to a variant only after its literal discriminator and complete * structure match. Use the same path for predicates and builders. */ private unionArmMatch( @@ -897,6 +940,21 @@ export class LlDyn { const name = `sc_dm_${this.dynMatchers.size}`; this.dynMatchers.set(matcherKey, name); this.host.journalAllocated?.(name, "dyn.dynMatchHelper", [t, preserveRefs]); + const codec = textCodecRecordClass(t, (id) => this.host.recordsById.get(id)); + if (codec !== null) { + // A codec matches only its own handle box. + const B = new BlockBuilder(); + const matched = this.handleTagMatches(B, TEXT_CODEC_TAG_NUM[codec]); + B.terminate(`ret i1 ${matched}`); + this.defs.push( + `define internal zeroext i1 @${name}(ptr %d) ${FN_ATTRS} { ; matches ${key}`, + B.render(), + `}`, + ``, + ); + this.host.journalBuilt?.(); + return name; + } const recursive = this.isRecursiveShape(t); const wrapped = recursive || this.holdsRecursiveShape(t); const B = new BlockBuilder(); @@ -1626,6 +1684,24 @@ export class LlDyn { this.host.journalBuilt?.(); return name; } + const codec = textCodecRecordClass(t, (id) => host.recordsById.get(id)); + if (codec !== null) { + // The record the codec's handle box holds (+1), or the path TypeError. + host.declare(`declare ptr @scr_text_codec_from_dyn(ptr, i1 zeroext, ptr, ptr)`); + const checked = B.tmp(); + B.line( + `${checked} = call ptr @scr_text_codec_from_dyn(ptr %d, i1 ${codec === "TextDecoder"}, ptr %path, ptr ${want})`, + ); + B.terminate(`ret ptr ${checked}`); + this.defs.push( + `define internal ptr @${name}(ptr %d, ptr %path) ${FN_ATTRS} {`, + B.render(), + `}`, + ``, + ); + this.host.journalBuilt?.(); + return name; + } if (isRefCounted(t) && t.kind !== "dyn") { host.declare(`declare ptr @scr_dyn_typed_ref_unbox(ptr)`); if (t.kind === "object" && t.className === "%Error") { @@ -2381,9 +2457,10 @@ export class LlDyn { throw new InternalCompilerError( `llvm emitter bug: dynCheck of unknown union ${t.unionId}`, ); - // Arms in CANONICAL order, first FULL match wins. The matched - // arm's builder can no longer fail. - def.arms.forEach((arm, i) => { + // Arms in unionArmOrder, first FULL match wins. The matched arm's + // builder can no longer fail. + for (const i of this.unionArmOrder(def)) { + const arm = def.arms[i]!; const lHit = B.newLabel("dcu.h"); const lNext = B.newLabel("dcu.n"); this.unionArmMatch(B, def, i, lHit, lNext, preserveRefs); @@ -2427,7 +2504,7 @@ export class LlDyn { B.terminate(`ret ptr ${u}`); } B.startBlock(lNext); - }); + } if (preserveRefs) { B.line( `call void @scr_dyn_check_fail(ptr %path, ptr ${host.cstr("reference with its original layout")}, ptr %d)`, @@ -2653,6 +2730,77 @@ export class LlDyn { return name; } + /** `i1`: %d is a handle box with tag `tag`. */ + private handleTagMatches(B: BlockBuilder, tag: number): string { + const kd = this.kindOf(B, "%d"); + const isHandle = B.tmp(); + B.line(`${isHandle} = icmp eq i32 ${kd}, ${DYN_KIND.HANDLE}`); + const lHandle = B.newLabel("dm.handle"); + const lDone = B.newLabel("dm.handle.done"); + const entry = B.currentLabel(); + B.condBr(isHandle, lHandle, lDone); + B.startBlock(lHandle); + const tagPtr = B.tmp(); + const loaded = B.tmp(); + B.line( + `${tagPtr} = getelementptr inbounds i8, ptr %d, i64 ${this.abiOffset(24, 20)} ; ->v.handle.tag`, + ); + B.line(`${loaded} = load i32, ptr ${tagPtr}`); + const same = B.tmp(); + B.line(`${same} = icmp eq i32 ${loaded}, ${tag}`); + B.br(lDone); + B.startBlock(lDone); + const matched = B.tmp(); + B.line(`${matched} = phi i1 [ false, %${entry} ], [ ${same}, %${lHandle} ]`); + return matched; + } + + /** A TextEncoder or TextDecoder record boxes as its handle (+1), which + * holds the record by reference. A decoder passes the byte offsets of the + * fields its boxed decode reads. */ + private textCodecToDyn( + B: BlockBuilder, + t: IrType & { kind: "record" }, + shape: IrRecordShape, + codec: "TextEncoder" | "TextDecoder", + ): void { + const host = this.host; + const rc = vAdapters(host, t); + const r = B.tmp(); + if (codec === "TextEncoder") { + host.declare(`declare ptr @scr_text_encoder_to_dyn(ptr, ptr, ptr)`); + B.line( + `${r} = call ptr @scr_text_encoder_to_dyn(ptr %v, ptr ${rc.retain}, ptr ${rc.release})`, + ); + } else { + const struct = mangleRecordStruct(t.shapeId); + const base = host.nullableUnions.fieldBase(t.shapeId); + const start = B.tmp(); + B.line(`${start} = ptrtoint ptr %v to ${this.S}`); + const offsets = TEXT_DECODER_FIELDS.map((field) => { + const index = shape.fields.findIndex((f) => f.name === field); + if (index < 0) + throw new InternalCompilerError(`llvm emitter bug: TextDecoder record lacks ${field}`); + const p = B.tmp(); + const at = B.tmp(); + const offset = B.tmp(); + B.line( + `${p} = getelementptr inbounds %${struct}, ptr %v, i64 0, i32 ${index + base} ; .${llvmCommentText(field)}`, + ); + B.line(`${at} = ptrtoint ptr ${p} to ${this.S}`); + B.line(`${offset} = sub ${this.S} ${at}, ${start}`); + return `${this.S} ${offset}`; + }); + host.declare( + `declare ptr @scr_text_decoder_to_dyn(ptr, ptr, ptr, ${this.S}, ${this.S}, ${this.S}, ${this.S})`, + ); + B.line( + `${r} = call ptr @scr_text_decoder_to_dyn(ptr %v, ptr ${rc.retain}, ptr ${rc.release}, ${offsets.join(", ")})`, + ); + } + B.terminate(`ret ptr ${r}`); + } + /* ── toDynHelper (walkers.ts, ported) ─────────────────────────── */ /** `sc_td_( v) -> ptr` — build a fresh dyn value from a @@ -2828,6 +2976,11 @@ export class LlDyn { const shape = host.recordsById.get(t.shapeId); if (!shape) throw new InternalCompilerError(`llvm emitter bug: to-dyn of unknown shape ${t.shapeId}`); + const codec = recordTextCodecClass(shape); + if (codec !== null) { + this.textCodecToDyn(B, t, shape, codec); + break; + } const struct = mangleRecordStruct(t.shapeId); const base = host.nullableUnions.fieldBase(t.shapeId); const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + base])); diff --git a/packages/compiler/src/backend/llvm/emit-statements.ts b/packages/compiler/src/backend/llvm/emit-statements.ts index c2846a9d4..55e31219c 100644 --- a/packages/compiler/src/backend/llvm/emit-statements.ts +++ b/packages/compiler/src/backend/llvm/emit-statements.ts @@ -52,13 +52,19 @@ import { matchIntegerPairedForLoop, } from "../../ir/integer-loops.js"; import { emitByteWindowGuard, matchByteWindow } from "./byte-windows.js"; -import { countedDenseShape, emitDenseLoopGuard, matchDenseArrayLoop } from "./dense-array-loops.js"; +import { + countedDenseShape, + emitDenseLoopGuard, + matchDenseArrayLoop, + pairedDenseShape, +} from "./dense-array-loops.js"; import { emitCountedLoopLimit, emitCountedLoopSafety, emitPairedLoopSafety, } from "./counted-loops.js"; import { emitBytesLoopVersionGuard, type BytesLoopVersion } from "./bytes-loop-versions.js"; +import { bytesHeaderAlias } from "./bytes-alias.js"; import { mangleLocal } from "../mangle.js"; import { LlvmUnsupportedError } from "./unsupported.js"; import type { LlValue } from "./expr-context.js"; @@ -966,7 +972,9 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void { } : countedLoop && !countedLoop.guarded && countedLoop.followers.length === 0 ? countedDenseShape(s, countedLoop) - : null; + : pairedLoop + ? pairedDenseShape(pairedLoop) + : null; const dense = denseShape && !window ? matchDenseArrayLoop( @@ -1043,15 +1051,17 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void { LOOPS.of(emitter).unaliasedArrays, ) : countedLoop; - const version = versionable ? emitter.matchBytesLoopVersion(s, versionCounted) : null; + const version = versionable + ? emitter.matchBytesLoopVersion(s, versionCounted, versionCounted ? null : pairedLoop) + : null; if (version) { emitVersionedLoop(emitter, s, version); break; } let countedJoin: string | null = null; if ( - (countedLoop?.guarded && !LOOPS.of(emitter).provenCountedGuards.has(s)) || - pairedLoop?.guarded + (countedLoop?.guarded || pairedLoop?.guarded) && + !LOOPS.of(emitter).provenCountedGuards.has(s) ) { const safe = countedLoop ? emitCountedLoopSafety(emitter, countedLoop) @@ -1144,11 +1154,15 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void { B.line(`${start} = fptosi double ${value.name} to i64`); B.line(`store i64 ${start}, ptr ${slot}`); pairedSlots.push(slot); + const dense = LOOPS.of(emitter).denseLoop; emitter.integerLoopBindings.set(counter.localId, { slot, type: "i64", signed: true, - range: pairedLoop.range, + range: + dense?.pair && dense.counter === pairedLoop.low.localId + ? dense.range + : pairedLoop.range, }); } } else if (s.init) { @@ -1218,10 +1232,7 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void { } B.line(`${inBounds} = icmp ult i32 ${narrow}, ${len}`); } else { - emitter.markMemoryPointer(lenPtr, "bytes:header"); - B.line( - `${len} = load ${emitter.sizeType}, ptr ${lenPtr}${emitter.fieldAliasAttachment(lenPtr)}`, - ); + B.line(`${len} = load ${emitter.sizeType}, ptr ${lenPtr}${bytesHeaderAlias(emitter)}`); B.line(`${inBounds} = icmp ult ${emitter.sizeType} ${index}, ${len}`); } B.condBr(inBounds, lb, exit); @@ -2085,7 +2096,8 @@ function emitVersionedLoop( if (version.owned.size > 0) emitter.bytesOwned = new Set([...originalOwned, ...version.owned]); if (version.ranges.size > 0) emitter.integerRanges = new Map([...originalRanges, ...version.ranges]); - if (version.counted?.guarded) LOOPS.of(emitter).provenCountedGuards.add(s); + if (version.counted?.guarded || version.paired?.loop.guarded) + LOOPS.of(emitter).provenCountedGuards.add(s); emitter.emitStmt(s); B.br(done); } finally { diff --git a/packages/compiler/src/backend/llvm/emitter.test.ts b/packages/compiler/src/backend/llvm/emitter.test.ts index d1fd62a83..e6334ab4c 100644 --- a/packages/compiler/src/backend/llvm/emitter.test.ts +++ b/packages/compiler/src/backend/llvm/emitter.test.ts @@ -459,6 +459,33 @@ test("class brand probes borrow a local only across an inert literal key", () => expect(computed).toContain("call ptr @scr_dyn_retain_v("); }); +test("class constructor probes test the function tag and borrow across a literal key", () => { + const dyn: IrType = { kind: "dyn" }; + const key: IrExpr = { kind: "strLit", value: "classval:Example", type: STRING, loc: receiverLoc }; + const probe: IrExpr = { + kind: "libCall", + fn: "dyn.classIs", + args: [ref(dyn), key], + type: BOOL, + loc: receiverLoc, + }; + const borrowed = work(probe, dyn); + expect(borrowed).not.toContain("call ptr @scr_dyn_retain_v("); + expect(borrowed).toContain("icmp ne ptr"); + expect(borrowed).toContain("icmp eq i32 %t"); + expect(borrowed).toMatch(/icmp eq i32 %t\d+, 8\b/); + expect(borrowed).toContain("call zeroext i1 @scr_dyn_class_is_key("); + const computed = work( + { + ...probe, + args: [ref(dyn), { kind: "toString", operand: ref(dyn), type: STRING, loc: receiverLoc }], + }, + dyn, + ); + expect(computed).toContain("call ptr @scr_dyn_retain_v("); + expect(computed).toContain("call zeroext i1 @scr_dyn_class_is_key("); +}); + test("class brand probes compare the program's key constant before the key text", () => { const dyn: IrType = { kind: "dyn" }; const key: IrExpr = { kind: "strLit", value: "object:Example", type: STRING, loc: receiverLoc }; @@ -472,6 +499,27 @@ test("class brand probes compare the program's key constant before the key text" const named = work(probe("dyn.typedRefIs"), dyn); expect(named).toContain("; ->v.typed_ref.type_key"); expect(named).toContain("@scr_dyn_typed_ref_is_key"); + // Another key pointer is rejected on its length and last eight bytes + // ("Example" plus the prefix's ':') before the out-of-line comparison. + expect(named).toContain("; ->v.typed_ref.type_key_len"); + expect(named).toContain("and i32 %t"); + expect(named).toMatch(/icmp eq i32 %t\d+, 14\b/); + const tail = Buffer.from("object:Example").readBigInt64LE(6); + expect(named).toContain(`icmp eq i64 %t`); + expect(named).toContain(`, ${tail}`); + const shortKey: IrExpr = { kind: "strLit", value: "o:E", type: STRING, loc: receiverLoc }; + const short = work( + { + kind: "libCall", + fn: "dyn.typedRefIs", + args: [ref(dyn), shortKey], + type: BOOL, + loc: receiverLoc, + }, + dyn, + ); + expect(short).toMatch(/icmp eq i32 %t\d+, 3\b/); + expect(short).not.toContain("load i64"); // The key-pointer test answers false for capsules boxed elsewhere. const exact = work(probe("dyn.typedRefKeyIs"), dyn); expect(exact).toContain("; ->v.typed_ref.type_key"); diff --git a/packages/compiler/src/backend/llvm/emitter.ts b/packages/compiler/src/backend/llvm/emitter.ts index c0cc9fef2..b64c244b0 100644 --- a/packages/compiler/src/backend/llvm/emitter.ts +++ b/packages/compiler/src/backend/llvm/emitter.ts @@ -60,7 +60,7 @@ import { captureFreeFunctions, observesStacks } from "../stack-observations.js"; import { findInitializedByteLoopBindings } from "./byte-windows.js"; import { matchBytesLoopVersion } from "./bytes-loop-versions.js"; import { globalSlotAlias } from "./bytes-alias.js"; -import type { IntegerCountedForLoop } from "../../ir/integer-loops.js"; +import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js"; import { materializeSplitPiece, type SplitSpan, @@ -68,7 +68,7 @@ import { type StringSplit, } from "./split-loops.js"; import { scalarizeNumericRecords } from "../../ir/scalar-records.js"; -import { addRecordResultEntries } from "./record-results.js"; +import { addRecordResultEntries, resultEntryName } from "./record-results.js"; import { specializeCallbackCalls } from "../../ir/callback-specialization.js"; import { type ChunkResult, @@ -82,6 +82,7 @@ import { versionLoopBoundParameters } from "../../ir/parameter-versions.js"; import { allocatedLiteral, fillAllocationLength, + int32ArrayKey, selectInt32Arrays, } from "../../ir/int32-arrays.js"; import { everyExpr, everyExprChild, everyStmtChild, everyStmtList } from "../../ir/traverse.js"; @@ -169,6 +170,7 @@ import { type MapEntryFusion } from "./map-entry-fusion.js"; import { emitBorrowedFieldSequence, guardedValue } from "./borrowed-receivers.js"; import { emitBorrowedInputs } from "./borrowed-inputs.js"; import { findWalkBorrows, type WalkBorrowHost } from "./walk-borrows.js"; +import { IrNodeIndex } from "../../ir/node-index.js"; import { lastUseRead } from "./local-extents.js"; import { findReboundParameters } from "./rebound-parameters.js"; import { ReferenceEffects } from "./reference-effects.js"; @@ -184,7 +186,7 @@ import { } from "./string-slices.js"; import { LazyCaptures } from "./lazy-captures.js"; import { StackCaptures } from "./stack-captures.js"; -import { emitFieldAbsentTest, f64Lit, SCR_ARR_FIELD } from "./common.js"; +import { emitFieldAbsentTest, f64Lit, SCR_ARR_FIELD, SCR_ARR_FROZEN } from "./common.js"; import { ffiExtendsNarrowIntegers } from "../targets.js"; import { emitLiteralExpr, @@ -509,6 +511,15 @@ function provenInt32Arrays(mod: IrModule): ReadonlySet { return new Set([...slots.int32Arrays, ...slots.checkedInt32Arrays]); } +/** The functions whose calls the backend may emit as calls of their + * result-slot entries (record-results.ts). */ +function resultEntryTargets(mod: IrModule): ReadonlySet { + const names = new Set(mod.functions.map((fn) => fn.name)); + const targets = new Set(); + for (const name of names) if (names.has(resultEntryName(name))) targets.add(name); + return targets; +} + /** The emitter for `input` after the module passes every emission runs. */ function moduleEmitter(input: IrModule, options: LlvmTargetOptions): LlEmitter { // Facts are keyed by node identity: emit the same module whether it was @@ -544,6 +555,7 @@ function moduleEmitter(input: IrModule, options: LlvmTargetOptions): LlEmitter { ), ), provenInt32Arrays, + resultEntryTargets, ) : mod, options, @@ -780,7 +792,7 @@ export class LlEmitter { everyStmtList(fn.body, { stmt: (s) => { if (s.kind !== "varDecl" && s.kind !== "assign") return true; - const key = `${fn.name}\0${s.localId}`; + const key = int32ArrayKey(fn.name, s.localId); const checked = checkedInt32Arrays.has(key); if (!checked && !int32Arrays.has(key)) return true; const value = s.kind === "varDecl" ? s.init : s.value; @@ -798,15 +810,16 @@ export class LlEmitter { }); } - /** Whether `arr` is one of the current function's int32 arrays - * (ir/int32-arrays.ts, kept by the int32 slot analysis). */ + /** Whether `arr` reads one of the int32 arrays (ir/int32-arrays.ts, kept + * by the int32 slot analysis): a local or parameter of the current + * function, or a module global. */ int32Array(arr: IrExpr): boolean { if (arr.kind !== "varRef") return false; if (this.int32KindArray(arr)) return true; if (this.int32Slots.int32Arrays.size === 0) return false; return ( this.int32ArrayTemps.has(arr.localId) || - this.int32Slots.int32Arrays.has(`${this.currentFunctionName}\0${arr.localId}`) + this.int32Slots.int32Arrays.has(int32ArrayKey(this.currentFunctionName, arr.localId)) ); } @@ -973,8 +986,8 @@ export class LlEmitter { } /** Whether an element access at `index` of `arr` lies in the hole-free - * prefix: the counter of a dense array loop's fast copy indexing one of - * the arrays its entry test covered. */ + * prefix: a counter of a dense array loop's fast copy indexing one of the + * arrays its entry test covered. */ provenDenseIndex(arr: IrExpr, index: IrExpr): boolean { const loop = LOOPS.of(this).denseLoop; return ( @@ -982,8 +995,8 @@ export class LlEmitter { arr.kind === "varRef" && loop.arrays.has(arr.localId) && index.kind === "varRef" && - index.localId === loop.counter && - this.integerLoopBindings.has(loop.counter) + (index.localId === loop.counter || index.localId === loop.pair?.high) && + this.integerLoopBindings.has(index.localId) ); } @@ -1183,6 +1196,9 @@ export class LlEmitter { /** Module constants that keep one value once initialized * (immutability.ts stableGlobals). */ stableGlobals: ReadonlySet = new Set(); + /** Every function body's nodes in walk order (ir/node-index.ts), for the + * constructor's analyses. */ + readonly nodeIndex: IrNodeIndex; private readonly tdzGlobals = new Map(); readonly constantNumericTables: ReadonlyMap; /** Constant-key reads on checked-dynamic objects, each with its own @@ -1251,7 +1267,7 @@ export class LlEmitter { /** The module's runtime features (moduleRuntimeFeatures), found with one * walk of the module the first time any of them is read. */ get runtimeFeatures(): RuntimeFeatures { - return (this.runtimeFeaturesValue ??= moduleRuntimeFeatures(this.mod)); + return (this.runtimeFeaturesValue ??= moduleRuntimeFeatures(this.mod, this.nodeIndex)); } /** Whether timer and tick scheduling goes through the *_als entry * points (RuntimeFeatures asyncContext). */ @@ -1548,8 +1564,15 @@ export class LlEmitter { ((options.functionWorkers ?? 1) > 1 && options.forkPrimitives !== undefined && options.debugSources === undefined); - this.stackTraces = observesStacks(mod); - this.nullableUnions = new NullableUnions(mod, options.debugSources === undefined); + // One walk of every body, which the analyses below read instead of + // walking the bodies themselves. + this.nodeIndex = new IrNodeIndex(mod); + this.stackTraces = observesStacks(mod, this.nodeIndex); + this.nullableUnions = new NullableUnions( + mod, + options.debugSources === undefined, + this.nodeIndex, + ); this.mockTimers = this.runtimeFeatures.mockTimers; this.debug = options.debugSources === undefined @@ -1562,19 +1585,20 @@ export class LlEmitter { this.nullableUnions.armTypes(), ); this.constantNumericTables = findConstantNumericTables(mod); - this.initializerBindings = findInitializerBindings(mod); + this.initializerBindings = findInitializerBindings(mod, this.nodeIndex); // One walk finds the dynamic field stores every analysis below consults. - const dynamicStores = dynamicFieldStores(mod); + const dynamicStores = dynamicFieldStores(mod, undefined, undefined, this.nodeIndex); this.int32Slots = analyzeInt32Slots(mod, { dynamicStores, initializerBindings: this.initializerBindings, + nodes: this.nodeIndex, }); this.liveViewsReadOnly = mod.lib === undefined && !dynamicStores.all && dynamicStores.names.size === 0; this.int32Abis = this.debug === null ? findInt32Abis(mod, this.int32Slots) : new Map(); this.immortalGlobals = new Set(mod.contracts?.immortalGlobals ?? []); this.programLifetime = new Set(mod.contracts?.programLifetime ?? []); - this.finalFields = analyzeFinalFields(mod, false, dynamicStores); + this.finalFields = analyzeFinalFields(mod, false, dynamicStores, this.nodeIndex); this.sizeType = options.pointerBits === 32 ? "i32" : "i64"; this.ffiExtendNarrowIntegers = options.wasi === true || ffiExtendsNarrowIntegers(options.targetTriple); @@ -1622,7 +1646,7 @@ export class LlEmitter { this.ffiHasRetainedCallback = hasRetainedFfiCallback(mod.ffiImports ?? []); this.ffiHasForeignCallback = hasForeignFfiCallback(mod.ffiImports ?? []); for (const fn of mod.functions) this.fnByName.set(fn.name, fn); - this.appendedArrayLiterals = findAppendedArrayLiterals(mod); + this.appendedArrayLiterals = findAppendedArrayLiterals(mod, this.nodeIndex); for (const entry of mod.ffiImports ?? []) { this.ffiByName.set(entry.name, entry); } @@ -1630,10 +1654,11 @@ export class LlEmitter { // and per async fiber; WASI, Windows and library builds have none. const triple = (options.targetTriple ?? "").toLowerCase(); const windows = triple === "" ? process.platform === "win32" : triple.includes("windows"); - this.publishedTypes = computePublishedTypes(mod); + this.publishedTypes = computePublishedTypes(mod, this.nodeIndex); const mt = computeMayThrow(mod, { stackChecks: !this.wasi && mod.lib === undefined && !windows, published: this.publishedTypes, + nodes: this.nodeIndex, }); this.mayThrow = mt.fns; this.workerEntryPolls = mt.workerEntryPolls ?? null; @@ -1692,12 +1717,14 @@ export class LlEmitter { new Set((mod.globals ?? []).map((g) => g.id)), { unions: mod.unions, records: mod.records }, this.immutableFields, + this.nodeIndex, ); const walkParameters = new Map>(); if (this.debug === null) { const host = { pointerLocal: (local: IrLocal) => this.plainReference(local.type), borrowsWithoutOwning: (e: IrExpr) => this.borrowsWithoutOwning(e), + nodes: this.nodeIndex, }; for (const fn of this.fnByName.values()) { const preserving = this.referenceEffects.functions.has(fn.name); @@ -1750,7 +1777,7 @@ export class LlEmitter { ); this.lentStringParameters = this.debug === null ? findLentStringParameters(this.fnByName, this.callLifetimes) : new Map(); - this.constantCallbacks = findConstantCallbacks(mod, this.callLifetimes); + this.constantCallbacks = findConstantCallbacks(mod, this.callLifetimes, this.nodeIndex); // Integer clones signal a stack overflow through their result, which // needs the single-threaded guard; worker programs also poll for // termination, and debug builds and stack traces keep every frame. @@ -1892,7 +1919,7 @@ export class LlEmitter { this.globalTypes.set(g.id, g.type); if (g.tdz) this.tdzGlobals.set(g.id, g.name); } - this.stableGlobals = stableGlobals(mod); + this.stableGlobals = stableGlobals(mod, this.nodeIndex); // Runtime error, EventEmitter, stream, EventTarget and Event classes // have known layouts. Subclasses embed the runtime prefixes from // classes.ts. @@ -4236,6 +4263,7 @@ export class LlEmitter { matchBytesLoopVersion( s: IrStmt & { kind: "for" | "while" }, counted: IntegerCountedForLoop | null, + paired: IntegerPairedForLoop | null = null, ): ReturnType { const fn = LOOPS.of(this).bytesLoopFunction; if (!fn) return null; @@ -4245,17 +4273,22 @@ export class LlEmitter { const invariantGlobals = new Set(this.stableGlobals); for (const global of this.initializerBindings.get(fn.name) ?? []) invariantGlobals.add(global.id); - return matchBytesLoopVersion(s, counted, { - locals: this.currentLocals, - invariantGlobals, - captured: this.captureIds, - ranges: this.integerRanges, - initialized, - proven: this.bytesBounds, - workers: this.mod.workers === true, - owned: this.bytesOwned, - scalarStrings: new Set(this.stringSlices.keys()), - }); + return matchBytesLoopVersion( + s, + counted, + { + locals: this.currentLocals, + invariantGlobals, + captured: this.captureIds, + ranges: this.integerRanges, + initialized, + proven: this.bytesBounds, + workers: this.mod.workers === true, + owned: this.bytesOwned, + scalarStrings: new Set(this.stringSlices.keys()), + }, + paired, + ); } /** The alias tag of a whole-slot load of a binding (module globals only). */ @@ -6264,6 +6297,41 @@ export class LlEmitter { return ok; } + /** Branches on "the array `arr` is frozen" (SCR_ARR_FROZEN in its storage + * flags, scr_array.h), or published too when `published`; like + * emitPublishedTest, leaves the builder in the cold block and answers the + * continuation label. */ + private emitFrozenArrayTest(arr: string, published: boolean): string { + const B = this.B; + const sparep = B.tmp(), + spare = B.tmp(), + bit = B.tmp(); + let frozen = B.tmp(); + B.line( + `${sparep} = getelementptr inbounds %ScrArr, ptr ${arr}, i32 0, i32 ${SCR_ARR_FIELD.spare}`, + ); + B.line(`${spare} = load i16, ptr ${sparep}`); + B.line(`${bit} = and i16 ${spare}, ${SCR_ARR_FROZEN}`); + B.line(`${frozen} = icmp ne i16 ${bit}, 0`); + if (published) { + const rc = B.tmp(), + immortal = B.tmp(), + either = B.tmp(); + B.line(`${rc} = load ${this.sizeType}, ptr ${arr}`); + B.line(`${immortal} = icmp eq ${this.sizeType} ${rc}, -1`); + B.line(`${either} = or i1 ${frozen}, ${immortal}`); + frozen = either; + } + this.declare(`declare i1 @llvm.expect.i1(i1, i1)`); + const expected = B.tmp(); + B.line(`${expected} = call i1 @llvm.expect.i1(i1 ${frozen}, i1 false)`); + const cold = B.newLabel("arr.frozen"), + ok = B.newLabel("arr.ok"); + B.condBr(expected, cold, ok); + B.startBlock(cold); + return ok; + } + private publishedNamesSym: string | null = null; /** The class-name table the hierarchy field guards index by preorder. */ @@ -6328,15 +6396,29 @@ export class LlEmitter { op: number, index = f64Lit(0), count = f64Lit(0), + third: string | null = null, ): void { - if (this.publishedTypes?.containerGuarded(arrType) !== true) return; + const published = this.publishedTypes; + if (published?.containerGuarded(arrType) !== true) return; const B = this.B; - const ok = this.emitPublishedTest(arr); - this.declare(`declare zeroext i1 @scr_throw_published_array(i32, ptr, double, double)`); + const guards = published.arrayGuards(arrType); + const ok = guards.frozen + ? this.emitFrozenArrayTest(arr, guards.published) + : this.emitPublishedTest(arr); const threw = B.tmp(); - B.line( - `${threw} = call zeroext i1 @scr_throw_published_array(i32 ${op}, ptr ${arr}, double ${index}, double ${count})`, - ); + if (third === null) { + this.declare(`declare zeroext i1 @scr_throw_published_array(i32, ptr, double, double)`); + B.line( + `${threw} = call zeroext i1 @scr_throw_published_array(i32 ${op}, ptr ${arr}, double ${index}, double ${count})`, + ); + } else { + this.declare( + `declare zeroext i1 @scr_throw_published_array_range(i32, ptr, double, double, double)`, + ); + B.line( + `${threw} = call zeroext i1 @scr_throw_published_array_range(i32 ${op}, ptr ${arr}, double ${index}, double ${count}, double ${third})`, + ); + } const unwind = B.newLabel("pub.throw"); B.condBr(threw, unwind, ok); B.startBlock(unwind); diff --git a/packages/compiler/src/backend/llvm/expr-calls.ts b/packages/compiler/src/backend/llvm/expr-calls.ts index a4e7cde2c..6ff52619e 100644 --- a/packages/compiler/src/backend/llvm/expr-calls.ts +++ b/packages/compiler/src/backend/llvm/expr-calls.ts @@ -44,6 +44,7 @@ import { LlvmUnsupportedError } from "./unsupported.js"; import type { LlvmEmitterContext, ExprOf, LlValue } from "./expr-context.js"; import { f64Lit, ffiNativeTypeLl, ffiNativeParamLl, ffiNativeReturnLl } from "./common.js"; import { canStackUnion, emitStackUnion } from "./stack-unions.js"; +import { emitStackRecordKeyGet, scalarRecordKeyGet } from "./expr-records.js"; import { emitCallArrayRead } from "./local-array-reads.js"; import { emitScalarOptional, isScalarOptionalSource } from "./local-union-storage.js"; import { emitStackMapRead } from "./map-read-lifetimes.js"; @@ -166,6 +167,9 @@ export function emitCallExpr( return { name: host.emitOwnedNullableStack(a).box, type: a.type }; // An inline union field read copies its tag and bits. if (host.inlineFieldGet(a)) return host.emitReadReceiver(a); + // A scalar keyed record read writes its tag and bits. + if (a.kind === "recordKeyGet" && scalarRecordKeyGet(host, a)) + return { name: emitStackRecordKeyGet(host, a), type: a.type }; } if (inputs[index]) return host.emitReadReceiver(a); } diff --git a/packages/compiler/src/backend/llvm/expr-containers.ts b/packages/compiler/src/backend/llvm/expr-containers.ts index 1ba5da818..93cc615a4 100644 --- a/packages/compiler/src/backend/llvm/expr-containers.ts +++ b/packages/compiler/src/backend/llvm/expr-containers.ts @@ -257,6 +257,16 @@ function emitDiscarded(host: LlvmEmitterContext, e: IrExpr): void { const start = host.emitExpr(e.args[0]!); const count = e.args[1] ? host.emitExpr(e.args[1]).name : F64_INF; const values = items.map((item) => host.emitExpr(item)); + // Before the values move into the buffer: a throw releases them with the + // frame. + host.emitPublishedArrayGuard( + r.name, + e.receiver.type, + 10, + start.name, + count, + f64Lit(values.length), + ); if (acc === "ref") values.forEach((value) => host.toHeap(value)); let buffer = "null"; if (values.length > 0) { @@ -978,7 +988,25 @@ export function emitArrIntrinsic( case "fill": case "fillUndefined": { const args = e.args.map((arg) => host.emitExpr(arg)); - host.emitPublishedArrayGuard(r.name, e.receiver.type, 7); + // fill's range is its last two arguments; copyWithin's target, + // start and end are its three. + if (e.method === "copyWithin") + host.emitPublishedArrayGuard( + r.name, + e.receiver.type, + 9, + args[0]!.name, + args[1]!.name, + args[2]!.name, + ); + else + host.emitPublishedArrayGuard( + r.name, + e.receiver.type, + 8, + args[args.length - 2]!.name, + args[args.length - 1]!.name, + ); if (host.checkedFills.has(e)) { // A new int32 array with checked stores, filled with a value that // may not be an int32: widen it first then. @@ -1042,7 +1070,7 @@ export function emitArrIntrinsic( // removes to the end (+Infinity, the slice convention). const start = host.emitExpr(e.args[0]!); const cnt = e.args[1] ? host.emitExpr(e.args[1]).name : F64_INF; - host.emitPublishedArrayGuard(r.name, e.receiver.type, 7); + host.emitPublishedArrayGuard(r.name, e.receiver.type, 10, start.name, cnt, f64Lit(0)); host.declare(`declare ptr @scr_arr_splice(ptr, double, double)`); const t = B.tmp(); B.line(`${t} = call ptr @scr_arr_splice(ptr ${r.name}, double ${start.name}, double ${cnt})`); @@ -1052,7 +1080,12 @@ export function emitArrIntrinsic( const start = host.emitExpr(e.args[0]!); const count = host.emitExpr(e.args[1]!); const items = emitBorrowedInput(host, e.args[2]!); - host.emitPublishedArrayGuard(r.name, e.receiver.type, 7); + if (host.publishedTypes?.containerGuarded(e.receiver.type) === true) { + host.declare(`declare double @scr_arr_len(ptr)`); + const n = B.tmp(); + B.line(`${n} = call double @scr_arr_len(ptr ${items.name})`); + host.emitPublishedArrayGuard(r.name, e.receiver.type, 10, start.name, count.name, n); + } host.declare(`declare ptr @scr_arr_splice_insert(ptr, double, double, ptr)`); const t = B.tmp(); B.line( diff --git a/packages/compiler/src/backend/llvm/expr-context.ts b/packages/compiler/src/backend/llvm/expr-context.ts index df6dfc608..c6dc506f4 100644 --- a/packages/compiler/src/backend/llvm/expr-context.ts +++ b/packages/compiler/src/backend/llvm/expr-context.ts @@ -46,4 +46,7 @@ export interface LlStreamTypedRefAdapter { recordOps?: string; /** `@sym` of the ScrDynTypedClassOps table for class instance capsules. */ classOps?: string; + /** `@sym` of the ScrDynTypedBagClassOps table for capsules of bag-tracked + * class instances whose views inherit from prototype data. */ + bagClassOps?: string; } diff --git a/packages/compiler/src/backend/llvm/expr-dynamic.ts b/packages/compiler/src/backend/llvm/expr-dynamic.ts index 07a538184..26905a7ef 100644 --- a/packages/compiler/src/backend/llvm/expr-dynamic.ts +++ b/packages/compiler/src/backend/llvm/expr-dynamic.ts @@ -18,6 +18,7 @@ import { type IrExpr, type IrType, isDynTypedRefType, + textCodecRecordClass, isRefCounted, isUnitType, typeEquals, @@ -86,7 +87,10 @@ export function emitDynamicExpr( // Bytes already box their shared mutable storage directly. A second // capsule would split identity between checked and native views. const errorRef = v.type.kind === "object" && host.shapeHost.isErrorClass(v.type.className); - if (!errorRef && ((e.liveRef && v.type.kind !== "bytes") || identityRef)) { + // A TextEncoder or TextDecoder record boxes as its own handle, which + // keeps the record's identity (toDynHelper). + const codecRef = textCodecRecordClass(v.type, (id) => host.recordsById.get(id)) !== null; + if (!errorRef && !codecRef && ((e.liveRef && v.type.kind !== "bytes") || identityRef)) { if (v.type.kind === "union") { const adapter = host.liveDynUnionRefAdapter(v.type); const boxed = B.tmp(); diff --git a/packages/compiler/src/backend/llvm/expr-primitives.ts b/packages/compiler/src/backend/llvm/expr-primitives.ts index 1eec6a965..2c6ce35db 100644 --- a/packages/compiler/src/backend/llvm/expr-primitives.ts +++ b/packages/compiler/src/backend/llvm/expr-primitives.ts @@ -1076,9 +1076,10 @@ export function emitContainerExpr( const idx = host.emitExpr(e.index); if (e.arr.type.kind !== "array") throw new InternalCompilerError("llvm emitter bug: arrayState on non-array"); + const proven = host.provenDenseIndex(e.arr, e.index); const t = host.checkedInt32Array(e.arr) ? host.emitByElementKind(arr.name, "double", (kind) => - emitDenseArrayState(host, arr.name, idx, e.index, "state", slotWidth(kind)), + emitDenseArrayState(host, arr.name, idx, e.index, "state", slotWidth(kind), proven), )! : emitDenseArrayState( host, @@ -1087,6 +1088,7 @@ export function emitContainerExpr( e.index, "state", host.int32Array(e.arr) ? slotWidth("i32") : denseSlotWidth(host, e.arr.type.elem), + proven, ); return { name: t, type: e.type }; } diff --git a/packages/compiler/src/backend/llvm/expr-records.ts b/packages/compiler/src/backend/llvm/expr-records.ts index bca728e9d..59ab625e5 100644 --- a/packages/compiler/src/backend/llvm/expr-records.ts +++ b/packages/compiler/src/backend/llvm/expr-records.ts @@ -7,6 +7,7 @@ import type { LlvmEmitterContext, LlValue } from "./expr-context.js"; import { f64Lit, llvmCommentText } from "./common.js"; import { borrowableInputs, emitBorrowedInputs, emitBorrowedInput } from "./borrowed-inputs.js"; import { regexLiteralTemplate } from "./expr-primitives.js"; +import { isScalarUnionDef, loadInlineUnion, presentBits } from "./inline-unions.js"; import { preservesRegexInputs } from "./reference-effects.js"; /** Widen one tagged read. The source is borrowed unless the overflow map @@ -365,6 +366,114 @@ export function emitRecordKeyGet( return host.own({ name: t, type: e.type }); } +/** A keyed read whose result is a scalar union (`r[k]` on a + * `Record`, at `number | undefined`) and whose every hit + * is a number or boolean arm of it, or an inline field of the same union. + * Projections (`??`, `!`, tag tests) read it as a private stack box, so + * the read allocates nothing. */ +export function scalarRecordKeyGet(host: LlvmEmitterContext, e: IrExpr): boolean { + if (e.kind !== "recordKeyGet" || e.missNaN === true || e.type.kind !== "union") return false; + const resultType = e.type; + const def = host.unionsById.get(resultType.unionId); + if (!isScalarUnionDef(def) || undefinedArmTag(resultType, host.unionsById) < 0) return false; + const isArm = (t: IrType): boolean => + (t.kind === "f64" || t.kind === "bool") && def!.arms.some((a) => typeEquals(a, t)); + const shape = host.recordShape(e.shapeId); + if (e.overflowOnly !== true) { + for (const f of shape.fields) { + const inline = host.recordFieldInline(e.shapeId, f.name); + if (!isArm(f.type) && !(inline && typeEquals(f.type, resultType))) return false; + } + } + return shape.indexValue === undefined || isArm(shape.indexValue); +} + +/** A scalarRecordKeyGet read as a private stack box: the tag and bits of + * the declared field or overflow entry the key names, or the undefined arm + * for a miss. The box must never reach an RC entry point. */ +export function emitStackRecordKeyGet( + host: LlvmEmitterContext, + e: IrExpr & { kind: "recordKeyGet" }, +): string { + const B = host.B; + if (e.type.kind !== "union") + throw new InternalCompilerError("llvm emitter bug: stack keyed read of a non-union"); + const resultType = e.type; + const def = host.unionsById.get(resultType.unionId)!; + const armTag = (t: IrType): number => def.arms.findIndex((a) => typeEquals(a, t)); + const obj = host.emitExpr(e.obj); + const key = host.emitExpr(e.key); + const box = B.slot(); + B.entryAllocas.push(`${box} = alloca %ScrUnion`); + const tagPtr = B.tmp(), + bitsPtr = B.tmp(); + B.line(`store ${host.sizeType} -1, ptr ${box}`); + B.line(`${tagPtr} = getelementptr inbounds %ScrUnion, ptr ${box}, i64 0, i32 1`); + B.line(`${bitsPtr} = getelementptr inbounds %ScrUnion, ptr ${box}, i64 0, i32 5`); + const join = B.newLabel("rks.j"); + const storeScalar = (t: IrType, value: string): void => { + const bits = B.tmp(); + if (t.kind === "f64") B.line(`${bits} = bitcast double ${value} to i64`); + else B.line(`${bits} = zext i1 ${value} to i64`); + B.line(`store i32 ${armTag(t)}, ptr ${tagPtr}`); + B.line(`store i64 ${bits}, ptr ${bitsPtr}`); + }; + const shape = host.recordShape(e.shapeId); + for (const f of e.overflowOnly === true ? [] : shape.fields) { + const hit = B.tmp(); + B.line( + `${hit} = call zeroext i1 ${host.literalEquals(f.name)}(ptr ${key.name}) ; ${llvmCommentText(f.name)}`, + ); + const lh = B.newLabel("rks.h"); + const ln = B.newLabel("rks.n"); + B.condBr(hit, lh, ln); + B.startBlock(lh); + const { ptr, type, inline } = host.recordFieldPtr(obj.name, e.shapeId, f.name); + if (inline) { + const { tag, bits } = loadInlineUnion(B, ptr); + const present = presentBits(B, tag, bits, undefinedArmTag(type, host.unionsById)); + B.line(`store i32 ${tag}, ptr ${tagPtr}`); + B.line(`store i64 ${present}, ptr ${bitsPtr}`); + } else { + storeScalar(type, host.loadRecordField(ptr, type, false)); + } + B.br(join); + B.startBlock(ln); + } + const iv = shape.indexValue; + if (iv) { + const ovf = host.recordOvfPtr(obj.name, e.shapeId); + const outTy = iv.kind === "f64" ? "double" : "i8"; + const outSlot = B.slot(); + B.entryAllocas.push(`${outSlot} = alloca ${outTy}`); + B.line(`store ${outTy} ${iv.kind === "f64" ? f64Lit(0) : "0"}, ptr ${outSlot}`); + const getter = `scr_map_get_str_${iv.kind === "f64" ? "f64" : "bool"}`; + host.declare(`declare zeroext i1 @${getter}(ptr, ptr, ptr)`); + const found = B.tmp(); + B.line(`${found} = call zeroext i1 @${getter}(ptr ${ovf}, ptr ${key.name}, ptr ${outSlot})`); + const lh = B.newLabel("rks.h"); + const ln = B.newLabel("rks.n"); + B.condBr(found, lh, ln); + B.startBlock(lh); + const raw = B.tmp(); + B.line(`${raw} = load ${outTy}, ptr ${outSlot}`); + let value = raw; + if (iv.kind === "bool") { + value = B.tmp(); + B.line(`${value} = trunc i8 ${raw} to i1`); + } + storeScalar(iv, value); + B.br(join); + B.startBlock(ln); + } + // A miss is the undefined arm. + B.line(`store i32 ${undefinedArmTag(resultType, host.unionsById)}, ptr ${tagPtr}`); + B.line(`store i64 0, ptr ${bitsPtr}`); + B.br(join); + B.startBlock(join); + return box; +} + export function keyedRecordReadInto( host: LlvmEmitterContext, slot: string, diff --git a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts index ceaca3778..70a8dcdee 100644 --- a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts +++ b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts @@ -16,6 +16,7 @@ import { KEY_ORDER_FIELD, shapeKeyOrders, isDynTypedRefType, + textCodecRecordClass, isRefCounted, shapeHasAccessorSlots, typeKey, @@ -597,12 +598,21 @@ export function liveDynUnionRefAdapter( if (!union) { throw new InternalCompilerError(`llvm emitter bug: live dyn ref of unknown union ${t.unionId}`); } + // Codec records box as their own handles, by reference, through toDynHelper. + const codecArm = (arm: IrType): boolean => + textCodecRecordClass(arm, (id) => host.recordsById.get(id)) !== null; const mutableArms = union.arms .map((arm, tag) => ({ arm, tag })) .filter( - ({ arm }) => arm.kind !== "bytes" && (streamTypedRefEligible(arm) || isDynTypedRefType(arm)), + ({ arm }) => + arm.kind !== "bytes" && + !codecArm(arm) && + (streamTypedRefEligible(arm) || isDynTypedRefType(arm)), ); - if (mutableArms.length === 0 && !union.arms.some((arm) => arm.kind === "bytes")) { + if ( + mutableArms.length === 0 && + !union.arms.some((arm) => arm.kind === "bytes" || codecArm(arm)) + ) { throw new InternalCompilerError(`llvm emitter bug: live dyn ref of immutable union ${key}`); } @@ -712,7 +722,13 @@ export function streamTypedRefBoxValue( B.line(`${boxed} = call ptr @${host.liveDynUnionRefAdapter(t)}(ptr ${value})`); return boxed; } - if (t.kind === "bytes" || (!streamTypedRefEligible(t) && !isDynTypedRefType(t))) { + // A codec record boxes as its own handle, by reference (toDynHelper), as + // it does outside a live view. + if ( + t.kind === "bytes" || + textCodecRecordClass(t, (id) => host.recordsById.get(id)) !== null || + (!streamTypedRefEligible(t) && !isDynTypedRefType(t)) + ) { const valueTy = host.llType(t); B.line(`${boxed} = call ptr @${host.dyn.toDynHelper(t)}(${valueTy} ${value})`); return boxed; @@ -767,6 +783,7 @@ export function streamTypedRefMaterializeAdapter( // class view ops are published first. const opsMeta = isDynTypedRefType(t) ? host.classMeta.get(t.className) : undefined; if (opsMeta && liveClassOpsEligible(opsMeta)) adapter.classOps = `@${snapshot}_cops`; + else if (opsMeta && liveBagClassOpsEligible(opsMeta)) adapter.bagClassOps = `@${snapshot}_bops`; adapter.commit = `@${snapshot}_commit`; adapter.commit = host.streamTypedRefCommitAdapter(t, snapshot); const B = new BlockBuilder(); @@ -779,6 +796,7 @@ export function streamTypedRefMaterializeAdapter( ); } if (adapter.classOps !== undefined) liveClassOps(host, snapshot, meta); + if (adapter.bagClassOps !== undefined) liveBagClassOps(host, snapshot, meta, t); if (meta.hierarchy && meta.children.length) { const pre = B.tmp(); for (const line of classPreLines(meta, classStructSym(t.className), "%p", pre, host.sizeType)) @@ -850,11 +868,7 @@ export function streamTypedRefMaterializeAdapter( B.line(`call void @scr_dyn_release_v(ptr ${prototype})`); B.line(`call void @scr_dyn_release_v(ptr ${attached})`); } - for (const field of meta.def.fields.filter( - (f) => - isClassOwnEnumerableFieldName(f.name) && - !(meta.root.def.name === "%Error" && (f.name === "message" || f.name === "name")), - )) { + for (const field of classViewFields(meta)) { const afterField = meta.def.tracksOwnFields ? B.newLabel("field.after") : null; if (afterField) { host.declare(`declare ptr @scr_dyn_obj_get(ptr, ptr, ${host.sizeType})`); @@ -868,32 +882,7 @@ export function streamTypedRefMaterializeAdapter( B.condBr(present, yes, afterField); B.startBlock(yes); } - const { index, compressed } = classFieldIndex(meta, field.name, "compressed"); - const fieldPtr = B.tmp(); - let fieldValue = B.tmp(); - B.line( - `${fieldPtr} = getelementptr inbounds %${classStructSym(t.className)}, ptr %p, i64 0, i32 ${index}`, - ); - if (host.int32Slots.isField(t.className, field.name)) { - const raw = B.tmp(); - B.line(`${raw} = load i32, ptr ${fieldPtr}`); - B.line(`${fieldValue} = sitofp i32 ${raw} to double`); - } else if (host.inlineUnions.classField(t.className, field.name)) - fieldValue = loadStructField(B, host.sizeType, fieldPtr, field.type, true); - else if (compressed) { - const idx = B.tmp(); - B.line(`${idx} = load i32, ptr ${fieldPtr}`); - for (const line of compactRefLoadLines(idx, fieldValue, `${fieldValue}.r`)) - B.line(line.trimStart()); - } else B.line(`${fieldValue} = load ${llFieldType(field.type, false)}, ptr ${fieldPtr}`); - if (llFieldType(field.type, false) === "i8") { - const boolValue = B.tmp(); - B.line(`${boolValue} = trunc i8 ${fieldValue} to i1`); - fieldValue = boolValue; - } - const nullable = host.nullableFields.get(t.className, field.name); - if (nullable) fieldValue = nullableFieldUnion(host, B, fieldValue, nullable); - const boxed = host.streamTypedRefBoxValue(B, field.type, fieldValue); + const { boxed, nullable, fieldValue } = classFieldDynValue(host, B, meta, t, field); // Attributes copied from the bag do not stop the view taking the value. const setter = withAttributes ? "scr_dyn_obj_set_field" : "scr_dyn_obj_set"; if (withAttributes) @@ -1491,14 +1480,157 @@ function liveClassOps(host: LlvmEmitterContext, snapshot: string, meta: LlClassM ); } +/** The checked-dynamic value a class instance's view holds for `field`, read + * from its native slot of `%p`: `boxed` (+1), and the nullable union + * `fieldValue` the caller releases when `nullable` is set. */ +function classFieldDynValue( + host: LlvmEmitterContext, + B: BlockBuilder, + meta: LlClassMeta, + t: Extract, + field: LlClassMeta["def"]["fields"][number], +): { boxed: string; nullable: NullableRefField | null; fieldValue: string } { + const { index, compressed } = classFieldIndex(meta, field.name, "compressed"); + const fieldPtr = B.tmp(); + let fieldValue = B.tmp(); + B.line( + `${fieldPtr} = getelementptr inbounds %${classStructSym(t.className)}, ptr %p, i64 0, i32 ${index}`, + ); + if (host.int32Slots.isField(t.className, field.name)) { + const raw = B.tmp(); + B.line(`${raw} = load i32, ptr ${fieldPtr}`); + B.line(`${fieldValue} = sitofp i32 ${raw} to double`); + } else if (host.inlineUnions.classField(t.className, field.name)) + fieldValue = loadStructField(B, host.sizeType, fieldPtr, field.type, true); + else if (compressed) { + const idx = B.tmp(); + B.line(`${idx} = load i32, ptr ${fieldPtr}`); + for (const line of compactRefLoadLines(idx, fieldValue, `${fieldValue}.r`)) + B.line(line.trimStart()); + } else B.line(`${fieldValue} = load ${llFieldType(field.type, false)}, ptr ${fieldPtr}`); + if (llFieldType(field.type, false) === "i8") { + const boolValue = B.tmp(); + B.line(`${boolValue} = trunc i8 ${fieldValue} to i1`); + fieldValue = boolValue; + } + const nullable = host.nullableFields.get(t.className, field.name); + if (nullable) fieldValue = nullableFieldUnion(host, B, fieldValue, nullable); + return { boxed: host.streamTypedRefBoxValue(B, field.type, fieldValue), nullable, fieldValue }; +} + +/** The fields an instance's view takes from their native slots, in order: + * its own enumerable fields, except an Error's message and name. */ +function classViewFields(meta: LlClassMeta): LlClassMeta["def"]["fields"] { + return meta.def.fields.filter( + (f) => + isClassOwnEnumerableFieldName(f.name) && + !(meta.root.def.name === "%Error" && (f.name === "message" || f.name === "name")), + ); +} + +/** Classes whose instance views the materializer builds from the property + * bag and the prototype data alone (see liveBagClassOps): JavaScript classes + * tracking their own fields, outside the Error and runtime families, with no + * subclass whose view another materializer builds. */ +function liveBagClassOpsEligible(meta: LlClassMeta): boolean { + return ( + meta.def.tracksOwnFields === true && + (meta.def.prototypeDataHelper !== undefined || + meta.def.instancePrototypeHelper !== undefined) && + !meta.def.runtime && + !meta.root.def.runtime && + meta.root.def.name !== "%Error" && + !(meta.hierarchy && meta.children.length > 0) && + meta.def.fields.some((field) => field.name === DYN_CLASS_PROPERTIES) && + !meta.def.fields.some( + (field) => + field.name === CLASS_ATTRIBUTES || + meta.def.fields.some((f) => f.name === classFieldAttributesName(field.name)), + ) + ); +} + +/** The hooks of a bag-tracked class instance capsule + * (ScrDynTypedBagClassOps). The materializer gives the instance's view + * exactly the bag's string-keyed properties, the values of the declared + * fields among them from their native slots, and the prototype these hooks + * return, so a method call resolves without building the view. */ +function liveBagClassOps( + host: LlvmEmitterContext, + snapshot: string, + meta: LlClassMeta, + t: Extract, +): void { + const struct = classStructSym(t.className); + const bag = new BlockBuilder(); + const slot = bag.tmp(), + value = bag.tmp(); + bag.line( + `${slot} = getelementptr inbounds %${struct}, ptr %p, i64 0, i32 ${classFieldIndex(meta, DYN_CLASS_PROPERTIES).index}`, + ); + bag.line(`${value} = load ptr, ptr ${slot}`); + bag.terminate(`ret ptr ${value}`); + const proto = new BlockBuilder(); + const prototype = proto.tmp(); + if (meta.def.instancePrototypeHelper) { + const receiver = proto.tmp(); + proto.line(`${receiver} = call ptr ${vAdapters(host.shapeHost, t).retain}(ptr %p)`); + proto.line( + `${prototype} = call ptr @${mangleFunction(meta.def.instancePrototypeHelper)}(ptr ${receiver})`, + ); + } else proto.line(`${prototype} = call ptr @${mangleFunction(meta.def.prototypeDataHelper!)}()`); + proto.terminate(`ret ptr ${prototype}`); + // Field `%i` of the view's field list, as the materializer boxes it. + const fields = classViewFields(meta); + const get = new BlockBuilder(); + const cases = fields.map((_, i) => get.newLabel(`field.${i}`)); + const none = get.newLabel("field.none"); + get.terminate( + `switch ${host.sizeType} %i, label %${none} [ ${cases.map((label, i) => `${host.sizeType} ${i}, label %${label}`).join(" ")} ]`, + ); + fields.forEach((field, i) => { + get.startBlock(cases[i]!); + const { boxed, nullable, fieldValue } = classFieldDynValue(host, get, meta, t, field); + if (nullable) { + host.declare(`declare void @scr_union_release(ptr)`); + get.line(`call void @scr_union_release(ptr ${fieldValue})`); + } + get.terminate(`ret ptr ${boxed}`); + }); + get.startBlock(none); + get.terminate(`ret ptr null`); + const names = fields.map((field) => `ptr ${host.internLiteral(field.name)}`); + host.resolveThunkDefs.push( + `define internal ptr @${snapshot}_field(ptr %p, ${host.sizeType} %i) ${FN_ATTRS} { ; view field of ${meta.def.name}`, + get.render(), + `}`, + ``, + `@${snapshot}_fnames = internal constant [${names.length} x ptr] ${names.length ? `[${names.join(", ")}]` : "zeroinitializer"}`, + ); + host.resolveThunkDefs.push( + `define internal ptr @${snapshot}_bag(ptr %p) ${FN_ATTRS} { ; property bag of ${meta.def.name}`, + bag.render(), + `}`, + ``, + `define internal ptr @${snapshot}_proto(ptr %p) ${FN_ATTRS} { ; view prototype of ${meta.def.name}`, + proto.render(), + `}`, + ``, + `@${snapshot}_bops = internal constant { ptr, ptr, ${host.sizeType}, ptr, ptr } { ptr @${snapshot}_bag, ptr @${snapshot}_proto, ${host.sizeType} ${fields.length}, ptr @${snapshot}_fnames, ptr @${snapshot}_field }`, + ``, + ); +} + /** The live view a capsule of this adapter binds: the runtime bind call * and its ops constant, with the view code a deferred receiver recipe - * records (1 array, 2 record, 3 class). */ + * records (1 array, 2 record, 3 class, 4 bag-tracked class). */ export function liveRefView( adapter: LlStreamTypedRefAdapter, -): { bind: string; ops: string; code: 1 | 2 | 3 } | null { +): { bind: string; ops: string; code: 1 | 2 | 3 | 4 } | null { if (adapter.classOps !== undefined) return { bind: "scr_dyn_typed_ref_bind_class", ops: adapter.classOps, code: 3 }; + if (adapter.bagClassOps !== undefined) + return { bind: "scr_dyn_typed_ref_bind_bag_class", ops: adapter.bagClassOps, code: 4 }; if (adapter.recordOps !== undefined) return { bind: "scr_dyn_typed_ref_bind_record", ops: adapter.recordOps, code: 2 }; if (adapter.arrayOps !== undefined) diff --git a/packages/compiler/src/backend/llvm/expr-stream-callbacks.ts b/packages/compiler/src/backend/llvm/expr-stream-callbacks.ts index 14a2e3432..b368c91f4 100644 --- a/packages/compiler/src/backend/llvm/expr-stream-callbacks.ts +++ b/packages/compiler/src/backend/llvm/expr-stream-callbacks.ts @@ -727,6 +727,17 @@ export function streamCbThunkFor( const ty = host.llType(p); host.declare(`declare zeroext i1 @scr_exc_pending()`); if (hasThis) host.declare(`declare void @scr_stream_release_v(ptr)`); + // An entry `encoding` declared as a string (the declarations' + // `encoding: string`) receives "" where Node passes undefined or + // null (an object-mode write), as a write's encoding parameter does. + const entry = + p.kind === "array" && p.elem.kind === "record" + ? host.recordsById.get(p.elem.shapeId) + : undefined; + if (entry?.fields.some((f) => f.name === "encoding" && f.type.kind === "string")) { + host.declare(`declare void @scr_stream_writev_fill_encodings(ptr)`); + d.push(` call void @scr_stream_writev_fill_encodings(ptr %chunks)`); + } d.push( ` %vt${i} = call ${ty === "i1" ? "zeroext i1" : ty} @${helper}(ptr %chunks, ptr null)`, ` %vtbad${i} = call zeroext i1 @scr_exc_pending()`, diff --git a/packages/compiler/src/backend/llvm/lib-dispatch.ts b/packages/compiler/src/backend/llvm/lib-dispatch.ts index c8fae3192..b7f383921 100644 --- a/packages/compiler/src/backend/llvm/lib-dispatch.ts +++ b/packages/compiler/src/backend/llvm/lib-dispatch.ts @@ -25,6 +25,9 @@ import { rawBytes } from "./lib-abi.js"; import { emitInlineAtomicsOp } from "./expr-bytes.js"; import { regexLiteralCaptures } from "./regex-matchers.js"; +/** ScrDyn.v.typed_ref.type_key_len: the low 27 bits of its bit-field word. */ +const TYPE_KEY_LEN_MASK = (1 << 27) - 1; + export function emitAssertInspectLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue { const B = host.B; if (e.fn === "assert.shapeStr" || e.fn === "assert.shapeRe") { @@ -290,6 +293,38 @@ export function emitGenericLibCall(host: LlvmEmitterContext, e: LibCallExpr): Ll B.line(`store i1 true, ptr ${slot}`); B.condBr(same, done, compare); B.startBlock(compare); + // A capsule with another key pointer usually belongs to another + // class: reject it on its key length and last eight bytes before + // the out-of-line name comparison, so a class table's chain of + // probes costs a few instructions per class it skips. + B.line(`store i1 false, ptr ${slot}`); + const bytes = Buffer.from(literal, "utf8"); + const lenSlot = B.tmp(), + lenWord = B.tmp(), + len = B.tmp(), + lenEq = B.tmp(); + B.line( + `${lenSlot} = getelementptr inbounds i8, ptr ${value.name}, i64 ${host.abiOffset(88, 52)} ; ->v.typed_ref.type_key_len`, + ); + B.line(`${lenWord} = load i32, ptr ${lenSlot}`); + B.line(`${len} = and i32 ${lenWord}, ${TYPE_KEY_LEN_MASK}`); + B.line(`${lenEq} = icmp eq i32 ${len}, ${bytes.length}`); + const full = B.newLabel("brand.full"); + if (bytes.length >= 8) { + const tail = B.newLabel("brand.tail"); + B.condBr(lenEq, tail, done); + B.startBlock(tail); + const tailPtr = B.tmp(), + word = B.tmp(), + wordEq = B.tmp(); + B.line(`${tailPtr} = getelementptr inbounds i8, ptr ${keyPtr}, i64 ${bytes.length - 8}`); + B.line(`${word} = load i64, ptr ${tailPtr}, align 1`); + B.line(`${wordEq} = icmp eq i64 ${word}, ${bytes.readBigInt64LE(bytes.length - 8)}`); + B.condBr(wordEq, full, done); + } else { + B.condBr(lenEq, full, done); + } + B.startBlock(full); } } if (e.fn === "dyn.typedRefIs") { @@ -306,6 +341,41 @@ export function emitGenericLibCall(host: LlvmEmitterContext, e: LibCallExpr): Ll B.line(`${result} = load i1, ptr ${slot}`); return { name: result, type: e.type }; } + if (e.fn === "dyn.classIs") { + // Only a class constructor's function value can match. Expose the + // function tag to LLVM so a chain of constructor probes (a static + // property bag's dispatch) costs one test for every other value, and + // borrow the receiver across a literal key as the capsule probes do. + const literal = e.args[1]!.kind === "strLit"; + const value = literal ? host.emitReadReceiver(e.args[0]!) : host.emitExpr(e.args[0]!); + const key = host.emitExpr(e.args[1]!); + const slot = B.slot(); + B.entryAllocas.push(`${slot} = alloca i1`); + B.line(`store i1 false, ptr ${slot}`); + const present = B.tmp(); + const inspect = B.newLabel("ctor.inspect"); + const compare = B.newLabel("ctor.compare"); + const done = B.newLabel("ctor.done"); + B.line(`${present} = icmp ne ptr ${value.name}, null`); + B.condBr(present, inspect, done); + B.startBlock(inspect); + const kind = host.dynKind(value.name); + const fn = B.tmp(); + B.line(`${fn} = icmp eq i32 ${kind}, ${DYN_KIND.FUNC}`); + B.condBr(fn, compare, done); + B.startBlock(compare); + host.declare("declare zeroext i1 @scr_dyn_class_is_key(ptr, ptr)"); + const matches = B.tmp(); + B.line( + `${matches} = call zeroext i1 @scr_dyn_class_is_key(ptr ${value.name}, ptr ${key.name})`, + ); + B.line(`store i1 ${matches}, ptr ${slot}`); + B.br(done); + B.startBlock(done); + const result = B.tmp(); + B.line(`${result} = load i1, ptr ${slot}`); + return { name: result, type: e.type }; + } if (e.fn === "crypto.randomFillCb" || e.fn === "crypto.randomFillCbDyn") { host.usesTimers = true; const args = e.args.map((arg) => host.emitExpr(arg)); @@ -779,6 +849,19 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue { return { name: t, type: e.type }; } if (e.fn === "bytes.absent") return host.own({ name: "null", type: e.type }); + if (e.fn === "record.isFrozen") { + // The frozen bit of the record's rc word (scr_rc.h SCR_RC_FROZEN) is + // its second-highest bit; an immortal word sets the highest one too. + const value = host.emitReadReceiver(e.args[0]!); + const S = host.sizeType; + const rc = host.B.tmp(), + top = host.B.tmp(), + t = host.B.tmp(); + host.B.line(`${rc} = load ${S}, ptr ${value.name}`); + host.B.line(`${top} = lshr ${S} ${rc}, ${S === "i64" ? 62 : 30}`); + host.B.line(`${t} = icmp eq ${S} ${top}, 1`); + return { name: t, type: e.type }; + } const prefix = e.fn.slice(0, e.fn.indexOf(".")) as LibCallPrefix; // events.once builds typed listener adapters beside the emitter calls. if (e.fn === "events.once" || e.fn === "events.onceSignal") @@ -887,6 +970,7 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue { case "blob": case "wasm": case "array": + case "collection": case "record": case "disposable": case "punycode": diff --git a/packages/compiler/src/backend/llvm/lib-shared.ts b/packages/compiler/src/backend/llvm/lib-shared.ts index 13f9fcdc5..71cad2fe5 100644 --- a/packages/compiler/src/backend/llvm/lib-shared.ts +++ b/packages/compiler/src/backend/llvm/lib-shared.ts @@ -560,6 +560,7 @@ export const LIB_FN_SYMS: Record = { "text.decoderEncoding": "scr_text_decoder_encoding", "text.decoderName": "scr_text_decoder_name", "text.encodeInto": "scr_text_encode_into", + "text.codecIs": "scr_text_codec_is", "buffer.isAscii": "scr_buffer_is_ascii", "buffer.isUtf8": "scr_buffer_is_utf8", "buffer.transcode": "scr_buffer_transcode", @@ -585,6 +586,7 @@ export const LIB_FN_SYMS: Record = { "crypto.ecdhConvertKey": "scr_crypto_ecdh_convert_key", "crypto.dhNew": "scr_crypto_dh_new", "crypto.checkPrimeSync": "scr_crypto_check_prime_sync", + "crypto.getCipherInfo": "scr_crypto_get_cipher_info", "crypto.checkPrimeCb": "scr_crypto_check_prime_async", "crypto.generatePrimeSync": "scr_crypto_generate_prime_sync", "crypto.generatePrimeCb": "scr_crypto_generate_prime_async", @@ -897,6 +899,7 @@ export const LIB_FN_SYMS: Record = { // ambient-this read never throw. The fs dyn read is the sync-fs story. "fetch.start": "scr_fetch_static", "fetch.responseNew": "scr_fetch_response_new", + "fetch.responseLight": "scr_fetch_response_light", "fetch.responseJsonStatic": "scr_fetch_response_json_static", "fetch.input": "scr_fetch_input", "abort.controllerNew": "scr_fetch_abort_controller_new", @@ -957,6 +960,7 @@ export const LIB_FN_SYMS: Record = { "fetch.streamIs": "scr_fetch_stream_is", "zlib.streamIs": "scr_zlib_stream_is", "fetch.streamNew": "scr_fetch_stream_new", + "fetch.transformNew": "scr_fetch_transform_new", "fetch.streamFrom": "scr_fetch_stream_from", "fetch.readerRead": "scr_fetch_reader_read", "json.parse": "scr_json_parse", @@ -1018,6 +1022,12 @@ export const LIB_FN_SYMS: Record = { "class.descriptorHasValue": "scr_class_descriptor_has_value", "class.throwReadOnly": "scr_class_throw_read_only", "class.throwNotExtensible": "scr_class_throw_not_extensible", + "record.freeze": "scr_record_freeze", + "record.throwFrozen": "scr_record_throw_frozen", + "array.freeze": "scr_arr_freeze", + "array.isFrozen": "scr_arr_is_frozen", + "collection.preventExtensions": "scr_map_prevent_extensions", + "collection.isExtensible": "scr_map_is_extensible", "dyn.nativeSetNew": "scr_dyn_native_set_new", "dyn.nativeSetIs": "scr_dyn_native_set_is", "dyn.nativeMapIs": "scr_dyn_native_map_is", @@ -1139,6 +1149,8 @@ export const LIB_FN_SYMS: Record = { "readable.pushEncoding": "scr_stream_set_push_encoding", "stream.objectMode": "scr_stream_object_mode", "stream.decodeStrings": "scr_stream_decode_strings", + "stream.optionValues": "scr_stream_option_values", + "stream.checkEncodingOption": "scr_stream_check_encoding_option", "stream.duplexFromGen": "scr_stream_duplex_from_gen", "stream.duplexFromDyn": "scr_stream_duplex_from_dyn", "stream.duplexifyReadable": "scr_stream_duplexify_readable", @@ -1323,6 +1335,7 @@ export const LIB_FN_SYMS: Record = { "http.reqPipeClient": "scr_http_req_pipe_client", "http.reqPipeSock": "scr_http_req_pipe_sock", "http.reqResume": "scr_http_req_resume", + "http.reqPause": "scr_http_req_pause", "http.reqDestroy": "scr_http_req_destroy", "http.reqSetEncoding": "scr_http_req_set_encoding", "http.resSetHeader": "scr_http_res_set_header", @@ -1687,6 +1700,7 @@ export const USES_TIMERS_LIB_FNS = new Set([ "fetch.abortAny", "fetch.streamNew", "fetch.streamFrom", + "fetch.transformNew", "readable.new", // fs.ReadStream / fs.WriteStream open, read and write on the worker pool. "fs.createReadStream", @@ -1772,6 +1786,7 @@ export const USES_TIMERS_LIB_FNS = new Set([ "stream.setTransform", "stream.setFlush", "stream.setConstruct", + "stream.replaceConstruct", "stream.duplexFromGen", "stream.duplexFromDyn", "stream.duplexifyReadable", diff --git a/packages/compiler/src/backend/llvm/lib-stream.ts b/packages/compiler/src/backend/llvm/lib-stream.ts index 5fd4af1a9..fdbf13d4f 100644 --- a/packages/compiler/src/backend/llvm/lib-stream.ts +++ b/packages/compiler/src/backend/llvm/lib-stream.ts @@ -189,6 +189,7 @@ export function emitStreamLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlV e.fn === "stream.setTransform" || e.fn === "stream.setFlush" || e.fn === "stream.setConstruct" || + e.fn === "stream.replaceConstruct" || e.fn === "stream.setWritev" || e.fn === "stream.setWritevDefault" ) { @@ -203,6 +204,7 @@ export function emitStreamLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlV "stream.setTransform": "t", "stream.setFlush": "l", "stream.setConstruct": "c", + "stream.replaceConstruct": "c", "stream.setWritev": "v", "stream.setWritevDefault": "v", }; @@ -214,6 +216,7 @@ export function emitStreamLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlV "stream.setTransform": "scr_stream_set_transform", "stream.setFlush": "scr_stream_set_flush", "stream.setConstruct": "scr_stream_set_construct", + "stream.replaceConstruct": "scr_stream_replace_construct", "stream.setWritev": "scr_stream_set_writev", "stream.setWritevDefault": "scr_stream_set_writev_default", }; diff --git a/packages/compiler/src/backend/llvm/local-array-reads.ts b/packages/compiler/src/backend/llvm/local-array-reads.ts index 1c540295a..80ed2c440 100644 --- a/packages/compiler/src/backend/llvm/local-array-reads.ts +++ b/packages/compiler/src/backend/llvm/local-array-reads.ts @@ -797,7 +797,33 @@ export function emitLocalArrayRead( ); return null; }); - else if (inline) + else if ( + inline && + loopIndex !== null && + read.element.kind === "f64" && + host.provenDenseIndex(read.array, read.index) + ) { + // In a dense array loop's fast copy, the entry test proved the counter + // below the array's hole-free prefix: the slot holds a present value, + // and the read has no missing arm. + const dataPtr = B.tmp(), + data = B.tmp(), + valuePtr = B.tmp(), + value = B.tmp(), + width = int32 ? "i32" : "i64"; + B.line( + `${dataPtr} = getelementptr inbounds %ScrArr, ptr ${array.name}, i32 0, i32 ${SCR_ARR_FIELD.data}`, + ); + host.markMemoryPointer(dataPtr, "array:header"); + B.line(`${data} = load ptr, ptr ${dataPtr}${host.fieldAliasAttachment(dataPtr)}`); + B.line( + `${valuePtr} = getelementptr inbounds ${width}, ptr ${data}, ${host.sizeType} ${loopIndex}`, + ); + host.markMemoryPointer(valuePtr, "array:elements"); + B.line(`${value} = load ${width}, ptr ${valuePtr}${host.fieldAliasAttachment(valuePtr)}`); + storeValue(value); + B.br(join); + } else if (inline) emitDenseReferenceArrayRead( host, array, diff --git a/packages/compiler/src/backend/llvm/nullable-unions.ts b/packages/compiler/src/backend/llvm/nullable-unions.ts index 2c355f551..d9b9cdfcf 100644 --- a/packages/compiler/src/backend/llvm/nullable-unions.ts +++ b/packages/compiler/src/backend/llvm/nullable-unions.ts @@ -6,6 +6,7 @@ import { type IrUnionDef, } from "../../ir/ir.js"; import { everyModuleNode } from "../../ir/traverse.js"; +import type { IrNodeIndex } from "../../ir/node-index.js"; /** A union of exactly one reference arm and one or two unit arms * (`C | undefined`, `C | null`, `C | null | undefined`) whose VALUES are @@ -120,12 +121,12 @@ export class NullableUnions { /** `shapes`: allow shape unions (debug builds keep boxes, whose * debug-info layouts describe every arm). */ - constructor(mod: IrModule, shapes = true) { + constructor(mod: IrModule, shapes = true, nodes?: IrNodeIndex) { const emitted = new Set(); for (const cls of mod.classes ?? []) if (!cls.runtime) emitted.add(cls.name); const records = new Set(); for (const shape of mod.records ?? []) records.add(shape.id); - const boxed = boxedUnionIds(mod); + const boxed = boxedUnionIds(mod, nodes); // Shape unions stay out of library builds (their host ABI marshals // boxes) and of programs with lifetime regions (whose walks report a // reference slot by its type's release entry point). @@ -277,13 +278,20 @@ function shapeUnion( /** Unions the runtime itself inspects as tagged boxes keep that * representation: Map keys and Set elements hash and compare union keys by * tag and payload inside the C map. */ -function boxedUnionIds(mod: IrModule): Set { +function boxedUnionIds(mod: IrModule, nodes?: IrNodeIndex): Set { const boxed = new Set(); // Only the module's union definitions consult the result. if (!mod.unions?.length) return boxed; const mark = (t: IrType): void => { if (t.kind === "union") boxed.add(t.unionId); }; + if (nodes !== undefined) { + for (const t of nodes.types) { + if (t.kind === "map") mark(t.key); + else if (t.kind === "set") mark(t.elem); + } + return boxed; + } everyModuleNode(mod, { expr: () => true, stmt: () => true, diff --git a/packages/compiler/src/backend/llvm/receivers.ts b/packages/compiler/src/backend/llvm/receivers.ts index 67ca17188..2fe9dd66b 100644 --- a/packages/compiler/src/backend/llvm/receivers.ts +++ b/packages/compiler/src/backend/llvm/receivers.ts @@ -18,6 +18,7 @@ import { canStackUnion, emitStackUnion } from "./stack-unions.js"; import { emitDowncastGuard, emitUnionNarrowGuard } from "./narrow-guards.js"; import { emitScalarOptional, isScalarOptionalSource } from "./local-union-storage.js"; import { borrowableInputs } from "./borrowed-inputs.js"; +import { emitStackRecordKeyGet, scalarRecordKeyGet } from "./expr-records.js"; import { type WalkBorrowHost, analyzeWalks } from "./walk-borrows.js"; /** Read a receiver whose value is consumed immediately by a field or tag @@ -95,6 +96,8 @@ export function emitReadReceiver(emitter: LlEmitter, e: IrExpr): LlValue { const narrowed = narrow ? emitCheckedNarrowReceiver(emitter, narrow) : null; if (narrowed) return narrowed; if (canStackUnion(e, emitter.boxedUnionsById)) return emitStackUnion(emitter, e).value; + if (e.kind === "recordKeyGet" && scalarRecordKeyGet(emitter, e)) + return { name: emitStackRecordKeyGet(emitter, e), type: e.type }; if (e.kind === "unionNarrow") { const union = emitter.emitUnionProjection(e.value); emitUnionNarrowGuard(emitter, union.name, e.unionId, e.tag); @@ -342,6 +345,7 @@ export function analyzeBorrowedReturns(emitter: LlEmitter): void { borrowsWithoutOwning: (e: IrExpr) => emitter.borrowsWithoutOwning(e), borrowedReturn: (callee: string) => emitter.borrowedReturns.has(callee), parameterAllowed: (id: string) => allowed.has(id), + nodes: emitter.nodeIndex, }; return emitter.referenceEffects.functions.has(fn.name) ? host @@ -409,6 +413,7 @@ export function isStackUnionSource(emitter: LlEmitter, e: IrExpr): boolean { emitter.optionalArrayReads.get(e) !== null || isScalarOptionalSource(e, emitter.unionSources) || emitter.scalarResultCall(e) || - emitter.inlineFieldGet(e) + emitter.inlineFieldGet(e) || + scalarRecordKeyGet(emitter, e) ); } diff --git a/packages/compiler/src/backend/llvm/reference-writes.ts b/packages/compiler/src/backend/llvm/reference-writes.ts index df2ec5032..89132a715 100644 --- a/packages/compiler/src/backend/llvm/reference-writes.ts +++ b/packages/compiler/src/backend/llvm/reference-writes.ts @@ -9,6 +9,7 @@ import { type IrUnionDef, } from "../../ir/ir.js"; import { everyExprChild, everyStmtChild } from "../../ir/traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "../../ir/node-index.js"; import { borrowsMapMutationReceiver, borrowsMapReadInputs } from "./map-read-lifetimes.js"; import { preservesRegexInputs, VirtualTargets } from "./reference-effects.js"; import { borrowsStringInputs } from "./string-lifetimes.js"; @@ -227,6 +228,8 @@ export class ReferenceWrites { /** Fields no code writes once their object is constructed * (immutability.ts, from cycle-mutability's construction-only facts). */ private readonly immutable: ImmutableFields | null = null, + /** The module's node index, read instead of walking the bodies. */ + private readonly nodes?: IrNodeIndex, ) { this.key(HEAP_KEY); this.unions = new Map((types.unions ?? []).map((u) => [u.id, u])); @@ -236,25 +239,25 @@ export class ReferenceWrites { const constructors = new Set(classes.map((cls) => `%${cls.name}.constructor`)); this.constructors = constructors; this.freshConstructors = [...functions.values()].every((fn) => - fn.body.every((s) => this.onlySuperCalls(fn, s, constructors)), + this.onlySuperCalls(fn, constructors), ); - this.callTargets = new CallTargets(functions, this.virtualTargets); + this.callTargets = new CallTargets(functions, this.virtualTargets, nodes); this.solve(constructors); } - /** Every direct call of a constructor is a `super` call from a + /** Every direct call of a constructor in `fn` is a `super` call from a * constructor on its own `this`. */ - private onlySuperCalls(fn: IrFunction, s: IrStmt, constructors: ReadonlySet): boolean { + private onlySuperCalls(fn: IrFunction, constructors: ReadonlySet): boolean { const thisId = constructors.has(fn.name) ? fn.params[0]?.localId : undefined; - const expr = (e: IrExpr): boolean => { - if (e.kind === "call" && constructors.has(e.callee)) { - const self = e.args[0]; - if (thisId === undefined || !self || !isThisRef(self, thisId)) return false; + let only = true; + eachNodeOfKinds(fn, this.nodes?.scan(CALL_KINDS), (node) => { + if (node.kind === "call" && constructors.has(node.callee)) { + const self = node.args[0]; + if (thisId === undefined || !self || !isThisRef(self, thisId)) only = false; } - return everyExprChild(e, expr, stmt); - }; - const stmt = (node: IrStmt): boolean => everyStmtChild(node, expr, stmt); - return stmt(s); + return only; + }); + return only; } private key(k: string): number { @@ -299,7 +302,7 @@ export class ReferenceWrites { } const thisId = external && constructors.has(fn.name) ? (fn.params[0]?.localId ?? null) : null; const boxed = boxedLocals(fn); - const privates = privateArrays(fn, boxed); + const privates = privateArrays(fn, boxed, this.nodes); const sink: WriteSink = { add: (k) => { // Interned for queries; a callee's locals are private to it. @@ -749,7 +752,7 @@ export class ReferenceWrites { enter(fn: IrFunction): void { this.current = fn; this.currentBoxed = boxedLocals(fn); - this.currentPrivateArrays = privateArrays(fn, this.currentBoxed); + this.currentPrivateArrays = privateArrays(fn, this.currentBoxed, this.nodes); this.currentLocals = new Set(fn.locals.map((local) => local.id)); this.inConstructor = this.constructors.has(fn.name); for (const p of fn.params) this.currentLocals.add(p.localId); @@ -973,6 +976,29 @@ function privateReceiver(arr: IrExpr, privates: ReadonlySet): boolean { * element out nor let the array escape. */ const PRIVATE_ARRAY_METHODS = new Set(["length", "push"]); +/** The nodes the scans below read. */ +const CALL_KINDS: readonly IrNodeKind[] = ["call"]; +const CALL_SITE_KINDS: readonly IrNodeKind[] = ["call", "new", "virtualCall", "closure"]; +const DEFINITION_KINDS: readonly IrNodeKind[] = ["assignExpr", "incDec", "varDecl", "assign"]; + +/** The nodes privateArrays reads. */ +const PRIVATE_ARRAY_KINDS: readonly IrNodeKind[] = [ + "varRef", + "assignExpr", + "incDec", + "closure", + "classRef", + "arrIntrinsic", + "arrayHas", + "varDecl", + "assign", + "arraySet", + "arraySetUndefined", + "arrayDelete", + "arraySetLength", + "return", +]; + /** Array locals whose storage the function allocated itself and that * nothing else can reach before the function returns them: declared once * from an array literal or `new Array(n)`, never rebound, unboxed, and @@ -985,7 +1011,11 @@ const PRIVATE_ARRAY_METHODS = new Set(["length", "push"]); * stay out of the function's summary (a caller's borrows predate the * array). The literal map, filter and fused-pipeline helpers fill their * results this way. */ -function privateArrays(fn: IrFunction, boxed: ReadonlySet): ReadonlySet { +function privateArrays( + fn: IrFunction, + boxed: ReadonlySet, + nodes: IrNodeIndex | undefined, +): ReadonlySet { if (fn.async || fn.generator) return new Set(); const candidates = new Set(); for (const local of fn.locals) @@ -1001,54 +1031,50 @@ function privateArrays(fn: IrFunction, boxed: ReadonlySet): ReadonlySet< if (e.kind === "varRef" && candidates.has(e.localId)) permitted.set(e, (permitted.get(e) ?? 0) + 1); }; - const expr = (e: IrExpr): boolean => { - switch (e.kind) { + eachNodeOfKinds(fn, nodes?.scan(PRIVATE_ARRAY_KINDS), (node) => { + switch (node.kind) { case "varRef": { - const left = permitted.get(e) ?? 0; - if (left === 0) rejected.add(e.localId); - else permitted.set(e, left - 1); + const left = permitted.get(node) ?? 0; + if (left === 0) rejected.add(node.localId); + else permitted.set(node, left - 1); break; } case "assignExpr": case "incDec": - rejected.add(e.localId); + rejected.add(node.localId); break; case "closure": case "classRef": - for (const id of e.captures ?? []) rejected.add(id); + for (const id of node.captures ?? []) rejected.add(id); break; case "arrIntrinsic": - if (PRIVATE_ARRAY_METHODS.has(e.method)) permit(e.receiver); + if (PRIVATE_ARRAY_METHODS.has(node.method)) permit(node.receiver); break; case "arrayHas": - permit(e.arr); + permit(node.arr); break; - } - return everyExprChild(e, expr, stmt); - }; - const stmt = (s: IrStmt): boolean => { - switch (s.kind) { case "varDecl": - declarations.set(s.localId, (declarations.get(s.localId) ?? 0) + 1); - if (s.init === null || (s.init.kind !== "arrayLit" && s.init.kind !== "arrayNewLen")) - rejected.add(s.localId); + declarations.set(node.localId, (declarations.get(node.localId) ?? 0) + 1); + if ( + node.init === null || + (node.init.kind !== "arrayLit" && node.init.kind !== "arrayNewLen") + ) + rejected.add(node.localId); break; case "assign": - rejected.add(s.localId); + rejected.add(node.localId); break; case "arraySet": case "arraySetUndefined": case "arrayDelete": case "arraySetLength": - permit(s.arr); + permit(node.arr); break; case "return": - if (s.value !== null) permit(s.value); + if (node.value !== null) permit(node.value); break; } - return everyStmtChild(s, expr, stmt); - }; - fn.body.every(stmt); + }); const out = new Set(); for (const id of candidates) if (!rejected.has(id) && declarations.get(id) === 1) out.add(id); return out; @@ -1081,30 +1107,30 @@ class CallTargets { constructor( private readonly functions: ReadonlyMap, virtualTargets: VirtualTargets, + private readonly nodes: IrNodeIndex | undefined, ) { const site = (callee: string, caller: IrFunction, args: readonly IrExpr[]): void => { let list = this.sites.get(callee); if (!list) this.sites.set(callee, (list = [])); list.push({ caller, args }); }; + const scan = nodes?.scan(CALL_SITE_KINDS); for (const fn of functions.values()) { if (fn.captures !== undefined || fn.classCaptures !== undefined) this.opaque.add(fn.name); - const expr = (e: IrExpr): boolean => { - if (e.kind === "call") site(e.callee, fn, e.args); - else if (e.kind === "new") site(`%${e.className}.constructor`, fn, [e, ...e.args]); - else if (e.kind === "virtualCall") { - const targets = virtualTargets.targets(e.className, e.method); - for (const t of targets ?? []) site(t, fn, e.args); - } else if (e.kind === "closure") { - this.opaque.add(e.fnName); - let list = this.creators.get(e.fnName); - if (!list) this.creators.set(e.fnName, (list = [])); + eachNodeOfKinds(fn, scan, (node) => { + if (node.kind === "call") site(node.callee, fn, node.args); + else if (node.kind === "new") + site(`%${node.className}.constructor`, fn, [node, ...node.args]); + else if (node.kind === "virtualCall") { + const targets = virtualTargets.targets(node.className, node.method); + for (const t of targets ?? []) site(t, fn, node.args); + } else if (node.kind === "closure") { + this.opaque.add(node.fnName); + let list = this.creators.get(node.fnName); + if (!list) this.creators.set(node.fnName, (list = [])); list.push(fn); } - return everyExprChild(e, expr, stmt); - }; - const stmt = (st: IrStmt): boolean => everyStmtChild(st, expr, stmt); - fn.body.every(stmt); + }); } // Fixpoint over function-typed parameter positions. const positions: [IrFunction, number][] = []; @@ -1225,18 +1251,13 @@ class CallTargets { defs = new Map(); const boxed = new Set(fn.locals.filter((l) => l.boxed).map((l) => l.id)); const counts = new Map(); - const expr = (e: IrExpr): boolean => { - if (e.kind === "assignExpr" || e.kind === "incDec") counts.set(e.localId, 2); - return everyExprChild(e, expr, stmt); - }; - const stmt = (st: IrStmt): boolean => { - if (st.kind === "varDecl") { - counts.set(st.localId, (counts.get(st.localId) ?? 0) + 1); - defs!.set(st.localId, st.init); - } else if (st.kind === "assign") counts.set(st.localId, 2); - return everyStmtChild(st, expr, stmt); - }; - fn.body.every(stmt); + eachNodeOfKinds(fn, this.nodes?.scan(DEFINITION_KINDS), (node) => { + if (node.kind === "assignExpr" || node.kind === "incDec") counts.set(node.localId, 2); + else if (node.kind === "varDecl") { + counts.set(node.localId, (counts.get(node.localId) ?? 0) + 1); + defs!.set(node.localId, node.init); + } else if (node.kind === "assign") counts.set(node.localId, 2); + }); for (const [local, n] of counts) if (n !== 1 || boxed.has(local)) defs.set(local, null); this.definitions.set(fn, defs); } diff --git a/packages/compiler/src/backend/llvm/shapes.test.ts b/packages/compiler/src/backend/llvm/shapes.test.ts index f7327a688..7af16ab83 100644 --- a/packages/compiler/src/backend/llvm/shapes.test.ts +++ b/packages/compiler/src/backend/llvm/shapes.test.ts @@ -14,6 +14,7 @@ import { type ShapeHost, vAdapters, } from "./shapes.js"; +import { mangleRecordRelease } from "../mangle.js"; import { NullableUnions } from "./nullable-unions.js"; import { InlineUnionFields } from "./inline-unions.js"; @@ -315,6 +316,42 @@ describe("emitted object releases", () => { root.indexOf("call void @scr_cyc_on_release(ptr %o)"), ); }); + + test("freezable records count their references without the frozen bit", () => { + const mod: IrModule = { + irVersion: 17, + sourceFile: "frozen.ts", + entry: "main", + functions: [], + records: [ + { id: "plain", fields: [{ name: "n", type: F64 }] }, + { id: "frozen", fields: [{ name: "n", type: F64 }], freezable: true }, + ], + }; + const body = (defs: string, name: string): string => + new RegExp(`^define internal void @${name}\\([^]*?^}`, "m").exec(defs)?.[0] ?? ""; + for (const [pointerBits, notFrozen, sharedBelow] of [ + [64, "-4611686018427387905", "-4611686018427387907"], + [32, "-1073741825", "-1073741827"], + ] as const) { + const { host } = declarationHost(pointerBits); + const defs = emitRecordShapes({ ...host, objectAudit: false }, mod).defs.join("\n"); + const S = pointerBits === 64 ? "i64" : "i32"; + const plain = body(defs, mangleRecordRelease("plain")); + expect(plain).toContain(`%shared = icmp ult ${S} %rcm2, -3`); + expect(plain).not.toContain("%refs"); + // The immortal word's count is ~FROZEN, which the compare sends to the + // immortal test, like counts 0 and 1. + const fast = body(defs, mangleRecordRelease("frozen")); + expect(fast).toContain(`%refs = and ${S} %rc, ${notFrozen}`); + expect(fast).toContain(`%rcm2 = sub ${S} %refs, 2`); + expect(fast).toContain(`%shared = icmp ult ${S} %rcm2, ${sharedBelow}`); + expect(fast).toContain(`%imm = icmp eq ${S} %rc, -1`); + const slow = body(defs, `${mangleRecordRelease("frozen")}_slow`); + expect(slow).toContain(`%refs = and ${S} %n, ${notFrozen}`); + expect(slow).toContain(`%dead = icmp eq ${S} %refs, 0`); + } + }); }); describe("live-object audit notes", () => { diff --git a/packages/compiler/src/backend/llvm/shapes.ts b/packages/compiler/src/backend/llvm/shapes.ts index d13f80a78..813b54859 100644 --- a/packages/compiler/src/backend/llvm/shapes.ts +++ b/packages/compiler/src/backend/llvm/shapes.ts @@ -34,6 +34,7 @@ import { cycTypeDescriptor, emitObjectAlloc, emitObjectFree, + emitSegmentFree, CYC_APPEND, cycleAppendDecls, cycleAppendHelper, @@ -1405,17 +1406,23 @@ export interface DirectDestroy { /** The release of child `value` (a loaded field, possibly NULL) inside a * teardown whose chain length is `%d`: the child's own release for a shared * or immortal reference, its teardown for the last one. Labels use `tag`; - * the lines end in the open block `${tag}.next`. */ + * the lines end in the open block `${tag}.next`. + * + * With `segment` (the child has the teardown's own type, in a segmented + * teardown; segmentPrologueLines), the child's teardown appends its blocks + * to the teardown's segment. */ export function directChildReleaseLines( host: ShapeHost, tag: string, value: string, target: DirectDestroy, comment: string, + segment = false, ): string[] { const S = host.sizeType; const C = target.count ?? S; - host.declare(`declare void @scr_rc_destroy(ptr, ptr)`); + if (segment) host.declare(`declare ptr @scr_rc_destroy_seg(ptr, ptr, ptr, ptr)`); + else host.declare(`declare void @scr_rc_destroy(ptr, ptr)`); const lines = [ ` %${tag}.nn = icmp ne ptr ${value}, null`, ` br i1 %${tag}.nn, label %${tag}.chk, label %${tag}.next`, @@ -1447,16 +1454,139 @@ export function directChildReleaseLines( ` br i1 %${tag}.short, label %${tag}.direct, label %${tag}.bounded`, `${tag}.direct:`, ` %${tag}.d = add i32 %d, 1`, - ` call void @${target.destroy}_d(ptr ${value}, i32 %${tag}.d)`, + ); + if (!segment) { + lines.push( + ` call void @${target.destroy}_d(ptr ${value}, i32 %${tag}.d)`, + ` br label %${tag}.next`, + `${tag}.bounded:`, + ` call void @scr_rc_destroy(ptr ${value}, ptr @${target.destroy})`, + ` br label %${tag}.next`, + `${tag}.next:`, + ); + return lines; + } + lines.push( + ` %${tag}.t = load ptr, ptr %sg.t`, + ` %${tag}.dt = call ptr @${target.destroy}_s(ptr ${value}, i32 %${tag}.d, ptr %${tag}.t)`, + ` store ptr %${tag}.dt, ptr %sg.t`, ` br label %${tag}.next`, `${tag}.bounded:`, - ` call void @scr_rc_destroy(ptr ${value}, ptr @${target.destroy})`, + ` %${tag}.bt = load ptr, ptr %sg.t`, + ` %${tag}.rt = call ptr @scr_rc_destroy_seg(ptr ${value}, ptr @${target.destroy}_sc, ptr @${target.destroy}, ptr %${tag}.bt)`, + ` store ptr %${tag}.rt, ptr %sg.t`, ` br label %${tag}.next`, `${tag}.next:`, ); return lines; } +/* Teardown segments. A typed teardown that frees each block onto its free + * list as it goes frees a tree in walk order, and the free lists, which are + * LIFO, hand the blocks to the next allocations in the reverse order: a tree + * built again in the same order as the last (each generation of a structure + * a loop rebuilds) is laid out mirrored, and every walk over it, the next + * teardown included, runs through memory backwards, which defeats the + * hardware prefetchers. The runtime frees large dead sets of the cycle + * collector in address order for the same reason (scr_mem_order_for_free). + * + * So the teardown of a type with children of its own type (the `_s` form, + * segmentedTeardown) keeps the blocks it frees in a segment, linked through + * their first words, which it builds in order through a tail pointer: the + * `_s` form takes the segment's last link (the first word of its last block, + * or the head slot of an empty segment) and returns the new one. A node + * first releases its own block (emitObjectFree, keeping it), then appends it + * ahead of its children's blocks when its address is the lower, behind them + * otherwise (segmentTeardownLines), and the children append theirs in field + * order, so a structure + * built parent first (object literals allocate before their fields) or + * children first (constructor arguments) comes back in ascending order. The + * chain's first teardown (`_d`) frees the whole segment onto the class's + * free list in one step (emitSegmentFree), and the walks over the next + * generation run forward. A teardown past DIRECT_DESTROY_CHAIN appends + * through scr_rc_destroy_seg, under the same depth budget as + * scr_rc_destroy. Blocks outside the calling thread's arena are freed as + * before; one type's blocks are all of one class. */ + +/** Whether the teardown of a type whose reference fields have `fieldTypes` + * (with direct targets `targets`) and whose own teardown is `destroy` keeps + * a segment: a child field has its own type, and the inline allocation paths + * are in use. */ +export function segmentedTeardown( + host: ShapeHost, + destroy: string, + fieldTypes: readonly IrType[], + targets: ReadonlyMap, +): boolean { + if (host.inlineAlloc !== true) return false; + return fieldTypes.some((t) => directDestroyTarget(host, t, targets)?.destroy === destroy); +} + +/** The first lines of a segmented teardown, in its entry block: the + * segment's last link (`%sg.t`, from the `%tail` argument) and the kept own + * block (`%sg.own`, for emitObjectFree). */ +export function segmentPrologueLines(): string[] { + return [ + ` %sg.t = alloca ptr`, + ` %sg.own = alloca ptr`, + ` store ptr %tail, ptr %sg.t`, + ` store ptr null, ptr %sg.own`, + ]; +} + +/** The rest of a segmented teardown after emitObjectFree kept (or freed) + * its own block: the block goes ahead of its children's when its address is + * below the first non-NULL one of `children` (loaded fields of the + * teardown's own type, which stand for their blocks) or there is none, and + * behind them otherwise; then the children's releases (`releases`, given a + * label prefix), and the segment's last link is returned. Each order has its + * own copy of the releases, so neither keeps the decision, or the block once + * it is appended, across the children's teardowns. */ +export function segmentTeardownLines( + children: readonly string[], + releases: (prefix: string) => string[], +): string[] { + const lines = [` %sg.o = load ptr, ptr %sg.own`, ` %sg.kept = icmp ne ptr %sg.o, null`]; + let ref = "null"; + [...children].reverse().forEach((child, i) => { + lines.push( + ` %sg.nn${i} = icmp ne ptr ${child}, null`, + ` %sg.r${i} = select i1 %sg.nn${i}, ptr ${child}, ptr ${ref}`, + ); + ref = `%sg.r${i}`; + }); + lines.push( + ` %sg.leaf = icmp eq ptr ${ref}, null`, + ` %sg.lower = icmp ult ptr %sg.o, ${ref}`, + ` %sg.first = or i1 %sg.leaf, %sg.lower`, + ` %sg.ahead = and i1 %sg.kept, %sg.first`, + ` br i1 %sg.ahead, label %sg.front, label %sg.back`, + `sg.front:`, + ...segmentAppendLines("a"), + ...releases("ta"), + ` br label %sg.ret`, + `sg.back:`, + ...releases("tb"), + ` %sg.keep = icmp ne ptr %sg.o, null`, + ` br i1 %sg.keep, label %sg.last, label %sg.ret`, + `sg.last:`, + ...segmentAppendLines("l"), + ` br label %sg.ret`, + `sg.ret:`, + ` %sg.tr = load ptr, ptr %sg.t`, + ` ret ptr %sg.tr`, + ); + return lines; +} + +function segmentAppendLines(tag: string): string[] { + return [ + ` %sg.t${tag} = load ptr, ptr %sg.t`, + ` store ptr %sg.o, ptr %sg.t${tag}`, + ` store ptr %sg.o, ptr %sg.t`, + ]; +} + /** The direct-teardown target of a reference of `type`, if `targets` (keyed * `record:` / `object:`) has one. Nullable unions hold the arm's * pointer, NULL or an immortal sentinel, which the child release handles. */ @@ -1499,12 +1629,42 @@ export function usedDirectTargets( /** The teardown definitions for `destroy`: with `chained`, the body takes * the chain length `%d` (directChildReleaseLines) and the runtime's - * one-argument entry starts a chain. */ + * one-argument entry starts a chain. With `segment` (the host, for a + * segmented teardown), the body is a segment teardown's + * (segmentPrologueLines to segmentTeardownLines), defined as `_s`, with + * `_sc` for scr_rc_destroy_seg, and `_d` starts a segment and frees it. */ export function destroyDefinitions( destroy: string, body: readonly string[], chained: boolean, + segment: ShapeHost | null = null, ): string[] { + if (segment !== null) + return [ + `define internal ptr @${destroy}_s(ptr %o, i32 %d, ptr %tail) ${FN_ATTRS} {`, + `entry:`, + ...body, + `}`, + `define internal ptr @${destroy}_sc(ptr %o, ptr %tail) ${FN_ATTRS} {`, + `entry:`, + ` %t = call ptr @${destroy}_s(ptr %o, i32 0, ptr %tail)`, + ` ret ptr %t`, + `}`, + `define internal void @${destroy}_d(ptr %o, i32 %d) ${FN_ATTRS} {`, + `entry:`, + ` %head = alloca ptr`, + ` store ptr null, ptr %head`, + ` %last = call ptr @${destroy}_s(ptr %o, i32 %d, ptr %head)`, + ` %first = load ptr, ptr %head`, + ...emitSegmentFree(segment, "sgf", "%first", "%last"), + ` ret void`, + `}`, + `define internal void @${destroy}(ptr %o) ${FN_ATTRS} {`, + `entry:`, + ` call void @${destroy}_d(ptr %o, i32 0)`, + ` ret void`, + `}`, + ]; if (!chained) return [ `define internal void @${destroy}(ptr %o) ${FN_ATTRS} {`, @@ -1527,6 +1687,17 @@ export function destroyDefinitions( ]; } +/** The frozen bit of a freezable record's rc word (scr_rc.h SCR_RC_FROZEN): + * bit 62 on 64-bit targets, bit 30 on 32-bit ones. */ +function recordFrozenBit(sizeType: RcCount): bigint { + return 1n << (sizeType === "i64" ? 62n : 30n); +} + +/** `value` as a signed LLVM integer literal of `sizeType`. */ +function sizeLiteral(sizeType: RcCount, value: bigint): string { + return BigInt.asIntN(sizeType === "i64" ? 64 : 32, value).toString(); +} + /** Common NULL/immortal/decrement skeleton for ordinary object releases. * `freeBody` owns the zero-ref teardown and must leave the current block at * its end; traced objects also get the possible-cycle-root branch. @@ -1543,13 +1714,17 @@ export function releaseBody( bounded = true, immortal = false, chained = false, + segmented = false, + frozen = false, ): string[] { const S = host.sizeType; if (bounded) host.declare(`declare void @scr_rc_destroy(ptr, ptr)`); const destroy = `${fnName}_destroy`; + // A freezable record's count is the word without its frozen bit. + const notFrozen = sizeLiteral(S, ~recordFrozenBit(S)); const lines = [ - ...(bounded ? destroyDefinitions(destroy, freeBody, chained) : []), - ...releaseFastPath(host, fnName, `${fnName}_slow`, traced, comment, immortal), + ...(bounded ? destroyDefinitions(destroy, freeBody, chained, segmented ? host : null) : []), + ...releaseFastPath(host, fnName, `${fnName}_slow`, traced, comment, immortal, S, frozen), `define internal void @${fnName}_slow(ptr %o) noinline ${FN_ATTRS} {${comment ? ` ; ${comment}` : ""}`, `entry:`, ` %isnull = icmp eq ptr %o, null`, @@ -1561,7 +1736,9 @@ export function releaseBody( `dec:`, ` %n = sub ${S} %rc, 1`, ` store ${S} %n, ptr %o`, - ` %dead = icmp eq ${S} %n, 0`, + ...(frozen + ? [` %refs = and ${S} %n, ${notFrozen}`, ` %dead = icmp eq ${S} %refs, 0`] + : [` %dead = icmp eq ${S} %n, 0`]), ` br i1 %dead, label %free, label %${traced ? "root" : "done"}`, `free:`, ...(traced ? cycleDeadLines(host, "teardown") : []), @@ -1591,9 +1768,14 @@ export function releaseFastPath( comment = "", immortal = false, count: RcCount = host.sizeType, + frozen = false, ): string[] { const S = count; if (immortal) return immortalRcBody(host, fnName, "void", comment); + // A freezable record compares its count without the frozen bit: an + // immortal word's count is then ~FROZEN, which the same compare sends to + // `slow`. + const notFrozen = ~recordFrozenBit(S); return [ `define internal void @${fnName}(ptr %o) ${FN_ATTRS} {${comment ? ` ; ${comment} (fast path)` : ""}`, `entry:`, @@ -1601,8 +1783,13 @@ export function releaseFastPath( ` br i1 %isnull, label %done, label %check`, `check:`, ` %rc = load ${S}, ptr %o`, - ` %rcm2 = sub ${S} %rc, 2`, - ` %shared = icmp ult ${S} %rcm2, -3`, + ...(frozen + ? [ + ` %refs = and ${S} %rc, ${sizeLiteral(S, notFrozen)}`, + ` %rcm2 = sub ${S} %refs, 2`, + ` %shared = icmp ult ${S} %rcm2, ${sizeLiteral(S, notFrozen - 2n)}`, + ] + : [` %rcm2 = sub ${S} %rc, 2`, ` %shared = icmp ult ${S} %rcm2, -3`]), ` br i1 %shared, label %dec, label %slow`, `dec:`, ` %n = sub ${S} %rc, 1`, @@ -1780,26 +1967,58 @@ export function emitRecordShapes( const chained = directTargets.has(`record:${shape.id}`) || refMembers.some((m) => directDestroyTarget(host, m.type, directTargets) !== null); - for (const m of refMembers) { - freeBody.push( - ` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${m.index}`, - ` %v${t} = load ptr, ptr %f${t}`, + const destroy = `${mangleRecordRelease(shape.id)}_destroy`; + const bounded = refMembers.some((member) => needsBoundedRelease(member.type)); + const segmented = + bounded && + segmentedTeardown( + host, + destroy, + refMembers.map((m) => m.type), + directTargets, ); - const direct = directDestroyTarget(host, m.type, directTargets); - if (direct) { - freeBody.push( - ...directChildReleaseLines(host, `td${t}`, `%v${t}`, direct, llvmCommentText(m.name)), - ); - } else { - freeBody.push( - ` call void ${releaseSym(host, m.type)}(ptr %v${t}) ; ${llvmCommentText(m.name)}`, - ); - } - t++; - } // Closed layouts can carry a kept record source (record-sources.ts). const observers = recordMayKeepSource(shape) ? "weak-or-source" : "weak"; - freeBody.push(...emitObjectFree(host, traced, observers)); + // A segmented teardown loads its fields, releases its own block, and + // then releases the fields (segmentTeardownLines). + const loads: string[] = []; + const ownType: string[] = []; + const release = (prefix: string, m: (typeof refMembers)[number], i: number): string[] => { + const direct = directDestroyTarget(host, m.type, directTargets); + if (direct === null) + return [` call void ${releaseSym(host, m.type)}(ptr %v${i}) ; ${llvmCommentText(m.name)}`]; + const own = segmented && direct.destroy === destroy; + return directChildReleaseLines( + host, + `${prefix}${i}`, + `%v${i}`, + direct, + llvmCommentText(m.name), + own, + ); + }; + for (const m of refMembers) { + const fieldLines = [ + ` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${m.index}`, + ` %v${t} = load ptr, ptr %f${t}`, + ]; + if (segmented) { + loads.push(...fieldLines); + if (directDestroyTarget(host, m.type, directTargets)?.destroy === destroy) + ownType.push(`%v${t}`); + } else freeBody.push(...fieldLines, ...release("td", m, t)); + t++; + } + if (segmented) + freeBody.push( + ...segmentPrologueLines(), + ...loads, + ...emitObjectFree(host, traced, observers, "%sg.own"), + ...segmentTeardownLines(ownType, (prefix) => + refMembers.flatMap((m, i) => release(prefix, m, i)), + ), + ); + else freeBody.push(...emitObjectFree(host, traced, observers)); defs.push( ...releaseBody( host, @@ -1807,9 +2026,11 @@ export function emitRecordShapes( traced, freeBody, "", - refMembers.some((member) => needsBoundedRelease(member.type)), + bounded, false, chained, + segmented, + shape.freezable === true, ), ``, ); diff --git a/packages/compiler/src/backend/llvm/stack-objects.ts b/packages/compiler/src/backend/llvm/stack-objects.ts index c7a58a7ea..ae2b904db 100644 --- a/packages/compiler/src/backend/llvm/stack-objects.ts +++ b/packages/compiler/src/backend/llvm/stack-objects.ts @@ -44,8 +44,9 @@ export type StackObjectFieldOwner = (fields: readonly StackObjectField[]) => voi * standalone scalar class whose constructor never lets `this` escape * (record-results.ts). The caller proves that every use of the value only * projects it. Record fields may own references; the frame releases them - * (stackObjectFields). Published shapes keep heap storage: their field - * guards treat the immortal header as a frozen object. */ + * (stackObjectFields). Published and freezable shapes keep heap storage: + * their field guards treat the immortal header as a frozen object, and a + * frozen record keeps its frozen bit in a counted header. */ export function canStackObject(host: LlvmEmitterContext, e: IrExpr): e is StackObjectSource { // A ThreadSlots key keeps its slot index for its lifetime and has its // values released when it is freed: it lives on the heap. @@ -53,6 +54,8 @@ export function canStackObject(host: LlvmEmitterContext, e: IrExpr): e is StackO return false; if (e.type.kind === "record" && host.publishedTypes?.recordGuarded(e.type.shapeId) === true) return false; + if (e.type.kind === "record" && host.recordsById.get(e.type.shapeId)?.freezable === true) + return false; // A producer with a result-slot entry builds its fresh result in the // caller's storage. if (e.kind === "call") return host.fnByName.has(resultEntryName(e.callee)); diff --git a/packages/compiler/src/backend/llvm/walk-borrows.ts b/packages/compiler/src/backend/llvm/walk-borrows.ts index dc3e03c19..86b8f3d21 100644 --- a/packages/compiler/src/backend/llvm/walk-borrows.ts +++ b/packages/compiler/src/backend/llvm/walk-borrows.ts @@ -1,5 +1,5 @@ import { isRefCounted, type IrExpr, type IrFunction, type IrLocal } from "../../ir/ir.js"; -import { everyStmtList } from "../../ir/traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "../../ir/node-index.js"; /** What the emitter knows about a projection's storage. */ export interface WalkBorrowHost { @@ -22,8 +22,24 @@ export interface WalkBorrowHost { * (whose own rebinding releases nothing). Absent: the body preserves * every edge. */ unwritten?(value: IrExpr, candidates: ReadonlySet): boolean; + /** The module's node index, read instead of walking the body. */ + nodes?: IrNodeIndex; } +/** The node kinds analyzeWalks reads from a body. */ +const WALK_KINDS: readonly IrNodeKind[] = [ + "varDecl", + "assign", + "forOf", + "rethrow", + "tryCatch", + "return", + "assignExpr", + "incDec", + "closure", + "classRef", +]; + export interface WalkFacts { locals: ReadonlySet; /** Every return yields a walk (or a throw), so the body can return its @@ -71,44 +87,36 @@ export function analyzeWalks(fn: IrFunction, host: WalkBorrowHost): WalkFacts { if (!list) definitions.set(id, (list = [])); list.push(value); }; - everyStmtList(fn.body, { - stmt: (s) => { - switch (s.kind) { - case "varDecl": - define(s.localId, s.init); - break; - case "assign": - define(s.localId, s.value); - break; - case "forOf": - excluded.add(s.localId); - break; - case "rethrow": - excluded.add(s.localId); - break; - case "tryCatch": - guarded = true; - if (s.catchLocalId !== null) excluded.add(s.catchLocalId); - break; - case "return": - returns.push(s.value); - break; - } - return true; - }, - expr: (e) => { - switch (e.kind) { - case "assignExpr": - case "incDec": - excluded.add(e.localId); - break; - case "closure": - case "classRef": - for (const id of e.captures ?? []) excluded.add(id); - break; - } - return true; - }, + eachNodeOfKinds(fn, host.nodes?.scan(WALK_KINDS), (node) => { + switch (node.kind) { + case "varDecl": + define(node.localId, node.init); + break; + case "assign": + define(node.localId, node.value); + break; + case "forOf": + excluded.add(node.localId); + break; + case "rethrow": + excluded.add(node.localId); + break; + case "tryCatch": + guarded = true; + if (node.catchLocalId !== null) excluded.add(node.catchLocalId); + break; + case "return": + returns.push(node.value); + break; + case "assignExpr": + case "incDec": + excluded.add(node.localId); + break; + case "closure": + case "classRef": + for (const id of node.captures ?? []) excluded.add(id); + break; + } }); const plain = (id: string): boolean => { const local = locals.get(id); diff --git a/packages/compiler/src/backend/may-throw.ts b/packages/compiler/src/backend/may-throw.ts index a04cce856..0cfab9244 100644 --- a/packages/compiler/src/backend/may-throw.ts +++ b/packages/compiler/src/backend/may-throw.ts @@ -3,7 +3,8 @@ import type { IrExpr, IrStmt, IrModule, IrType, SrcLoc } from "../ir/ir.js"; import { isFfiCallbackParam, MAY_THROW_ARR_METHODS, MAY_THROW_BYTES_METHODS } from "../ir/ir.js"; import { MAY_THROW_LIB_FNS } from "../ir/builtin-effects.js"; -import { everyExprChild, everyStmtChild, everyStmtList } from "../ir/traverse.js"; +import { everyExprChild, everyStmtChild } from "../ir/traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex } from "../ir/node-index.js"; import { hasRetainedFfiCallback } from "./ffi-callbacks.js"; import { computePublishedTypes, writesPublished } from "./publication.js"; @@ -40,6 +41,8 @@ export function computeMayThrow( explain?: boolean; /** computePublishedTypes(mod), when the caller already has it. */ published?: ReturnType; + /** The module's node index, read instead of walking the bodies. */ + nodes?: IrNodeIndex; } = {}, ): { fns: Set; @@ -52,8 +55,9 @@ export function computeMayThrow( * entry (and may therefore throw the stack-overflow RangeError). */ stackChecks?: Set; } { - const terminations = mod.workers === true ? workerTerminationPolls(mod) : null; - const stackChecks = options.stackChecks === true ? stackCheckedFunctions(mod) : null; + const terminations = mod.workers === true ? workerTerminationPolls(mod, options.nodes) : null; + const stackChecks = + options.stackChecks === true ? stackCheckedFunctions(mod, options.nodes) : null; // Stores that can reach a published object throw the frozen TypeError. const published = options.published !== undefined ? options.published : computePublishedTypes(mod); @@ -898,7 +902,19 @@ function boundedNode(rec: IrExpr | IrStmt): boolean { * finishes after finitely many steps or reaches a polling callee. `polls` * names the functions that poll (and therefore may unwind with the * termination sentinel); `entry` the subset that polls on entry. */ -export function workerTerminationPolls(mod: IrModule): { +/** The kinds call-graph census below can skip: boundedNode answers true + * for them, and the census reads nothing else from them. */ +const CENSUS_SKIPPED_KINDS: ReadonlySet = new Set( + [...BOUNDED_KINDS].filter( + (kind) => + !["while", "doWhile", "for", "forOf", "call", "new", "virtualCall", "closure"].includes(kind), + ), +); + +export function workerTerminationPolls( + mod: IrModule, + nodes?: IrNodeIndex, +): { entry: Set; polls: Set; } { @@ -914,11 +930,12 @@ export function workerTerminationPolls(mod: IrModule): { const edges = new Map(); const entry = new Set(); const polls = new Set(); + const scan = nodes?.scanExcept(CENSUS_SKIPPED_KINDS); for (const fn of mod.functions) { const out: string[] = []; let bounded = fn.async !== true && fn.generator === undefined && fn.name !== mod.entry; let loops = false; - const visit = (rec: IrExpr | IrStmt): boolean => { + const visit = (rec: IrExpr | IrStmt): void => { if (!boundedNode(rec)) bounded = false; switch (rec.kind) { case "while": @@ -937,9 +954,8 @@ export function workerTerminationPolls(mod: IrModule): { out.push(...(methodImpls.get(rec.method) ?? [])); break; } - return true; }; - everyStmtList(fn.body, { expr: visit, stmt: visit }); + eachNodeOfKinds(fn, scan, visit); edges.set( fn.name, out.filter((callee) => names.has(callee)), @@ -960,7 +976,7 @@ export function workerTerminationPolls(mod: IrModule): { * library call; see boundedNode) also reaches every function that such an * operation can invoke: closure targets and class members. Non-recursive * functions, including leaf callbacks, stay check-free. */ -export function stackCheckedFunctions(mod: IrModule): Set { +export function stackCheckedFunctions(mod: IrModule, nodes?: IrNodeIndex): Set { const reentry = "\0reentry"; const methodImpls = new Map(); const memberPrefixes = new Set(); @@ -975,12 +991,13 @@ export function stackCheckedFunctions(mod: IrModule): Set { const names = new Set(mod.functions.map((fn) => fn.name)); const edges = new Map(); const targets = new Set(); + const scan = nodes?.scanExcept(CENSUS_SKIPPED_KINDS); for (const fn of mod.functions) { const dot = fn.name.indexOf("."); if (dot > 0 && memberPrefixes.has(fn.name.slice(0, dot + 1))) targets.add(fn.name); const out: string[] = []; let reenters = false; - const visit = (rec: IrExpr | IrStmt): boolean => { + const visit = (rec: IrExpr | IrStmt): void => { if (!boundedNode(rec)) reenters = true; switch (rec.kind) { case "call": @@ -996,9 +1013,8 @@ export function stackCheckedFunctions(mod: IrModule): Set { targets.add(rec.fnName); break; } - return true; }; - everyStmtList(fn.body, { expr: visit, stmt: visit }); + eachNodeOfKinds(fn, scan, visit); const callees = out.filter((callee) => names.has(callee)); if (reenters) callees.push(reentry); edges.set(fn.name, callees); diff --git a/packages/compiler/src/backend/native-link-info-core.ts b/packages/compiler/src/backend/native-link-info-core.ts index e018037fd..c073bb9a6 100644 --- a/packages/compiler/src/backend/native-link-info-core.ts +++ b/packages/compiler/src/backend/native-link-info-core.ts @@ -81,6 +81,9 @@ export interface NativeLinkFeatures { dnsPromises: boolean; /** dgram sockets as untyped values: scr_dgram_dyn.c. */ dgramDyn: boolean; + /** TextEncoder and TextDecoder records as untyped values: + * scr_text_codec_dyn.c. */ + textCodecDyn: boolean; watch: boolean; foreignFfi: boolean; nodeTest: boolean; diff --git a/packages/compiler/src/backend/native/contracts.ts b/packages/compiler/src/backend/native/contracts.ts index 61de82e72..53e160984 100644 --- a/packages/compiler/src/backend/native/contracts.ts +++ b/packages/compiler/src/backend/native/contracts.ts @@ -220,6 +220,10 @@ export interface CcOptions { * scr_dgram_dyn.c over the dgram unit, so typed dgram binaries keep * their exact link line. */ dgramDyn?: boolean; + /** A TextEncoder or TextDecoder record converts to or from an untyped + * value: compiles scr_text_codec_dyn.c, so typed codec binaries keep + * their exact link line. */ + textCodecDyn?: boolean; /** The program uses fs.watch (moduleUsesFsWatch on the IR): compiles * scr_watch.c into the binary — the net gating precedent, so watch-free diff --git a/packages/compiler/src/backend/native/executable.ts b/packages/compiler/src/backend/native/executable.ts index 233b9cac2..1578a0c80 100644 --- a/packages/compiler/src/backend/native/executable.ts +++ b/packages/compiler/src/backend/native/executable.ts @@ -302,6 +302,7 @@ const RUNTIME_UNITS: readonly RuntimeUnit[] = [ unit((u) => u.opts.dgram, "scr_dgram.c"), unit((u) => u.opts.dgram && u.opts.dnsPromises, "scr_dns_promises.c"), unit((u) => u.opts.dgram && u.opts.dgramDyn, "scr_dgram_dyn.c"), + unit((u) => u.opts.textCodecDyn, "scr_text_codec_dyn.c"), unit((u) => u.opts.watch, "scr_watch.c"), unit((u) => u.opts.foreignFfi, "scr_ffi_queue.c"), unit((u) => u.opts.nodeTest, "scr_test.c"), diff --git a/packages/compiler/src/backend/native/runtime-inputs.ts b/packages/compiler/src/backend/native/runtime-inputs.ts index bc9643670..6fb51fd32 100644 --- a/packages/compiler/src/backend/native/runtime-inputs.ts +++ b/packages/compiler/src/backend/native/runtime-inputs.ts @@ -72,6 +72,7 @@ export const EXECUTABLE_RUNTIME_SOURCES = [ "scr_itable.c", "scr_slots.c", "scr_alloc.c", + "scr_cipher_info.c", ] as const; export function runtimeSrcDir(): string { diff --git a/packages/compiler/src/backend/publication.test.ts b/packages/compiler/src/backend/publication.test.ts index 149ea4bb9..7b3e5f5cc 100644 --- a/packages/compiler/src/backend/publication.test.ts +++ b/packages/compiler/src/backend/publication.test.ts @@ -85,3 +85,34 @@ test("published types close over fields, containers and whole hierarchies", () = computePublishedTypes({ ...withStores, functions: withStores.functions.slice(1) }), ).toBeNull(); }); + +test("freezable array types carry the array write guard without a publish", () => { + const numbers = arrayOf(F64); + const push: IrStmt = { + kind: "exprStmt", + expr: { + kind: "arrIntrinsic", + method: "push", + receiver: ref("a", numbers), + args: [{ kind: "numLit", value: 1, type: F64, loc }], + type: F64, + loc, + }, + loc, + }; + const mod: IrModule = { + irVersion: 17, + sourceFile: loc.file, + entry: "main", + functions: [fn("main", []), fn("pushNumber", [push])], + freezableArrays: ["array"], + }; + const guarded = computePublishedTypes(mod)!; + expect(guarded.containerGuarded(numbers)).toBe(true); + expect(guarded.arrayGuards(numbers)).toEqual({ published: false, frozen: true }); + expect(guarded.containerGuarded(arrayOf(STRING))).toBe(false); + // A frozen array's push throws, so the function may throw. + expect(computeMayThrow(mod).fns.has("pushNumber")).toBe(true); + const { freezableArrays: _frozen, ...plain } = mod; + expect(computePublishedTypes(plain)).toBeNull(); +}); diff --git a/packages/compiler/src/backend/publication.ts b/packages/compiler/src/backend/publication.ts index af65fa39a..38e429a47 100644 --- a/packages/compiler/src/backend/publication.ts +++ b/packages/compiler/src/backend/publication.ts @@ -3,7 +3,7 @@ * emitter guards them. */ import type { IrExpr, IrModule, IrStmt, IrType } from "../ir/ir.js"; import { typeKey } from "../ir/ir.js"; -import { everyStmtList } from "../ir/traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "../ir/node-index.js"; import { computeConstructionFacts } from "./immutability.js"; /** Array methods that write their receiver (the guarded ones). */ @@ -46,6 +46,10 @@ export class PublishedTypes { readonly records = new Set(); /** typeKeys of array, map and set types. */ readonly containers = new Set(); + /** typeKeys of the array types the program may freeze + * (IrModule.freezableArrays): a frozen array refuses writes like a + * published one, so their stores carry the same guard. */ + readonly frozenArrays = new Set(); /** Constructor stores into a `this` nothing else can reach yet * (immutability.ts): never a published target, so never guarded. */ privateStores: ReadonlySet = new Set(); @@ -64,25 +68,42 @@ export class PublishedTypes { } containerGuarded(t: IrType): boolean { - return ( - (t.kind === "array" || t.kind === "map" || t.kind === "set") && - this.containers.has(typeKey(t)) - ); + if (t.kind !== "array" && t.kind !== "map" && t.kind !== "set") return false; + const key = typeKey(t); + return this.containers.has(key) || this.frozenArrays.has(key); + } + + /** Whether an array of type `t` may be published, and whether it may be + * frozen: the two states its write guard tests. */ + arrayGuards(t: IrType): { published: boolean; frozen: boolean } { + const key = typeKey(t); + return { published: this.containers.has(key), frozen: this.frozenArrays.has(key) }; } } -export function computePublishedTypes(mod: IrModule): PublishedTypes | null { +/** The nodes that publish a value: intrinsics. */ +const PUBLISHING_KINDS: readonly IrNodeKind[] = ["intrinsic"]; + +export function computePublishedTypes(mod: IrModule, nodes?: IrNodeIndex): PublishedTypes | null { const roots: IrType[] = []; - const visit = (e: IrExpr): boolean => { - // Values read from an InternTable were published by whichever thread - // stored them. - if (e.kind === "intrinsic" && (e.name === "threads.publish" || e.name === "itable.value")) - roots.push(e.type); - return true; - }; + const scan = nodes?.scan(PUBLISHING_KINDS); for (const fn of mod.functions) - everyStmtList(fn.body, { expr: visit, stmt: (_s: IrStmt) => true }); - if (roots.length === 0) return null; + eachNodeOfKinds(fn, scan, (node) => { + // Values read from an InternTable were published by whichever thread + // stored them. + if ( + node.kind === "intrinsic" && + (node.name === "threads.publish" || node.name === "itable.value") + ) + roots.push(node.type); + }); + const frozenArrays = mod.freezableArrays ?? []; + if (roots.length === 0 && frozenArrays.length === 0) return null; + if (roots.length === 0) { + const frozen = new PublishedTypes(); + for (const key of frozenArrays) frozen.frozenArrays.add(key); + return frozen; + } const classes = new Map((mod.classes ?? []).map((c) => [c.name, c])); const children = new Map(); @@ -95,7 +116,8 @@ export function computePublishedTypes(mod: IrModule): PublishedTypes | null { const records = new Map((mod.records ?? []).map((r) => [r.id, r])); const unions = new Map((mod.unions ?? []).map((u) => [u.id, u])); const out = new PublishedTypes(); - out.privateStores = computeConstructionFacts(mod).privateStores; + for (const key of frozenArrays) out.frozenArrays.add(key); + out.privateStores = computeConstructionFacts(mod, nodes).privateStores; const seen = new Set(); const stack = [...roots]; const addHierarchy = (className: string): void => { diff --git a/packages/compiler/src/backend/runtime-abi.ts b/packages/compiler/src/backend/runtime-abi.ts index 582e516ae..11bde6c2b 100644 --- a/packages/compiler/src/backend/runtime-abi.ts +++ b/packages/compiler/src/backend/runtime-abi.ts @@ -2,8 +2,8 @@ * of the link contract: a program object keeps this symbol undefined and the * matching runtime defines it, so incompatible manual links fail before the * program can start. */ -export const RUNTIME_ABI_VERSION = 36 as const; -export const RUNTIME_ABI_MARKER = "scr_runtime_abi_v36" as const; +export const RUNTIME_ABI_VERSION = 37 as const; +export const RUNTIME_ABI_MARKER = "scr_runtime_abi_v37" as const; /** Object consumption is useful today, but the complete scr_* surface may * still change before 1.0. The versioned marker prevents accidental mixing; diff --git a/packages/compiler/src/backend/runtime-pack-targets.test.ts b/packages/compiler/src/backend/runtime-pack-targets.test.ts index 739f6a385..9846e8d89 100644 --- a/packages/compiler/src/backend/runtime-pack-targets.test.ts +++ b/packages/compiler/src/backend/runtime-pack-targets.test.ts @@ -52,7 +52,7 @@ async function fixture(target: NativeTargetSpec, workers = false) { object_format: target.objectFormat, minimum_os: target.minimumOs, }, - runtime_abi: { version: 36, marker: "scr_runtime_abi_v36" }, + runtime_abi: { version: 37, marker: "scr_runtime_abi_v37" }, compiler: { command: "fixture", identity: "fixture", target: target.llvmTriple }, macros: { executable: [], excluded: ["SCR_LIB"], sanitizer: "external-toolchain-required" }, flavors: { @@ -118,6 +118,7 @@ const features = { dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, diff --git a/packages/compiler/src/backend/runtime-pack.test.ts b/packages/compiler/src/backend/runtime-pack.test.ts index 74bc32e73..e2b24f8ca 100644 --- a/packages/compiler/src/backend/runtime-pack.test.ts +++ b/packages/compiler/src/backend/runtime-pack.test.ts @@ -54,6 +54,7 @@ const BASE: NativeLinkFeatures = { dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, @@ -127,7 +128,7 @@ async function fixture(target: NativeTargetSpec = MACOS_ARM64_TARGET) { object_format: target.objectFormat, minimum_os: target.minimumOs, }, - runtime_abi: { version: 36, marker: "scr_runtime_abi_v36" }, + runtime_abi: { version: 37, marker: "scr_runtime_abi_v37" }, compiler: { command: "clang", identity: "fixture clang", diff --git a/packages/compiler/src/backend/stack-observations.ts b/packages/compiler/src/backend/stack-observations.ts index c448897f1..0709d588e 100644 --- a/packages/compiler/src/backend/stack-observations.ts +++ b/packages/compiler/src/backend/stack-observations.ts @@ -1,15 +1,32 @@ /* Whole-program stack-trace gate (see observesStacks). Pure function of the * IR module; the emitter keeps a source frame in every function only when * it answers true. */ -import { exprFunctionReference, moduleFunctionReferences } from "../ir/function-references.js"; +import { + exprFunctionReference, + FUNCTION_REFERENCE_KINDS, + moduleFunctionReferences, +} from "../ir/function-references.js"; +import { + eachNodeOfKinds, + type IrKindScan, + type IrNodeIndex, + type IrNodeKind, +} from "../ir/node-index.js"; import { MAY_THROW_LIB_FNS } from "../ir/builtin-effects.js"; import type { IrExpr, IrFunction, IrModule, IrStmt, IrType } from "../ir/ir.js"; import { MAY_THROW_ARR_METHODS } from "../ir/ir.js"; -import { everyExprChild, everyStmtChild, everyStmtList } from "../ir/traverse.js"; +import { everyExprChild, everyStmtChild } from "../ir/traverse.js"; import { unreachableStatements } from "./may-throw.js"; import type { PublishedTypes } from "./publication.js"; import { writesPublished } from "./publication.js"; +/** The nodes the scans below read: stack reads, the "stack" literal, call + * sites and function references, declarations. */ +const STACK_READ_KINDS: readonly IrNodeKind[] = ["libCall", "dynKeyGet"]; +const STRING_KINDS: readonly IrNodeKind[] = ["strLit"]; +const CENSUS_KINDS: readonly IrNodeKind[] = ["call", ...FUNCTION_REFERENCE_KINDS]; +const DECLARATION_KINDS: readonly IrNodeKind[] = ["varDecl"]; + /** Lib calls that read a captured stack trace. */ const STACK_READS: ReadonlySet = new Set(["error.stack"]); @@ -40,10 +57,13 @@ class StackKeys { // The module walks run on the first question that needs them: a program // without a dynamic key read asks none. - constructor(private readonly mod: IrModule) {} + constructor( + private readonly mod: IrModule, + private readonly nodes: IrNodeIndex | undefined, + ) {} private spellsStack(): boolean { - return (this.spelled ??= spellsStack(this.mod)); + return (this.spelled ??= spellsStack(this.mod, this.nodes)); } private callCensus(): { @@ -56,20 +76,18 @@ class StackKeys { const named = new Set(moduleFunctionReferences(mod)); // Class methods, accessors and constructors are called through dispatch. const classNames = new Set((mod.classes ?? []).map((cls) => cls.name)); + const scan = this.nodes?.scan(CENSUS_KINDS); for (const fn of mod.functions) { if (isClassMember(fn.name, classNames)) named.add(fn.name); - everyStmtList(fn.body, { - stmt: () => true, - expr: (expr) => { - if (expr.kind === "call") { - const sites = calls.get(expr.callee) ?? []; - sites.push({ caller: fn, args: expr.args }); - calls.set(expr.callee, sites); - } - const referenced = exprFunctionReference(expr); + eachNodeOfKinds(fn, scan, (node) => { + if (node.kind === "call") { + const sites = calls.get(node.callee) ?? []; + sites.push({ caller: fn, args: node.args }); + calls.set(node.callee, sites); + } else if (node.kind === "closure" || node.kind === "orDefault") { + const referenced = exprFunctionReference(node); if (referenced !== undefined) named.add(referenced); - return true; - }, + } }); } return (this.census = { calls, named }); @@ -127,13 +145,9 @@ class StackKeys { let inits = this.inits.get(fn); if (inits === undefined) { inits = new Map(); - everyStmtList(fn.body, { - stmt: (stmt) => { - if (stmt.kind === "varDecl") - inits!.set(stmt.localId, inits!.has(stmt.localId) ? null : (stmt.init ?? null)); - return true; - }, - expr: () => true, + eachNodeOfKinds(fn, this.nodes?.scan(DECLARATION_KINDS), (node) => { + if (node.kind === "varDecl") + inits!.set(node.localId, inits!.has(node.localId) ? null : (node.init ?? null)); }); this.inits.set(fn, inits); } @@ -161,21 +175,25 @@ function readsStack(fn: IrFunction, expr: IrExpr, keys: StackKeys): boolean { } /** Whether the module contains the string literal "stack". */ -function spellsStack(mod: IrModule): boolean { - return mod.functions.some( - (fn) => - !everyStmtList(fn.body, { - stmt: () => true, - expr: (expr) => !(expr.kind === "strLit" && expr.value === "stack"), - }), - ); +function spellsStack(mod: IrModule, nodes: IrNodeIndex | undefined): boolean { + const scan = nodes?.scan(STRING_KINDS); + return mod.functions.some((fn) => { + let found = false; + eachNodeOfKinds(fn, scan, (node) => { + found = node.kind === "strLit" && node.value === "stack"; + return !found; + }); + return found; + }); } -function containsStackRead(fn: IrFunction, keys: StackKeys): boolean { - return !everyStmtList(fn.body, { - stmt: () => true, - expr: (expr) => !readsStack(fn, expr, keys), +function containsStackRead(fn: IrFunction, keys: StackKeys, scan: IrKindScan | undefined): boolean { + let found = false; + eachNodeOfKinds(fn, scan, (node, statement) => { + found = !statement && readsStack(fn, node as IrExpr, keys); + return !found; }); + return found; } /** Visit every expression of `body` that execution can reach: statements @@ -199,9 +217,10 @@ function visitLiveExprs(body: IrStmt[], visit: (expr: IrExpr) => void): void { * Flight server keep development-only stack parsing behind `throw` stubs, * and those reads must not cost every call of the program a frame. * Programs without any stack read answer false without the walk. */ -export function observesStacks(mod: IrModule): boolean { - const keys = new StackKeys(mod); - if (!mod.functions.some((fn) => containsStackRead(fn, keys))) return false; +export function observesStacks(mod: IrModule, nodes?: IrNodeIndex): boolean { + const keys = new StackKeys(mod, nodes); + const reads = nodes?.scan(STACK_READ_KINDS); + if (!mod.functions.some((fn) => containsStackRead(fn, keys, reads))) return false; const byName = new Map(mod.functions.map((fn) => [fn.name, fn])); const classNames = new Set((mod.classes ?? []).map((cls) => cls.name)); const pending: string[] = [...moduleFunctionReferences(mod)]; diff --git a/packages/compiler/src/backend/warm-cache.ts b/packages/compiler/src/backend/warm-cache.ts index b48cf6739..02acb5462 100644 --- a/packages/compiler/src/backend/warm-cache.ts +++ b/packages/compiler/src/backend/warm-cache.ts @@ -61,6 +61,7 @@ export async function warmNativeCaches( dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, diff --git a/packages/compiler/src/cli/usage.ts b/packages/compiler/src/cli/usage.ts index 23e57c66f..c4d07656a 100644 --- a/packages/compiler/src/cli/usage.ts +++ b/packages/compiler/src/cli/usage.ts @@ -77,9 +77,9 @@ Options: --npm-static compile the named npm packages' shipped JS statically as program modules (repeatable; "auto" opts in every eligible - direct import: own .d.ts, unminified JS, no build-transform - markers; "pkg..." also names every package pkg depends - on). Packages that fail preflight are refused with a + direct import: own .d.ts, no build-transform markers; + "pkg..." also names every package pkg depends on). + Packages that fail preflight are refused with a diagnostic. Eligible dependencies compile automatically. --provenance-sources EXPERIMENTAL: compile npm dependencies from their diff --git a/packages/compiler/src/compat/fetch-profile.ts b/packages/compiler/src/compat/fetch-profile.ts index c4d1e3f50..ff7cedb22 100644 --- a/packages/compiler/src/compat/fetch-profile.ts +++ b/packages/compiler/src/compat/fetch-profile.ts @@ -900,7 +900,7 @@ export const NODE24_FETCH_COMPAT_PROFILE = { { name: "WritableStream/TransformStream", reason: - "native support currently covers readable fetch bodies; general Web Streams graphs are unsupported", + "native support covers readable fetch bodies and the identity `new TransformStream()`; the WritableStream constructor, transformers, and general Web Streams graphs are unsupported", }, { name: "Readable byte/BYOB streams", diff --git a/packages/compiler/src/executable/executable-cache.test.ts b/packages/compiler/src/executable/executable-cache.test.ts index 5cf8c6588..e066c0412 100644 --- a/packages/compiler/src/executable/executable-cache.test.ts +++ b/packages/compiler/src/executable/executable-cache.test.ts @@ -58,6 +58,7 @@ const native: EarlyExecutableNativeFeatures = { dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, diff --git a/packages/compiler/src/executable/executable-cache.ts b/packages/compiler/src/executable/executable-cache.ts index 7b7febc06..96922ca13 100644 --- a/packages/compiler/src/executable/executable-cache.ts +++ b/packages/compiler/src/executable/executable-cache.ts @@ -92,6 +92,7 @@ export interface EarlyExecutableNativeFeatures { dgram: boolean; dnsPromises: boolean; dgramDyn: boolean; + textCodecDyn: boolean; watch: boolean; foreignFfi: boolean; nodeTest: boolean; @@ -235,6 +236,7 @@ const BOOLEAN_NATIVE_KEYS = [ "dgram", "dnsPromises", "dgramDyn", + "textCodecDyn", "watch", "foreignFfi", "nodeTest", diff --git a/packages/compiler/src/frontend/decl-projection.ts b/packages/compiler/src/frontend/decl-projection.ts index f2340e984..eaf7486df 100644 --- a/packages/compiler/src/frontend/decl-projection.ts +++ b/packages/compiler/src/frontend/decl-projection.ts @@ -252,6 +252,16 @@ class TypePrinter { return props.length > 0 && props.every((p) => String(p.escapedName).startsWith("__@")); } + /** Whether an interface type and every type it extends are declared in + * user files. */ + private userInterface(type: ts.Type, seen = new Set()): boolean { + if (!declaredInUserFile(type.getSymbol())) return false; + const target = type.isTypeReference() ? type.getTarget() : type; + if (target === undefined || !target.isClassOrInterface() || seen.has(target)) return true; + seen.add(target); + return this.checker.getBaseTypes(target).every((base) => this.userInterface(base, seen)); + } + private printObject(type: ts.Type, depth: number): string | null { const checker = this.checker; if (checker.isArrayType(type)) { @@ -280,8 +290,11 @@ class TypePrinter { if (symbol.flags & (ts.SymbolFlags.Class | ts.SymbolFlags.Function | ts.SymbolFlags.Method)) return null; // Interfaces: only user-declared ones describe plain data reliably - // (package and lib interfaces often describe class instances). - if (symbol.flags & ts.SymbolFlags.Interface && !declaredInUserFile(symbol)) return null; + // (package and lib interfaces often describe class instances), and + // only when every interface they extend is user-declared too: a user + // interface extending a package one (effect's Request.Request) + // inherits members the package's own constructors create. + if (symbol.flags & ts.SymbolFlags.Interface && !this.userInterface(type)) return null; } if ( checker.getCallSignatures(type).length > 0 || diff --git a/packages/compiler/src/frontend/lowering/array-values.ts b/packages/compiler/src/frontend/lowering/array-values.ts index 314cdfd24..f191ad942 100644 --- a/packages/compiler/src/frontend/lowering/array-values.ts +++ b/packages/compiler/src/frontend/lowering/array-values.ts @@ -15,6 +15,7 @@ import { typeKey, unionContainerArmsOk, } from "../../ir/ir.js"; +import { everyExpr } from "../../ir/traverse.js"; import type { Lowerer } from "./lowerer.js"; import type * as ts from "../ts7/adapter.js"; @@ -193,6 +194,81 @@ export function unionArrayValueRead( return result; } +/** Expression kinds that evaluate without running other code or writing + * a binding. */ +const PURE_KINDS = new Set([ + "varRef", + "numLit", + "strLit", + "boolLit", + "unitLit", + "bin", + "unary", + "logical", + "ternary", + "toBool", + "fieldGet", + "recordGet", + "arrayGet", + "arrayState", + "arrayHas", + "unionIsTag", + "unionNarrow", + "unionWrap", +]); + +/** The locals evaluating `value` may assign, and whether it may run other + * code (anything but the pure kinds and the read-only element helpers), + * which can assign locals and fields through closures. */ +function valueEffects(lowerer: Lowerer, value: IrExpr): { writes: Set; opaque: boolean } { + const readers = new Set(); + for (const [key, name] of lowerer.arrHofHelpers) if (key.startsWith("idxOr:")) readers.add(name); + const writes = new Set(); + let opaque = false; + everyExpr(value, { + expr: (e) => { + if (e.kind === "assignExpr" || e.kind === "incDec") writes.add(e.localId); + else if (e.kind === "call" ? !readers.has(e.callee) : !PURE_KINDS.has(e.kind)) opaque = true; + return true; + }, + stmt: (s) => { + if (s.kind === "varDecl" || s.kind === "assign") writes.add(s.localId); + else opaque = true; + return true; + }, + }); + return { writes, opaque }; +} + +/** Whether evaluating `e` after `value` gives what it gives before it: a + * literal, or a local and field read `value` cannot change. */ +function unaffectedBy(e: IrExpr, effects: { writes: Set; opaque: boolean }): boolean { + switch (e.kind) { + case "numLit": + case "strLit": + return true; + case "varRef": + return !effects.writes.has(e.localId); + case "bin": + return unaffectedBy(e.left, effects) && unaffectedBy(e.right, effects); + case "fieldGet": + case "recordGet": + return !effects.opaque && unaffectedBy(e.obj, effects); + default: + return false; + } +} + +/** Whether `e` reads a `let` or `var` binding, which code the value runs + * can assign through a closure. */ +function readsMutableLocal(lowerer: Lowerer, e: IrExpr): boolean { + return !everyExpr(e, { + expr: (x) => + x.kind !== "varRef" || lowerer.ctx.locals.find((l) => l.id === x.localId)?.mutable === false, + stmt: () => true, + }); +} + /** Store a stabilized value without changing the array's element ABI. */ export function arrayValueStore( lowerer: Lowerer, @@ -210,10 +286,31 @@ export function arrayValueStore( // owned union temporary to release on every path. const unionId = value.type.unionId; let stable = value; - let prefix: IrStmt[] = []; + const prefix: IrStmt[] = []; + // The stores below evaluate the receiver and the index after the value. + // JavaScript evaluates them first (`out[n++] = src[n]` reads src[1]), so + // either may need its value held before the value runs. + if (value.kind !== "varRef") { + const valueFx = valueEffects(lowerer, value); + const hold = (e: IrExpr, name: string, effects: typeof valueFx): IrExpr => { + if (unaffectedBy(e, effects) && !(effects.opaque && readsMutableLocal(lowerer, e))) + return e; + const temp = lowerer.declareHiddenLocal(name, e.type); + prefix.push({ kind: "varDecl", localId: temp.id, init: e, loc }); + return varRef(temp.id, temp.type, loc); + }; + // The receiver runs before the index and the value, so the index's + // effects can change it too (`a[(a = b, 0)] = c[0]!`). + const indexFx = valueEffects(lowerer, index); + arr = hold(arr, "%arrayTarget", { + writes: new Set([...valueFx.writes, ...indexFx.writes]), + opaque: valueFx.opaque || indexFx.opaque, + }); + index = hold(index, "%arrayIndex", valueFx); + } if (value.kind !== "varRef") { const temp = lowerer.declareHiddenLocal("%arrayValue", value.type); - prefix = [{ kind: "varDecl", localId: temp.id, init: value, loc }]; + prefix.push({ kind: "varDecl", localId: temp.id, init: value, loc }); stable = varRef(temp.id, temp.type, loc); } const write: IrStmt = { diff --git a/packages/compiler/src/frontend/lowering/builtins/crypto.ts b/packages/compiler/src/frontend/lowering/builtins/crypto.ts index 47e66f9b2..051ec18c4 100644 --- a/packages/compiler/src/frontend/lowering/builtins/crypto.ts +++ b/packages/compiler/src/frontend/lowering/builtins/crypto.ts @@ -869,9 +869,17 @@ const IMPLEMENTED_CIPHERS = new Set([ "id-aes192-gcm", "id-aes256-gcm", "chacha20-poly1305", - ...["128", "192", "256"].flatMap((bits) => - ["ecb", "cfb", "cfb8", "cfb1", "ofb"].map((mode) => `aes-${bits}-${mode}`), - ), + ...["128", "192", "256"].flatMap((bits) => [ + ...["ecb", "cfb", "cfb8", "cfb1", "ofb", "ccm"].map((mode) => `aes-${bits}-${mode}`), + `id-aes${bits}-ccm`, + ...["wrap", "wrap-pad"].flatMap((wrap) => [ + `aes-${bits}-${wrap}`, + `aes${bits}-${wrap}`, + `id-aes${bits}-${wrap}`, + `aes-${bits}-${wrap}-inv`, + `aes${bits}-${wrap}-inv`, + ]), + ]), "des-ede3-cbc", "des3", "des-ede3", @@ -889,9 +897,10 @@ const IMPLEMENTED_CIPHERS = new Set([ ]); const SUPPORTED_CIPHERS_HINT = - "the supported ciphers are aes-128, aes-192 and aes-256 in cbc, ctr, gcm, ecb, cfb, cfb8, cfb1 " + - "and ofb modes, des-ede3 in cbc, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede in cbc, ecb, cfb " + - "and ofb modes, chacha20, and chacha20-poly1305"; + "the supported ciphers are aes-128, aes-192 and aes-256 in cbc, ctr, gcm, ccm, ecb, cfb, cfb8, " + + "cfb1 and ofb modes and their wrap, wrap-pad, wrap-inv and wrap-pad-inv key wraps, des-ede3 in " + + "cbc, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede in cbc, ecb, cfb and ofb modes, chacha20, and " + + "chacha20-poly1305"; /** A literal cipher name that Node knows but the runtime does not * implement is refused at compile time. Unknown literal names keep Node's @@ -1380,10 +1389,11 @@ function takesUndefinedError(lowerer: Lowerer, callback: ts.Expression): boolean } /** checkPrime, checkPrimeSync, generatePrime and generatePrimeSync - * (scr_dh.c): checked-dynamic arguments, which the runtime checks as Node - * does (a function in the options position is the callback), and the - * sync forms' boolean, ArrayBuffer or bigint converted to the call site's - * type. Null for other members. */ + * (scr_dh.c), and getCipherInfo (scr_cipher_info.c): checked-dynamic + * arguments, which the runtime checks as Node does (a function in the + * options position is the callback), and the sync forms' boolean, + * ArrayBuffer, bigint or info object converted to the call site's type. + * Null for other members. */ function lowerPrimeCall( lowerer: Lowerer, expr: ts.CallExpression, @@ -1395,6 +1405,7 @@ function lowerPrimeCall( checkPrimeSync: ["crypto.checkPrimeSync", 1, 2, "checkPrimeSync(candidate[, options])"], checkPrime: ["crypto.checkPrimeCb", 1, 3, "checkPrime(candidate[, options], callback)"], generatePrimeSync: ["crypto.generatePrimeSync", 1, 2, "generatePrimeSync(size[, options])"], + getCipherInfo: ["crypto.getCipherInfo", 0, 2, "getCipherInfo(nameOrNid[, options])"], generatePrime: ["crypto.generatePrimeCb", 1, 3, "generatePrime(size[, options], callback)"], }; const form = forms[member]; @@ -1455,14 +1466,10 @@ function lowerPkiCall( expr, `${supported} is supported`, ); - // generateKeyPair(type, callback): Node reads a function in the options - // position as the callback. - const shifted = fn === "crypto.generateKeyPairCb" && args.length === 2; + // generateKeyPair(type, callback): the runtime reads a function in the + // options position as the callback, as Node does. const values: IrExpr[] = []; - for (let i = 0; i < max; i++) { - const node = shifted ? (i === 1 ? undefined : i === 2 ? args[1] : args[i]) : args[i]; - values.push(dynArg(lowerer, node, loc)); - } + for (let i = 0; i < max; i++) values.push(dynArg(lowerer, args[i], loc)); if (fn === "crypto.generateKeyPairCb") return { kind: "libCall", fn, args: values, type: VOID, loc }; const call: IrExpr = { kind: "libCall", fn, args: values, type: DYN, loc }; diff --git a/packages/compiler/src/frontend/lowering/builtins/readline.ts b/packages/compiler/src/frontend/lowering/builtins/readline.ts index 846b9a926..aca209017 100644 --- a/packages/compiler/src/frontend/lowering/builtins/readline.ts +++ b/packages/compiler/src/frontend/lowering/builtins/readline.ts @@ -6,9 +6,11 @@ import { DYN, F64, type IrExpr, + type IrType, type IrStmt, type SrcLoc, STRING, + UNDEFINED_T, VOID, arrayOf, BYTES_U8, @@ -54,6 +56,21 @@ export function isReadlineTyped(lowerer: Lowerer, node: ts.Expression): boolean }); } +/** `rl.cursor`: @types/node declares a number, but an interface that is + * not a terminal answers undefined, so the read keeps its + * `number | undefined` union. */ +export function isOptionalReadlineCursorRead( + lowerer: Lowerer, + expr: ts.PropertyAccessExpression, +): boolean { + return ( + expr.name.text === "cursor" && + !expr.questionDotToken && + isReadlineTyped(lowerer, expr.expression) && + lowerer.isStdlibMember(expr) + ); +} + /** The runtime's listener event numbers (scr_readline.c). */ const RL_EVENTS: Record = { line: 0, @@ -1341,7 +1358,7 @@ export function lowerReadlineCreateCall( /** Property reads on an Interface: `history` (the history option itself, * or the interface's own empty array), `historySize`, `historyIndex`, * `crlfDelay`, `tabSize`, `escapeCodeTimeout`, `line`, `terminal` and - * `cursor` (a non-terminal interface's undefined cursor throws SC2020). + * `cursor` (undefined on a non-terminal interface). * Null for other receivers. */ export function lowerReadlineProperty( lowerer: Lowerer, @@ -1373,10 +1390,38 @@ export function lowerReadlineProperty( loc, }; } - const reads: Record = { + if (name === "cursor") { + // `number | undefined`: a non-terminal interface has no cursor + const recv = lowerer.lowerExprExpecting(access.expression, F64); + const type: IrType = { kind: "union", unionId: lowerer.unions.intern([F64, UNDEFINED_T]) }; + const local = recv.kind === "varRef" ? null : lowerer.declareHiddenLocal("%rlCursor", F64); + const id: IrExpr = local ? { kind: "varRef", localId: local.id, type: F64, loc } : recv; + const result: IrExpr = { + kind: "ternary", + cond: { kind: "libCall", fn: "rl.isTerminal", args: [id], type: BOOL, loc }, + then: lowerer.coerceToExpected( + { kind: "libCall", fn: "rl.cursor", args: [id], type: F64, loc }, + type, + ), + else_: lowerer.coerceToExpected( + { kind: "unitLit", unit: "undefined", type: UNDEFINED_T, loc }, + type, + ), + type, + loc, + }; + if (!local) return result; + return { + kind: "seqExpr", + stmts: [{ kind: "varDecl", localId: local.id, init: recv, loc }], + result, + type, + loc, + }; + } + const reads: Record = { line: ["rl.line", STRING], terminal: ["rl.isTerminal", BOOL], - cursor: ["rl.cursor", F64], }; const read = reads[name]; if (read !== undefined) { diff --git a/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.test.ts b/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.test.ts index 1a21a6aa5..bbdf271c0 100644 --- a/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.test.ts +++ b/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.test.ts @@ -560,3 +560,53 @@ test("class branches read their exact class's bag, also for classes with subclas then: [{ kind: "return", value: { kind: "dynObjLit", fields: [] } }], }); }); + +test("records a class's bag initializer only when all its class-table branches agree", () => { + const { lowerer } = context(); + const classBagInitializers = new Map(); + Object.assign(lowerer, { classBagInitializers }); + for (const name of ["Material", "Texture"]) + lowerer.classes.set(name, { def: { name, tracksOwnFields: true } } as unknown as ClassInfo); + const initializer = ( + className: string, + prototype: string, + start: number, + ): Extract => { + const at = { ...loc, start }; + return { + kind: "fieldSet", + obj: { + kind: "dynCheck", + value: { kind: "varRef", localId: "p.0", type: DYN, loc: at }, + type: { kind: "object", className }, + loc: at, + }, + className, + field: "%bag", + value: { + kind: "libCall", + fn: "dyn.objCreate", + args: [{ kind: "call", callee: prototype, args: [], type: DYN, loc: at }], + type: DYN, + loc: at, + }, + loc: at, + }; + }; + const dispatch = new ClassDynamicDispatch(); + const tables = (dispatch as unknown as { bagInitializers: Map }) + .bagInitializers; + // A base class whose own instances get its prototype and whose subclass + // instances get theirs through a per-instance dispatch has no single bag. + tables.set("Material", [ + initializer("Material", "%prototype.data.Material", 1), + initializer("Material", "%class.bagPrototype:Material", 2), + ]); + // Branches that differ only in source location agree. + tables.set("Texture", [ + initializer("Texture", "%prototype.data.Texture", 3), + initializer("Texture", "%prototype.data.Texture", 4), + ]); + dispatch.finalize(lowerer); + expect([...classBagInitializers.keys()]).toEqual(["Texture"]); +}); diff --git a/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts b/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts index b58dcae51..abf03811e 100644 --- a/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts +++ b/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts @@ -282,6 +282,11 @@ interface PropertyDispatch { fn: IrFunction; classes: Set; reflect?: boolean; + /** The helper's fallthrough statements as created, and the same access + * applied to the receiver itself: what the fallthrough reduces to for a + * value that is neither a class capsule nor a function, whose bag is the + * value. */ + plain?: { tail: readonly IrStmt[]; access: IrStmt[] }; } export function classInstanceOf( @@ -982,6 +987,11 @@ function initializeBag(lowerer: Lowerer, className: string, self: IrExpr, loc: S ]; } +/** `value` as JSON without its source locations. */ +function withoutLocations(value: IrExpr): string { + return JSON.stringify(value, (key, field: unknown) => (key === "loc" ? undefined : field)); +} + /** Whether `value` reads local `id`. */ function mentionsLocal(value: IrExpr, id: string): boolean { return JSON.stringify(value).includes(`"localId":${JSON.stringify(id)}`); @@ -1502,6 +1512,7 @@ export class ClassDynamicDispatch { }); changed = true; } + let live: Map[]> | null = null; for (const plan of this.propertyReceivers(lowerer, false, false)) { const info = plan.info; const className = info.def.name; @@ -1582,7 +1593,11 @@ export class ClassDynamicDispatch { ); const prototypeFor = (current: ClassInfo, receiver: IrExpr, loc: SrcLoc): IrExpr | null => this.bagPrototype(lowerer, current, receiver, loc); - for (const existingInit of this.bagInitializers.get(className) ?? []) { + live ??= this.liveBagInitializers(); + for (const existingInit of [ + ...(this.bagInitializers.get(className) ?? []), + ...(live.get(className) ?? []), + ]) { if (existingInit.value.kind === "dynObjLit" && needsPrototype(info)) { const prototype = prototypeFor(info, existingInit.obj, existingInit.loc); if (prototype) { @@ -2714,7 +2729,28 @@ export class ClassDynamicDispatch { loc, }); } - dispatch = { name, write: !!write, fn: helper, classes: new Set() }; + const receiverAccess: IrStmt[] = read + ? [{ kind: "return", value: { ...read, value: varRef("p.0", DYN, loc) }, loc }] + : [ + { + kind: "exprStmt", + expr: { + ...write!, + args: [varRef("p.0", DYN, loc), write!.args[1]!, varRef("p.1", DYN, loc)], + }, + loc, + }, + { kind: "return", value: null, loc }, + ]; + dispatch = { + name, + write: !!write, + fn: helper, + classes: new Set(), + ...(name !== "constructor" && name !== "toString" + ? { plain: { tail: [...helper.body], access: receiverAccess } } + : {}), + }; this.properties.set(key, dispatch); this.generated.add(helper); lowerer.liftedFns.push(helper); @@ -4767,6 +4803,27 @@ export class ClassDynamicDispatch { * A class with constructible subclasses that have prototype data selects * it through its `%class.bagPrototype` helper; any other class reads its * own prototype data. */ + /** The lazy bag stores as they stand in the generated bodies, by class. + * Worklist passes that transform every function rebuild statement + * objects, so a recorded initializer can be a detached copy; a store that + * gains its prototype must be updated where it runs. */ + private liveBagInitializers(): Map[]> { + const live = new Map[]>(); + const add = (stmt: IrStmt | undefined): void => { + if (stmt?.kind !== "if") return; + const store = stmt.then[0]; + if (store?.kind !== "fieldSet" || store.field !== PROPERTY_BAG) return; + const list = live.get(store.className); + if (list) list.push(store); + else live.set(store.className, [store]); + }; + // The complete dispatch guards each class's lazy store by its receiver. + for (const stmt of this.propertyBag?.body ?? []) if (stmt.kind === "if") add(stmt.then[0]); + for (const helper of this.typedPropertyBags.values()) add(helper.body[0]); + for (const helper of this.exactPropertyBags.values()) add(helper.body[0]); + return live; + } + private bagPrototype( lowerer: Lowerer, current: ClassInfo, @@ -4865,6 +4922,7 @@ export class ClassDynamicDispatch { ); } if (lowerer) this.guardClassTables(lowerer); + if (lowerer) this.guardPropertyHelpers(lowerer); } /** Each tracked class's bag initializer over a typed receiver (`this`), @@ -4877,6 +4935,18 @@ export class ClassDynamicDispatch { const self = varRef("this", { kind: "object", className }, init.loc); const value = withTypedReceiver(structuredClone(init.value), self); if (mentionsLocal(value, "p.0")) continue; + // The class-table branches can give the class's instances different + // bags (a base class's own prototype for its instances, the + // instance's class prototype for subclass instances); only a class + // whose branches all agree has one initializer for every instance. + const shape = withoutLocations(value); + if ( + initializers.some( + (other) => + withoutLocations(withTypedReceiver(structuredClone(other.value), self)) !== shape, + ) + ) + continue; // Lowerer stand-ins in unit tests may not carry the table. lowerer.classBagInitializers?.set(className, value); } @@ -4942,6 +5012,87 @@ export class ClassDynamicDispatch { } } + /** Fast exits for the generated property helpers: when every class + * branch tests a capsule key or a class key and the fallthrough is still + * the bag access the helper was created with, a value of another kind + * (a plain object, array or primitive) normalizes to itself, has itself + * as its bag, and so gets the plain access directly, without the + * normalization, bag and static-bag calls. */ + private guardPropertyHelpers(lowerer: Lowerer): void { + // Every class table the fallthrough or normalization calls must answer + // a plain value with the value itself: its kind exit is in place. + const exits = (name: string): boolean => { + const helper = lowerer.liftedFnNamed(name); + const first = helper?.body[0]; + const param = helper?.params[0]?.localId; + return ( + param !== undefined && + first?.kind === "if" && + first.cond.kind === "dynTest" && + first.cond.negated === true && + first.then.length === 1 && + first.then[0]!.kind === "return" && + first.then[0]!.value !== null && + returnsParam(first.then[0]!.value, param) + ); + }; + const tables = new Set([ + "%dyn.class.properties", + READ_PROPERTIES, + READ_PROPERTIES_OR_EMPTY, + "%dyn.class.normalize", + "%dyn.class.staticProperties", + ]); + const callsOnlyGuardedTables = (statements: IrStmt[]): boolean => + everyStmtList(statements, { + stmt: () => true, + expr: (expression) => + expression.kind !== "call" || (tables.has(expression.callee) && exits(expression.callee)), + }); + for (const dispatch of this.properties.values()) { + const plain = dispatch.plain; + if (!plain) continue; + const fn = dispatch.fn; + const param = fn.params[0]?.localId; + if (param === undefined) continue; + const head = fn.body.length - plain.tail.length; + if (head < 0 || plain.tail.some((stmt, i) => fn.body[head + i] !== stmt)) continue; + let ok = true; + for (const stmt of fn.body.slice(0, head)) { + if ( + stmt.kind === "assign" && + stmt.localId === param && + stmt.value.kind === "call" && + stmt.value.callee === "%dyn.class.normalize" + ) + continue; + if (stmt.kind === "if" && stmt.else_ === null && branchKind(stmt.cond, param) !== null) + continue; + ok = false; + break; + } + // Class branches never run for a plain value; the rest always does. + if (!ok || !callsOnlyGuardedTables(fn.body.filter((stmt) => stmt.kind !== "if"))) continue; + const loc = fn.loc; + const not = (test: "classCapsule" | "funcKind"): IrExpr => ({ + kind: "dynTest", + test, + negated: true, + value: varRef(param, DYN, loc), + type: BOOL, + loc, + }); + fn.body.unshift({ + kind: "if", + cond: not("classCapsule"), + then: [{ kind: "if", cond: not("funcKind"), then: plain.access, else_: null, loc }], + else_: null, + loc, + }); + delete dispatch.plain; + } + } + private normalizedDispatches(): IrFunction[] { return [ ...(this.propertyBag ? [this.propertyBag] : []), diff --git a/packages/compiler/src/frontend/lowering/containers/string-replacement.ts b/packages/compiler/src/frontend/lowering/containers/string-replacement.ts index 780c77d0f..156f74869 100644 --- a/packages/compiler/src/frontend/lowering/containers/string-replacement.ts +++ b/packages/compiler/src/frontend/lowering/containers/string-replacement.ts @@ -2,6 +2,7 @@ import { keyedHelperName } from "../stable-names.js"; import * as ts from "../../ts7/adapter.js"; import { BOOL, + DYN, F64, REGEX, STRING, @@ -14,7 +15,7 @@ import { type IrType, type SrcLoc, } from "../../../ir/ir.js"; -import { numLit, strLit, varRef } from "../../../ir/build.js"; +import { dynUndefinedExpr, numLit, strLit, varRef } from "../../../ir/build.js"; import { InternalCompilerError } from "../../../errors.js"; import { locOf } from "../../program.js"; import type { Lowerer } from "../lowerer.js"; @@ -65,14 +66,14 @@ export function lowerStringReplacement( const search = argument(args[0], false); const replacement = argument(args[1], true); if (search.type.kind === "dyn" || replacement.type.kind === "dyn") { - if (replacement.type.kind === "func") { - lowerer.noLowering(`string .${method} with checked search values and callbacks`, call); - } // Evaluate all inputs once before choosing the search protocol. Regexes // retain their native pattern; plain values use the string-search helper. + // A typed callback receives the regex match's arguments or the plain + // search's (match, offset, subject). const values = [receiver, search, replacement]; const locals = values.map((value) => lowerer.declareHiddenLocal("%replaceArg", value.type)); const refs = locals.map((local) => varRef(local.id, local.type, loc)); + const callback = replacement.type.kind === "func"; const checkedReplacement = replacement.type.kind === "dyn" ? lowerer.coerceInto(args[1] ?? call, refs[2]!, STRING) @@ -91,14 +92,27 @@ export function lowerStringReplacement( ? { kind: "ternary", cond: { kind: "libCall", fn: "dyn.nativeRegexIs", args: [refs[1]!], type: BOOL, loc }, - then: { - kind: "regexIntrinsic", - method, - receiver: refs[0]!, - args: [lowerer.coerceInto(args[0] ?? call, refs[1]!, REGEX), checkedReplacement], - type: STRING, - loc, - }, + then: callback + ? { + kind: "libCall", + fn: "regex.replaceCallback", + args: [ + refs[0]!, + lowerer.coerceInto(args[0] ?? call, refs[1]!, REGEX), + lowerer.coerceToExpected(refs[2]!, DYN), + { kind: "boolLit", value: method === "replaceAll", type: BOOL, loc }, + ], + type: STRING, + loc, + } + : { + kind: "regexIntrinsic", + method, + receiver: refs[0]!, + args: [lowerer.coerceInto(args[0] ?? call, refs[1]!, REGEX), checkedReplacement], + type: STRING, + loc, + }, else_: plain, type: STRING, loc, @@ -148,11 +162,17 @@ function lowerStringReplacementValues( if (callback.kind !== "func") { throw new InternalCompilerError(`string .${method} replacement of type ${callback.kind}`); } + // A checked parameter receives the boxed argument (undefined after the + // subject), and a checked, number or boolean result converts with + // ToString. const full = [STRING, F64, STRING]; + const ret = callback.ret.kind; if ( - callback.ret.kind !== "string" || - callback.params.length > full.length || - !callback.params.every((type, index) => typeEquals(type, full[index]!)) + (ret !== "string" && ret !== "dyn" && ret !== "f64" && ret !== "bool") || + !callback.params.every( + (type, index) => + type.kind === "dyn" || (index < full.length && typeEquals(type, full[index]!)), + ) ) { lowerer.noLowering( `string .${method} callback signature`, @@ -175,13 +195,21 @@ function lowerStringReplacementValues( const chunksType = arrayOf(STRING); const chunks = varRef("chunks.0", chunksType, loc); const builder = stringBuilder(loc, chunks); - const value: IrExpr = { + const result: IrExpr = { kind: "callValue", callee: replace, - args: [needle, match, subject].slice(0, callback.params.length), - type: STRING, + args: callback.params.map((type, index) => { + const arg = [needle, match, subject][index]; + if (arg === undefined) return dynUndefinedExpr(loc); + return type.kind === "dyn" ? lowerer.coerceToExpected(arg, DYN) : arg; + }), + type: callback.ret, loc, }; + const value: IrExpr = + ret === "dyn" + ? { kind: "libCall", fn: "dyn.toStringCoerce", args: [result], type: STRING, loc } + : lowerer.ensureString(result, call); const stop: IrStmt = { kind: "break", loc }; const body: IrStmt[] = [ builder.declare("chunks.0", { kind: "arrayLit", elems: [], type: chunksType, loc }), diff --git a/packages/compiler/src/frontend/lowering/expressions/binary.ts b/packages/compiler/src/frontend/lowering/expressions/binary.ts index 58f6e16b5..17aef7b85 100644 --- a/packages/compiler/src/frontend/lowering/expressions/binary.ts +++ b/packages/compiler/src/frontend/lowering/expressions/binary.ts @@ -2841,6 +2841,24 @@ function lowerLogicalPair(lowerer: Lowerer, expr: ts.BinaryExpression, left: IrE loc, }; } + // A JavaScript default that tsc types `any` (`e.title || null` in an + // instance of a function with untyped parameters, where `e.title` is a + // string): the checked domain, as for union operands above. + if ( + target === null && + isJsSourceFile(ts.getSourceFileOfNode(expr)) && + lowerer.dynConvertible(left.type) && + lowerer.dynConvertible(right.type) + ) { + return { + kind: "logical", + op: op === ts.SyntaxKind.AmpersandAmpersandToken ? "&&" : "||", + left: lowerer.coerceInto(expr.left, left, DYN), + right: lowerer.coerceInto(expr.right, right, DYN), + type: DYN, + loc, + }; + } lowerer.unsupported("SC1042", expr); } return { diff --git a/packages/compiler/src/frontend/lowering/expressions/element-access.ts b/packages/compiler/src/frontend/lowering/expressions/element-access.ts index 0ea94ec7a..9a7ebf401 100644 --- a/packages/compiler/src/frontend/lowering/expressions/element-access.ts +++ b/packages/compiler/src/frontend/lowering/expressions/element-access.ts @@ -1028,7 +1028,7 @@ function lowerGuardedRecordWrite( const value = stabilized(lowerer, lowerer.lowerExpr(expr.right), "%recordValue", receiver.stmts); const write = lowerer.withExpressionOverride(target.expression, receiver.present, () => lowerer.withExpressionOverride(target.argumentExpression, key, () => - lowerer.withExpressionOverride(expr.right, value, () => lowerElementWrite(lowerer, expr)), + lowerer.withExpressionOverride(expr.right, value, () => lowerer.lowerElementWrite(expr)), ), ); return { diff --git a/packages/compiler/src/frontend/lowering/expressions/narrowing.ts b/packages/compiler/src/frontend/lowering/expressions/narrowing.ts index fa8ad686f..01cab9901 100644 --- a/packages/compiler/src/frontend/lowering/expressions/narrowing.ts +++ b/packages/compiler/src/frontend/lowering/expressions/narrowing.ts @@ -33,6 +33,7 @@ import { isOptionalProcessVersionsRead, } from "../builtins/process.js"; import { requestGeneralStreamSurface } from "../general-streams.js"; +import { isOptionalReadlineCursorRead } from "../builtins/readline.js"; import { literalValues } from "../../literal-values.js"; import { literalUnionArm } from "../../union-discriminants.js"; import { checkedArmExtraction } from "../coercions/unions.js"; @@ -617,6 +618,8 @@ export function maybeNarrow(lowerer: Lowerer, expr: IrExpr, node: ts.Node): IrEx if (ts.isPropertyAccessExpression(node) && isOptionalProcessStreamProperty(lowerer, node)) return expr; if (isOptionalProcessVersionsRead(lowerer, node)) return expr; + if (ts.isPropertyAccessExpression(node) && isOptionalReadlineCursorRead(lowerer, node)) + return expr; // Represented fields and index values can include undefined even when // checker inference or lib.d.ts declares strings (RegExp captures). diff --git a/packages/compiler/src/frontend/lowering/expressions/nullish-assignment.ts b/packages/compiler/src/frontend/lowering/expressions/nullish-assignment.ts index 81b3b9d69..9664f591d 100644 --- a/packages/compiler/src/frontend/lowering/expressions/nullish-assignment.ts +++ b/packages/compiler/src/frontend/lowering/expressions/nullish-assignment.ts @@ -100,7 +100,9 @@ export function lowerShortCircuitAssignment(lowerer: Lowerer, expr: ts.BinaryExp write = (value) => eventAccessorWrite(receiver, name, value, lowerer, loc); } else if (ts.isPropertyAccessExpression(target)) { const probed = tryLowerExpression(lowerer, target.expression); - if (probed?.type.kind === "dyn") { + // `value.#x ??= v` on a checked-dynamic value names the brand-checked + // field (fieldTarget), never a property called "#x". + if (probed?.type.kind === "dyn" && !ts.isPrivateIdentifier(target.name)) { const receiver = save(probed, "%shortCircuitReceiver"); const key: IrExpr = { kind: "strLit", value: target.name.text, type: STRING, loc }; storage = DYN; diff --git a/packages/compiler/src/frontend/lowering/expressions/property-access.ts b/packages/compiler/src/frontend/lowering/expressions/property-access.ts index 7b12297a8..b0c7a66bb 100644 --- a/packages/compiler/src/frontend/lowering/expressions/property-access.ts +++ b/packages/compiler/src/frontend/lowering/expressions/property-access.ts @@ -1499,6 +1499,25 @@ export function lowerPropertyAccessExpr( ) { return afterReceiver(recvLowered, dynUndefinedExpr(loc), loc); } + // A typed array under a checker `any` (a JS binding its own initializer's + // inference refers to: hono's etag digest `nextChunk`) reads through its + // box, as the method-call twin dispatches. The box shares the native + // storage, so it needs no fresh-result restriction. + if ( + recvLowered.type.kind === "bytes" && + (lowerer.typeOf(expr.expression).flags & ts.TypeFlags.Any) !== 0 + ) { + return lowerer.maybeNarrow( + { + kind: "dynKeyGet", + value: lowerer.coerceToExpected(recvLowered, DYN), + key: { kind: "strLit", value: expr.name.text, type: STRING, loc }, + type: DYN, + loc, + }, + expr, + ); + } lowerer.unsupported( "SC1090", expr, diff --git a/packages/compiler/src/frontend/lowering/expressions/relational.ts b/packages/compiler/src/frontend/lowering/expressions/relational.ts index ece8731c7..1651bf0d9 100644 --- a/packages/compiler/src/frontend/lowering/expressions/relational.ts +++ b/packages/compiler/src/frontend/lowering/expressions/relational.ts @@ -338,6 +338,22 @@ export function lowerInstanceOf(lowerer: Lowerer, expr: ts.BinaryExpression, loc type: BOOL, loc, }; + // A codec in untyped storage is its record's handle box. + const codec = (["TextEncoder", "TextDecoder"] as const).find((name) => + lowerer.isStdlibGlobal(expr.right, name), + ); + if (codec && lowerer.mapTypeOf(lowerer.typeOf(expr.left))?.kind === "dyn") { + return { + kind: "libCall", + fn: "text.codecIs", + args: [ + lowerer.lowerExprExpecting(expr.left, DYN), + { kind: "boolLit", value: codec === "TextDecoder", type: BOOL, loc }, + ], + type: BOOL, + loc, + }; + } if (lowerer.isStdlibGlobal(expr.right, "ReadableStream")) { const value = lowerer.lowerExprExpecting(expr.left, DYN); return { kind: "libCall", fn: "fetch.streamIs", args: [value], type: BOOL, loc }; @@ -438,7 +454,58 @@ export function lowerInstanceOf(lowerer: Lowerer, expr: ts.BinaryExpression, loc // (itself — no descendants can flow into the slot), so the answer // folds statically exactly like the named-target folds below. const rhsClassval = storedClassValueType(lowerer, expr.right); - if (rhsClassval?.kind === "classval" && !lowerer.caughtLocalOf(expr.left)) { + // A catch binding against a class held by a binding (`var E = class + // extends Error {}`). The binding holds the declared class or one of its + // subclasses. Without subclasses the test is the class's caught test, + // after the right operand's read; otherwise a payload of the declared + // class is tested against the class the binding holds. + const caught = lowerer.caughtLocalOf(expr.left); + const caughtTarget = + rhsClassval?.kind === "classval" && caught + ? lowerer.classes.get(rhsClassval.className) + : undefined; + if (caught && caughtTarget && !caughtTarget.localClass && !caughtTarget.def.runtime) { + const classValue = lowerer.lowerExpr(expr.right); + if (caughtTarget.subclasses.length === 0) { + return { + kind: "seqExpr", + stmts: [{ kind: "exprStmt", expr: classValue, loc }], + result: caughtClassInstanceOf(lowerer, caught.id, caughtTarget, loc), + type: BOOL, + loc, + }; + } + if (classValue.type.kind !== "classval") + lowerer.badType(expr.right, lowerer.typeOf(expr.right)); + const held = lowerer.declareHiddenLocal("%instanceofClass", classValue.type); + const instance: IrType = { kind: "object", className: caughtTarget.def.name }; + return { + kind: "seqExpr", + stmts: [{ kind: "varDecl", localId: held.id, init: classValue, loc }], + result: { + kind: "logical", + op: "&&", + left: caughtClassInstanceOf(lowerer, caught.id, caughtTarget, loc), + right: { + kind: "instanceOfValue", + value: { + kind: "caughtNarrow", + value: varRef(caught.id, CAUGHT, loc), + type: instance, + loc, + }, + classValue: varRef(held.id, classValue.type, loc), + type: BOOL, + loc, + }, + type: BOOL, + loc, + }, + type: BOOL, + loc, + }; + } + if (rhsClassval?.kind === "classval" && !caught) { const targetInfo = lowerer.classes.get(rhsClassval.className); if (!targetInfo) { // The type world names a class the lowering never registered @@ -608,39 +675,8 @@ export function lowerInstanceOf(lowerer: Lowerer, expr: ts.BinaryExpression, loc // narrowing then types the branches; reads bridge through caughtRead. const caughtLocal = lowerer.caughtLocalOf(expr.left); if (caughtLocal) { - const caughtInstanceOf = (info: ClassInfo): IrExpr => { - if (!lowerer.inHierarchy(info)) { - const value = varRef(caughtLocal.id, CAUGHT, loc); - return { - kind: "ternary", - cond: { - kind: "caughtTest", - value, - test: "instanceof", - className: info.def.name, - type: BOOL, - loc, - }, - then: { kind: "boolLit", value: true, type: BOOL, loc }, - else_: classInstanceOf( - lowerer, - { kind: "caughtToDyn", value, type: DYN, loc }, - info, - loc, - ), - type: BOOL, - loc, - }; - } - return { - kind: "caughtTest", - value: { kind: "varRef", localId: caughtLocal.id, type: CAUGHT, loc }, - test: "instanceof", - className: info.def.name, - type: BOOL, - loc, - }; - }; + const caughtInstanceOf = (info: ClassInfo): IrExpr => + caughtClassInstanceOf(lowerer, caughtLocal.id, info, loc); let test = caughtInstanceOf(target); // A thrown checked-dynamic object whose prototype chain reaches // Error.prototype (an ES5 constructor function inheriting from it). @@ -1035,6 +1071,42 @@ function writableClassValueTest( }; } +/** A catch binding tested against a program class: the runtime test against + * the class's preorder interval (false for non-hierarchy-object payloads). */ +function caughtClassInstanceOf( + lowerer: Lowerer, + caughtId: string, + info: ClassInfo, + loc: SrcLoc, +): IrExpr { + if (!lowerer.inHierarchy(info)) { + const value = varRef(caughtId, CAUGHT, loc); + return { + kind: "ternary", + cond: { + kind: "caughtTest", + value, + test: "instanceof", + className: info.def.name, + type: BOOL, + loc, + }, + then: { kind: "boolLit", value: true, type: BOOL, loc }, + else_: classInstanceOf(lowerer, { kind: "caughtToDyn", value, type: DYN, loc }, info, loc), + type: BOOL, + loc, + }; + } + return { + kind: "caughtTest", + value: { kind: "varRef", localId: caughtId, type: CAUGHT, loc }, + test: "instanceof", + className: info.def.name, + type: BOOL, + loc, + }; +} + /** A statically decided `instanceof` still reads its class after the left * operand: a class with an initialization flag throws there before its * declaration ran. */ diff --git a/packages/compiler/src/frontend/lowering/expressions/union-fields.ts b/packages/compiler/src/frontend/lowering/expressions/union-fields.ts index 6454bc1fd..4f4532a6a 100644 --- a/packages/compiler/src/frontend/lowering/expressions/union-fields.ts +++ b/packages/compiler/src/frontend/lowering/expressions/union-fields.ts @@ -2,6 +2,7 @@ import { nodeThrowExpr, varRef } from "../../../ir/build.js"; import * as ts from "../../ts7/adapter.js"; import type { Lowerer } from "../lowerer.js"; import { narrowStoredClassValue } from "../class-unions.js"; +import { markSloppyRecordWrites } from "../frozen-records.js"; import { BOOL, type IrExpr, @@ -246,6 +247,8 @@ export function lowerUnionFieldWrite( value: slotValue, loc, }; + // A sloppy-mode write to a frozen record leaves it unchanged. + if (write.kind === "recordSet") markSloppyRecordWrites(lowerer, write, access); branch = { kind: "seqExpr", stmts: [write], result, type: result.type, loc }; } else { branch = nodeThrowExpr( diff --git a/packages/compiler/src/frontend/lowering/frozen-records.ts b/packages/compiler/src/frontend/lowering/frozen-records.ts new file mode 100644 index 000000000..30eb9e455 --- /dev/null +++ b/packages/compiler/src/frontend/lowering/frozen-records.ts @@ -0,0 +1,354 @@ +import * as ts from "../ts7/adapter.js"; +import { + BOOL, + VOID, + shapeHasAccessorSlots, + typeKey, + type IrExpr, + type IrFunction, + type IrStmt, + type IrType, + type SrcLoc, +} from "../../ir/ir.js"; +import { numLit, strLit, varRef } from "../../ir/build.js"; +import { everyStmtList, transformStmtList } from "../../ir/traverse.js"; +import type { Lowerer } from "./lowerer.js"; +import { recordHasOwnHelper } from "./lower-calls.js"; +import { isStrictCode } from "./strict-mode.js"; + +/* Object.freeze of records and arrays that other references can still + * write. Only the types a program may freeze pay for it. A record keeps its + * frozen bit in the rc word (scr_rc.h SCR_RC_FROZEN), which its release + * ignores, and every write to it checks the bit first + * (guardFrozenWrites). An array keeps it in its storage flags + * (scr_array.h SCR_ARR_FROZEN), which the backend's array write guards test + * for the array types the module lists (IrModule.freezableArrays). A type is + * freezable when a static Object.freeze names it, or, in a program that + * freezes untyped values, when a value of it or of a type reaching it can be + * held as a live untyped reference. */ + +type RecordType = Extract; +type RecordWrite = Extract; +type FreezeFn = "record.freeze" | "array.freeze" | "collection.preventExtensions"; + +/** Object.freeze of a record or an array, or Object.freeze, seal or + * preventExtensions of a Map or Set: `fn` sets the value's bit, and the + * call answers the value. */ +export function lowerNativeFreeze( + lowerer: Lowerer, + fn: FreezeFn, + target: IrExpr, + loc: SrcLoc, +): IrExpr { + const type = target.type; + const slot = lowerer.declareHiddenLocal("%freezeTarget", type); + const receiver = (): IrExpr => varRef(slot.id, type, loc); + return { + kind: "seqExpr", + stmts: [ + { kind: "varDecl", localId: slot.id, init: target, loc }, + { + kind: "exprStmt", + expr: { kind: "libCall", fn, args: [receiver()], type: VOID, loc }, + loc, + }, + ], + result: receiver(), + type, + loc, + }; +} + +function isRecordWrite(s: IrStmt): s is RecordWrite { + return s.kind === "recordSet" || s.kind === "recordKeySet" || s.kind === "recordKeyDelete"; +} + +/** The key of a record write's source range: strict mode is lexical, so + * every write lowered from the same range shares its mode. */ +function writeKey(loc: SrcLoc): string { + return `${loc.file}\0${loc.start}\0${loc.end}`; +} + +/** Records the record writes of `stmt`, lowered from `node`, as sloppy-mode + * writes when `node` is sloppy-mode code: a write to a frozen record then + * leaves it unchanged instead of throwing. `deletes` says `stmt` is the + * absent-state write of `delete r.f`, which Node refuses only while the + * field is present. */ +export function markSloppyRecordWrites( + lowerer: Lowerer, + stmt: IrStmt, + node: ts.Node, + deletes = false, +): IrStmt { + if (deletes) lowerer.recordFieldDeletes.add(writeKey(stmt.loc)); + if (isStrictCode(node)) return stmt; + everyStmtList([stmt], { + stmt: (s) => { + if (s.kind === "recordSet" || s.kind === "recordKeySet" || s.kind === "recordKeyDelete") + lowerer.sloppyRecordWrites.add(writeKey(s.loc)); + return true; + }, + expr: () => true, + }); + return stmt; +} + +/** Whether a field write's value can be an optional field's absent state, + * which only a field write itself can hold. */ +function holdsAbsentState(e: IrExpr): boolean { + if (e.kind === "fieldAbsent") return true; + return e.kind === "ternary" && (holdsAbsentState(e.then) || holdsAbsentState(e.else_)); +} + +/** The record shapes and array types (typeKeys) whose values the program + * may freeze. */ +function freezableTypes( + lowerer: Lowerer, + functions: IrFunction[], +): { shapes: Set; arrays: Set } { + const shapes = new Set(); + const arrays = new Set(); + const live: IrType[] = []; + let dynFreeze = false; + for (const fn of functions) + everyStmtList(fn.body, { + expr: (e) => { + if (e.kind === "libCall" && e.fn === "record.freeze") { + const target = e.args[0]; + if (target && target.type.kind === "record") shapes.add(target.type.shapeId); + } else if (e.kind === "libCall" && e.fn === "array.freeze") { + const target = e.args[0]; + if (target && target.type.kind === "array") arrays.add(typeKey(target.type)); + } else if (e.kind === "libCall" && e.fn === "dyn.freeze") dynFreeze = true; + else if (e.kind === "dynFrom") live.push(e.value.type); + return true; + }, + stmt: () => true, + }); + if (!dynFreeze) return { shapes, arrays }; + // An untyped reference to a class instance, or to a record or array + // boxed live, reaches the values its fields and elements hold as further + // live references. Class instances also cross into untyped code through + // compiled callbacks, so every class is a root. + const stack = [...live]; + for (const info of lowerer.classes.values()) + for (const field of info.def.fields) stack.push(field.type); + const seen = new Set(); + while (stack.length > 0) { + const type = stack.pop()!; + switch (type.kind) { + case "record": { + if (seen.has(`record:${type.shapeId}`)) break; + seen.add(`record:${type.shapeId}`); + shapes.add(type.shapeId); + const shape = lowerer.shapes.get(type.shapeId); + for (const field of shape?.fields ?? []) stack.push(field.type); + if (shape?.indexValue) stack.push(shape.indexValue); + break; + } + case "union": + if (seen.has(`union:${type.unionId}`)) break; + seen.add(`union:${type.unionId}`); + for (const arm of lowerer.unions.get(type.unionId)?.arms ?? []) stack.push(arm); + break; + case "array": { + const key = typeKey(type); + if (seen.has(key)) break; + seen.add(key); + arrays.add(key); + stack.push(type.elem); + break; + } + case "set": + stack.push(type.elem); + break; + case "map": + stack.push(type.key, type.value); + break; + case "promise": + stack.push(type.inner); + break; + case "generator": + stack.push(type.yieldT, type.retT, type.nextT); + break; + case "func": + for (const param of type.params) stack.push(param); + stack.push(type.ret); + break; + default: + break; + } + } + return { shapes, arrays }; +} + +/** The guarded form of a write to a freezable record: the receiver and the + * operands are evaluated first, in source order, then the frozen bit decides + * between the write and `reject`. */ +function guardedWrite( + receiver: IrExpr, + decls: IrStmt[], + reject: IrStmt[], + store: IrStmt, + loc: SrcLoc, +): IrStmt { + return { + kind: "block", + body: [ + ...decls, + { + kind: "if", + cond: { kind: "libCall", fn: "record.isFrozen", args: [receiver], type: BOOL, loc }, + then: reject, + else_: [store], + loc, + }, + ], + loc, + }; +} + +/** Checks the frozen bit before every write to a record of a freezable + * shape, and lists the freezable array types, whose writes the backend + * guards (IrModule.freezableArrays). A write to a frozen record throws + * Node's TypeError in strict mode code; in sloppy-mode code it leaves the + * record unchanged. Deleting a key the record does not have succeeds either + * way. */ +export function guardFrozenWrites(lowerer: Lowerer, functions: IrFunction[]): void { + const { shapes: freezable, arrays } = freezableTypes(lowerer, functions); + for (const id of freezable) { + const shape = lowerer.shapes.get(id); + if (shape) shape.freezable = true; + } + for (const key of arrays) lowerer.freezableArrays.add(key); + if (freezable.size === 0) return; + const lifted = lowerer.liftedFns.length; + for (const fn of functions) { + const local = (type: IrType, loc: SrcLoc): Extract => { + const id = `%frozenWrite.${fn.locals.length}`; + fn.locals.push({ id, name: id, type, mutable: false }); + return { kind: "varRef", localId: id, type, loc }; + }; + const guarded = new Set(); + fn.body = transformStmtList(fn.body, { + stmt: (statement) => { + if (!isRecordWrite(statement) || guarded.has(statement)) return statement; + if (!freezable.has(statement.shapeId)) return statement; + const sloppy = lowerer.sloppyRecordWrites.has(writeKey(statement.loc)); + const deletes = + statement.kind === "recordSet" && lowerer.recordFieldDeletes.has(writeKey(statement.loc)); + const loc = statement.loc; + const shapeId = statement.shapeId; + const receiver = local(statement.obj.type, loc); + const decls: IrStmt[] = [ + { kind: "varDecl", localId: receiver.localId, init: statement.obj, loc }, + ]; + const owner = strLit( + lowerer.shapes.get(shapeId)?.tuple ? "[object Array]" : "#", + loc, + ); + const thrown = (key: IrExpr, operation: number): IrStmt => ({ + kind: "exprStmt", + expr: { + kind: "libCall", + fn: "record.throwFrozen", + args: [key, owner, numLit(operation, loc)], + type: VOID, + loc, + }, + loc, + }); + if (statement.kind === "recordSet") { + let value = statement.value; + if (!holdsAbsentState(value)) { + const slot = local(value.type, loc); + decls.push({ kind: "varDecl", localId: slot.localId, init: value, loc }); + value = slot; + } + const store: IrStmt = { + kind: "recordSet", + obj: receiver, + shapeId, + field: statement.field, + value, + loc, + }; + guarded.add(store); + const field = strLit(statement.field, loc); + // `delete r.f` of an optional field writes its absent state: Node + // only refuses it when the field is present. + const reject: IrStmt[] = sloppy + ? [] + : deletes + ? [ + { + kind: "if", + cond: lowerer.recordFieldPresent(receiver, shapeId, statement.field, loc), + then: [thrown(field, 2)], + else_: null, + loc, + }, + ] + : [thrown(field, 0)]; + return guardedWrite(receiver, decls, reject, store, loc); + } + const key = local(statement.key.type, loc); + decls.push({ kind: "varDecl", localId: key.localId, init: statement.key, loc }); + const named = key.type.kind === "string"; + const present: IrExpr = named + ? { + kind: "call", + callee: recordHasOwnHelper(lowerer, shapeId, loc), + args: [receiver, key], + type: BOOL, + loc, + } + : { kind: "boolLit", value: true, type: BOOL, loc }; + const text: IrExpr = named ? key : strLit("[key]", loc); + if (statement.kind === "recordKeySet") { + const value = local(statement.value.type, loc); + decls.push({ kind: "varDecl", localId: value.localId, init: statement.value, loc }); + const store: IrStmt = { + kind: "recordKeySet", + obj: receiver, + shapeId, + key, + value, + ...(statement.overflowOnly ? { overflowOnly: true as const } : {}), + loc, + }; + guarded.add(store); + const reject: IrStmt[] = sloppy + ? [] + : [ + { + kind: "if", + cond: present, + then: [thrown(text, 0)], + else_: [thrown(text, 1)], + loc, + }, + ]; + return guardedWrite(receiver, decls, reject, store, loc); + } + const store: IrStmt = { kind: "recordKeyDelete", obj: receiver, shapeId, key, loc }; + guarded.add(store); + const reject: IrStmt[] = sloppy + ? [] + : [{ kind: "if", cond: present, then: [thrown(text, 2)], else_: null, loc }]; + return guardedWrite(receiver, decls, reject, store, loc); + }, + expr: (expression) => expression, + }); + } + // The key-presence helpers the guards call. + for (const helper of lowerer.liftedFns.slice(lifted)) functions.push(helper); +} + +/** Whether a statically typed Object.freeze can freeze a value of `type`: + * a record whose shape has no accessor properties. */ +export function freezableRecordType(lowerer: Lowerer, type: IrType): type is RecordType { + if (type.kind !== "record") return false; + const shape = lowerer.shapes.get(type.shapeId); + return shape !== undefined && !shapeHasAccessorSlots(shape); +} diff --git a/packages/compiler/src/frontend/lowering/iterator-adapters.ts b/packages/compiler/src/frontend/lowering/iterator-adapters.ts index a8b4ea496..9f2b1e31f 100644 --- a/packages/compiler/src/frontend/lowering/iterator-adapters.ts +++ b/packages/compiler/src/frontend/lowering/iterator-adapters.ts @@ -84,6 +84,26 @@ function collectionIteratorSlotValue( collT.kind === "map" ? { kind: "mapIntrinsic", method: m, receiver: coll, args: [index], type: t, loc } : { kind: "setIntrinsic", method: "iterKey", receiver: coll, args: [index], type: t, loc }; + // entries() answers [key, value] pairs: only a slot whose yield type is + // that pair as a record of the two fields takes the typed walk; a tuple + // or other pair type keeps the checked adapter. + if (method === "entries") { + const pair = + expected.yieldT.kind === "record" ? lowerer.shapes.get(expected.yieldT.shapeId) : undefined; + const field = (name: string): IrType | undefined => + pair?.fields.find((f) => f.name === name)?.type; + const first = field("0"); + const second = field("1"); + if ( + pair === undefined || + pair.fields.length !== 2 || + first === undefined || + second === undefined || + !typeEquals(first, keyT) || + !typeEquals(second, valueT) + ) + return null; + } const entry = (): IrExpr => method === "keys" || (collT.kind === "set" && method === "values") ? read("iterKey", keyT) @@ -99,9 +119,13 @@ function collectionIteratorSlotValue( loc, }; if (!typeEquals(entry().type, expected.yieldT)) return null; + // The entry is read once per step and kept: the parking yield of an + // early exit answers it again rather than rereading a slot the loop may + // have deleted. + const current = varRef("current.0", expected.yieldT, loc); const suspend = (): IrStmt => ({ kind: "exprStmt", - expr: { kind: "yieldExpr", value: entry(), type: resumeType(expected.nextT), loc }, + expr: { kind: "yieldExpr", value: current, type: resumeType(expected.nextT), loc }, loc, }); const finish: IrStmt = { @@ -120,6 +144,7 @@ function collectionIteratorSlotValue( { id: "native.0", name: "native", type: DYN, mutable: false }, { id: "index.0", name: "index", type: F64, mutable: true }, { id: "resumed.0", name: "resumed", type: BOOL, mutable: true }, + { id: "current.0", name: "current", type: expected.yieldT, mutable: false }, ], body: [ { @@ -173,6 +198,7 @@ function collectionIteratorSlotValue( loc, }, setResumed(false, loc), + { kind: "varDecl", localId: "current.0", init: entry(), loc }, { kind: "tryCatch", tryBody: [suspend(), setResumed(true, loc)], diff --git a/packages/compiler/src/frontend/lowering/js-class-fields.ts b/packages/compiler/src/frontend/lowering/js-class-fields.ts index 2f726ce97..28a7ceeda 100644 --- a/packages/compiler/src/frontend/lowering/js-class-fields.ts +++ b/packages/compiler/src/frontend/lowering/js-class-fields.ts @@ -2,11 +2,19 @@ import * as ts from "../ts7/adapter.js"; import { isClassOwnEnumerableFieldName } from "../../ir/ir.js"; import type { ClassInfo } from "./lower-classes.js"; -/** Reflection that can remove a property or change its attributes. */ -const PROPERTY_REFLECTION = new Set(["deleteProperty", "freeze", "seal", "preventExtensions"]); +/* Object.freeze, Object.seal and Object.preventExtensions do not appear + * here: they never create or remove a property, and a class instance keeps + * its integrity state outside the field layout. A statically typed call + * sets the instance's CLASS_ATTRIBUTES word (class-attributes.ts), which + * TypeScript classes use with the same fixed layouts; through an untyped + * value the runtime refuses the operation on any class instance + * (scr_dyn_class_integrity_fence). Lifting that fence must revisit this. */ -/** `Object.freeze({ ... })` and the like apply to a new literal, which is - * not a class instance. */ +/** Reflection that can remove any property. */ +const PROPERTY_REFLECTION = new Set(["deleteProperty"]); + +/** `Reflect.deleteProperty({ ... }, key)` applies to a new literal, which + * is not a class instance. */ function freshLiteralArgument(callee: ts.PropertyAccessExpression): boolean { const call = callee.parent; if (!ts.isCallExpression(call) || call.expression !== callee) return false; @@ -141,12 +149,35 @@ export function constructionPrefix( return { created, sealed }; } -/** Whether a class declares a getter or setter, which can intercept a - * base constructor's field assignment. */ -function declaresAccessor(info: ClassInfo): boolean { - return [...info.methods.keys()].some( - (name) => name.startsWith("get:") || name.startsWith("set:"), - ); +/** The names of the getters and setters among a class's own methods. */ +export function accessorNames(methods: Iterable): Set { + const names = new Set(); + for (const name of methods) + if (name.startsWith("get:") || name.startsWith("set:")) names.add(name.slice(4)); + return names; +} + +/** Whether one of `accessors` can intercept an assignment the base + * constructions make: a base constructor's `this.name = value` runs a + * subclass accessor of that name instead of creating the field. Only field + * names can reach the created prefix; any other assignment already ends + * it. */ +export function accessorInterceptsBase(accessors: ReadonlySet, base: ClassInfo): boolean { + for (const name of accessors) if (base.fields.has(name)) return true; + return false; +} + +/** Whether an accessor that instances of `info` inherit or declare shares + * a name with one of their fields. Such a field and accessor disagree on + * what the instance's property is; any other accessor lives on the + * prototype beside the fields and never touches the layout. */ +function accessorShadowsField(info: ClassInfo): boolean { + for (let current: ClassInfo | null = info; current; current = current.base) { + if (current.def.runtime) break; + for (const name of accessorNames(current.methods.keys())) + if (info.fields.has(name)) return true; + } + return false; } /** `constructionPrefix` for a collected class and its bases. */ @@ -156,7 +187,8 @@ export function construction(info: ClassInfo): Construction | null { if (info.base) { base = construction(info.base); if (!base) return null; - if (declaresAccessor(info)) base = { created: base.created, sealed: false }; + if (accessorInterceptsBase(accessorNames(info.methods.keys()), info.base)) + base = { created: base.created, sealed: false }; } return constructionPrefix(info.decl, base, (name) => info.fields.has(name)); } @@ -169,7 +201,7 @@ export function construction(info: ClassInfo): Construction | null { export function fixedFieldFamily(root: ClassInfo, deleted: Set | null): boolean { if (deleted === null) return false; const fixed = (info: ClassInfo): boolean => { - if (declaresAccessor(info)) return false; + if (accessorShadowsField(info)) return false; const built = construction(info); if (!built) return false; for (const field of info.fieldOrder) { diff --git a/packages/compiler/src/frontend/lowering/lower-calls.ts b/packages/compiler/src/frontend/lowering/lower-calls.ts index 82f5a27c1..31276263d 100644 --- a/packages/compiler/src/frontend/lowering/lower-calls.ts +++ b/packages/compiler/src/frontend/lowering/lower-calls.ts @@ -245,6 +245,7 @@ import { import { fenceSymbolFieldCopy } from "./symbol-fields.js"; import { lowerClassDataDescriptor, lowerClassDescriptorRead } from "./class-descriptors.js"; import { lowerClassFieldDefine, lowerClassIntegrity } from "./class-attributes.js"; +import { freezableRecordType, lowerNativeFreeze } from "./frozen-records.js"; import { classStaticDataFor } from "./class-static-data.js"; import { lowerUrlFactory } from "./lower-url.js"; import { nullishValueUnitOf } from "./binding-analysis.js"; @@ -306,6 +307,22 @@ function mayHoldError(lowerer: Lowerer, type: ts.Type, errorType: ts.Type | null ); } +/** The `f` of a `(0, f)` callee (a numeric left operand, so dropping it + * changes only the call's receiver), or null. */ +function receiverlessCalleeOf(callee: ts.Expression): ts.Expression | null { + let e = callee; + while (ts.isParenthesizedExpression(e)) e = e.expression; + if ( + !ts.isBinaryExpression(e) || + e.operatorToken.kind !== ts.SyntaxKind.CommaToken || + !ts.isNumericLiteral(e.left) + ) + return null; + let right = e.right; + while (ts.isParenthesizedExpression(right)) right = right.expression; + return right; +} + export function lowerCall(lowerer: Lowerer, expr: ts.CallExpression): IrExpr { const loc = locOf(expr); if (lowerer.errorAccessorReflection === null) noteErrorAccessorReflection(lowerer, expr); @@ -329,6 +346,17 @@ export function lowerCall(lowerer: Lowerer, expr: ts.CallExpression): IrExpr { const json = lowerJsonRequireCall(lowerer, expr); if (json) return json; + // `(0, ns.member)(...)` over a builtin module (tsc's and esbuild's call + // without a receiver): Node's module functions do not read `this`, so it + // is the member call. + { + const member = receiverlessCalleeOf(expr.expression); + if (member !== null && ts.isPropertyAccessExpression(member)) { + const builtin = lowerer.lowerNamespaceBuiltinCall(expr, member); + if (builtin) return builtin; + } + } + // eval(...) and Function(...): Node's EvalError under // --disallow-code-generation-from-strings (code-generation.ts). { @@ -3728,6 +3756,14 @@ function lowerDynReceiverMethodCall( // 'unknown' slot does. if (checkerUntyped && DYN_HANDLE_KINDS.has(recv.type.kind)) recv = lowerer.coerceToExpected(recv, DYN); + // A typed array whose checker type collapsed to `any` (a JS binding + // referenced in its own initializer's inference — hono's etag digest + // `const nextChunk = new Uint8Array(new ArrayBuffer(f(chunk)))` beside + // `chunk = nextChunk`) still lowered to its native bytes. The typed + // lowerings key on the checker type and decline it, so it boxes by + // reference (the box shares the native storage, so writes through the + // call stay visible) and the call dispatches on the runtime kind. + if (checkerUntyped && recv.type.kind === "bytes") recv = lowerer.coerceToExpected(recv, DYN); // The last-resort retry (boxTypedTemporary — every other lowering // already declined): a checker-`any` receiver whose value is a FRESH // typed result of an earlier step in the chain (`out.split("\n")` @@ -7244,7 +7280,7 @@ export function recordKeysArrayCall( * (a key is own exactly when Object.keys would list it — the two share * the test), everything else true. Unmatched keys probe the overflow * map when present, so explicit undefined values remain own keys. */ -function recordHasOwnHelper(lowerer: Lowerer, shapeId: string, loc: SrcLoc): string { +export function recordHasOwnHelper(lowerer: Lowerer, shapeId: string, loc: SrcLoc): string { const key = `obj.hasOwn:${shapeId}`; const existing = lowerer.arrHofHelpers.get(key); if (existing) return existing; @@ -8459,6 +8495,22 @@ function lowerObjectStaticCall( locOf(call), ); if (fnIntegrity) return fnIntegrity; + // A Map or Set has no own properties: freeze, seal and preventExtensions + // all only make it not extensible. + if (value.type.kind === "map" || value.type.kind === "set") { + if (member === "preventExtensions" || member === "seal") + return lowerNativeFreeze(lowerer, "collection.preventExtensions", value, locOf(call)); + const extensible: IrExpr = { + kind: "libCall", + fn: "collection.isExtensible", + args: [value], + type: BOOL, + loc: locOf(call), + }; + return member === "isExtensible" + ? extensible + : { kind: "unary", op: "!", operand: extensible, type: BOOL, loc: locOf(call) }; + } if (member === "preventExtensions" || member === "seal") { const integrity = lowerClassIntegrity(lowerer, call, member, value); if (integrity) return integrity; @@ -8493,10 +8545,44 @@ function lowerObjectStaticCall( // the argument — no alias exists, so the frozen bit is unobservable // (writes through the Readonly result are compile errors, and no // other reference can write). Primitives pass through per ES2015. - // Aliased objects keep a fence: a later write through the original - // reference would need the runtime frozen bit (strict mode throws). + // Aliased records get the frozen bit of their rc word (frozen-records.ts); + // other aliased objects keep a fence: a later write through the original + // reference would need a runtime frozen bit (strict mode throws). if (member === "isFrozen" && call.arguments.nodes.length === 1) { - const value = lowerer.lowerExprExpecting(call.arguments.nodes[0]!, DYN); + const argNode = call.arguments.nodes[0]!; + const probedKind = tryLowerExpression(lowerer, argNode)?.type.kind; + if ( + probedKind === "record" || + probedKind === "array" || + probedKind === "map" || + probedKind === "set" + ) { + const native = lowerer.lowerExpr(argNode); + const loc = locOf(call); + if (native.type.kind === "record" || native.type.kind === "array") + return { + kind: "libCall", + fn: native.type.kind === "record" ? "record.isFrozen" : "array.isFrozen", + args: [native], + type: BOOL, + loc, + }; + if (native.type.kind === "map" || native.type.kind === "set") + return { + kind: "unary", + op: "!", + operand: { + kind: "libCall", + fn: "collection.isExtensible", + args: [native], + type: BOOL, + loc, + }, + type: BOOL, + loc, + }; + } + const value = lowerer.lowerExprExpecting(argNode, DYN); return { kind: "libCall", fn: "dyn.isFrozen", args: [value], type: BOOL, loc: locOf(call) }; } if (member === "freeze") { @@ -8546,6 +8632,12 @@ function lowerObjectStaticCall( if (fnIntegrity) return fnIntegrity; const integrity = lowerClassIntegrity(lowerer, call, "freeze", value); if (integrity) return integrity; + if (freezableRecordType(lowerer, value.type)) + return lowerNativeFreeze(lowerer, "record.freeze", value, locOf(call)); + if (value.type.kind === "array") + return lowerNativeFreeze(lowerer, "array.freeze", value, locOf(call)); + if (value.type.kind === "map" || value.type.kind === "set") + return lowerNativeFreeze(lowerer, "collection.preventExtensions", value, locOf(call)); lowerer.noLowering( "Object.freeze of a possibly-aliased value", call, @@ -11262,6 +11354,7 @@ const SHIM_INTRINSICS: ReadonlyMap { for (let c = base; c; c = c.base) if (c.builtinStream) return c; @@ -2758,7 +2763,7 @@ export function collectClassShapeInner( lowerer.unsupported( "SC1090", memberName, - `declaring '_writev' on a ${streamBase.def.name.slice(1)} subclass (batched writes are supported on Writable and Duplex subclasses)`, + `declaring '_writev' on a ${streamBase.def.name.slice(1)} subclass (a Readable has no writable side)`, ); } const accepted = streamCtorShape(streamBase.def.name).accepted; @@ -4114,7 +4119,7 @@ export function collectClassShapeInner( // A subclass field created before anything can observe the instance // keeps its type: super() cannot run code that reads it earlier. const baseConstruction = - base && ![...methods.keys()].some((m) => m.startsWith("get:") || m.startsWith("set:")) + base && !accessorInterceptsBase(accessorNames(methods.keys()), base) ? construction(base) : null; const sealedPrefix = baseConstruction diff --git a/packages/compiler/src/frontend/lowering/lower-server.ts b/packages/compiler/src/frontend/lowering/lower-server.ts index d9746e1f6..cb79accea 100644 --- a/packages/compiler/src/frontend/lowering/lower-server.ts +++ b/packages/compiler/src/frontend/lowering/lower-server.ts @@ -7581,7 +7581,7 @@ function lowerHttpReqMethodCall( loc, }; } - if (name === "resume" || name === "destroy") { + if (name === "resume" || name === "pause" || name === "destroy") { requireStatementPosition(lowerer, call, `req.${name}()`); if (args.length !== 0) { lowerer.noLowering( @@ -7591,7 +7591,8 @@ function lowerHttpReqMethodCall( ); } const receiver = coerceToHandle(lowerer, access.expression, HTTPREQ_T); - const fn: IrLibFn = name === "resume" ? "http.reqResume" : "http.reqDestroy"; + const fn: IrLibFn = + name === "resume" ? "http.reqResume" : name === "pause" ? "http.reqPause" : "http.reqDestroy"; return { kind: "libCall", fn, args: [receiver], type: VOID, loc }; } if (name === "setEncoding") { diff --git a/packages/compiler/src/frontend/lowering/lower-stdlib.ts b/packages/compiler/src/frontend/lowering/lower-stdlib.ts index e671a0a08..dc9d3def8 100644 --- a/packages/compiler/src/frontend/lowering/lower-stdlib.ts +++ b/packages/compiler/src/frontend/lowering/lower-stdlib.ts @@ -1945,6 +1945,19 @@ export function lowerStaticReadableStreamNew( }; } +/** `new TransformStream()`: the identity transform, a native handle whose + * readable side is a native ReadableStream and whose writable side is the + * runtime's WritableStream. A transformer or queuing strategies have no + * native lowering yet and keep the SC2020 refusal. */ +export function lowerStaticTransformStreamNew( + lowerer: Lowerer, + expr: ts.NewExpression, +): IrExpr | null { + if (stdlibGlobalNameOf(lowerer, expr.expression) !== "TransformStream") return null; + if ((expr.arguments?.nodes.length ?? 0) > 0) return null; + return { kind: "libCall", fn: "fetch.transformNew", args: [], type: DYN, loc: locOf(expr) }; +} + /** Native Request and Headers values share the fetch runtime's handles. */ export function lowerFetchWebNew(lowerer: Lowerer, expr: ts.NewExpression): IrExpr | null { const name = stdlibGlobalNameOf(lowerer, expr.expression); diff --git a/packages/compiler/src/frontend/lowering/lower-stmts.ts b/packages/compiler/src/frontend/lowering/lower-stmts.ts index 66ecea602..e66d0e855 100644 --- a/packages/compiler/src/frontend/lowering/lower-stmts.ts +++ b/packages/compiler/src/frontend/lowering/lower-stmts.ts @@ -105,6 +105,7 @@ import { requireSpecOf, } from "../program.js"; import { isStrictCode } from "./strict-mode.js"; +import { markSloppyRecordWrites } from "./frozen-records.js"; import { COMPOUND_ASSIGN_OPS, type CompoundOp, @@ -7160,7 +7161,11 @@ function lowerDeleteStatement(lowerer: Lowerer, expr: ts.DeleteExpression): IrSt if (obj.type.kind === "record") { const shape = lowerer.shapes.get(obj.type.shapeId); if (shape?.indexValue && shape.fields.length === 0 && !shape.tuple) { - return { kind: "recordKeyDelete", obj, shapeId: obj.type.shapeId, key: lowerKey(), loc }; + return markSloppyRecordWrites( + lowerer, + { kind: "recordKeyDelete", obj, shapeId: obj.type.shapeId, key: lowerKey(), loc }, + expr, + ); } // `delete r.f` of a declared OPTIONAL field (undefined-armed slot) // writes the field's ABSENT state: a monomorphic shape cannot remove @@ -7181,14 +7186,19 @@ function lowerDeleteStatement(lowerer: Lowerer, expr: ts.DeleteExpression): IrSt if (field) { const absent = lowerer.absentFieldValue(field.type, loc); if (absent) { - return { - kind: "recordSet", - obj, - shapeId: obj.type.shapeId, - field: field.name, - value: absent, - loc, - }; + return markSloppyRecordWrites( + lowerer, + { + kind: "recordSet", + obj, + shapeId: obj.type.shapeId, + field: field.name, + value: absent, + loc, + }, + expr, + true, + ); } } if (shape?.indexValue) { diff --git a/packages/compiler/src/frontend/lowering/lower-stream.ts b/packages/compiler/src/frontend/lowering/lower-stream.ts index c37ae9074..04c2fa043 100644 --- a/packages/compiler/src/frontend/lowering/lower-stream.ts +++ b/packages/compiler/src/frontend/lowering/lower-stream.ts @@ -106,6 +106,61 @@ function pureRead(node: ts.Expression): boolean { return false; } +/** A literal, `undefined`, or a function expression: a value that reads + * no binding or property. */ +function constantOptionValue(node: ts.Expression): boolean { + let e = node; + while (ts.isParenthesizedExpression(e) || ts.isAsExpression(e) || ts.isTypeAssertion(e)) + e = e.expression; + if (ts.isIdentifier(e)) return e.text === "undefined"; + if ( + ts.isPrefixUnaryExpression(e) && + (e.operator === ts.SyntaxKind.MinusToken || + e.operator === ts.SyntaxKind.PlusToken || + e.operator === ts.SyntaxKind.ExclamationToken) + ) + return constantOptionValue(e.operand); + return ( + ts.isStringLiteralLike(e) || + ts.isNumericLiteral(e) || + ts.isBigIntLiteral(e) || + e.kind === ts.SyntaxKind.TrueKeyword || + e.kind === ts.SyntaxKind.FalseKeyword || + e.kind === ts.SyntaxKind.NullKeyword || + ts.isArrowFunction(e) || + ts.isFunctionExpression(e) + ); +} + +/** Whether a runtime decodeStrings or defaultEncoding value (`prop`'s + * `value`) may be read before the other option members. Those values are + * read into locals before the constructor evaluates the other members, + * while Node evaluates the members in order, so every earlier member must + * evaluate without effects, and when the value itself has effects, every + * earlier member must also read nothing those effects could change. */ +function earlierOptionsPure( + options: ts.ObjectLiteralExpression, + prop: ts.Node, + value: ts.Expression, +): boolean { + const valueHasEffects = !sideEffectFreeOptionValue(value) && !pureRead(value); + for (const member of options.properties.nodes) { + if (member === prop) return true; + if (ts.isPropertyAssignment(member)) { + const init = member.initializer; + if ( + valueHasEffects + ? !constantOptionValue(init) + : !sideEffectFreeOptionValue(init) && !pureRead(init) + ) + return false; + } else if (ts.isShorthandPropertyAssignment(member)) { + if (valueHasEffects) return false; + } else if (ts.isSpreadAssignment(member)) return false; + } + return true; +} + /** The stream sides of a receiver class: the nearest stream-class * ancestor's, or null off the stream hierarchy. */ export function streamSidesOf( @@ -179,15 +234,15 @@ export const UNDERSCORE_METHODS: ReadonlyMap = new Map([ /** The option callbacks applied after construction rather than through * the constructor's callback slots (option name → method name): construct - * on every stream class, and writev on the writable classes whose - * machinery hands it batches (Writable and Duplex). */ + * on every stream class, and writev on every class with a writable side + * (a Transform's batches reach writev, not the transform, as in Node). */ export const STREAM_EXTRA_CALLBACKS: ReadonlyMap = new Map([ ["construct", "_construct"], ["writev", "_writev"], ]); export function streamExtraCallbacks(cls: string): readonly string[] { - return cls === "%Writable" || cls === "%Duplex" ? ["construct", "writev"] : ["construct"]; + return cls === "%Readable" ? ["construct"] : ["construct", "writev"]; } /** The completion callback's parameter position in each underscore @@ -442,6 +497,13 @@ interface StreamOptions { * right after construction (Node's constructors call addAbortSignal * once the state exists), or null when absent. */ signal: IrExpr | null; + /** Non-literal decodeStrings and defaultEncoding values: valueDecls + * evaluates them into locals before construction (the defaultEncoding + * validated there, as the state constructors throw before the stream + * exists), and stream.optionValues applies them to the fresh stream. */ + valueDecls: IrStmt[]; + decodeStringsValue: IrExpr | null; + defaultEncodingValue: IrExpr | null; /** The objectMode option's sides (a Duplex's readableObjectMode and * writableObjectMode set one side): applied with stream.objectMode * right after construction. */ @@ -732,6 +794,12 @@ function pinAnyDataCompletion( if (!symbol || lowerer.stageSourceTypes.has(symbol)) return null; const declared = lowerer.paramShape(param).type; if (declared.kind !== "func" || declared.rest || declared.params[1]?.kind !== "dyn") return null; + // Only when every use calls the callback with a Buffer, a string or + // nothing as `data`: another value (or a callback that escapes) keeps + // `data` checked-dynamic, so the runtime pushes it as push(data) does + // and reports ERR_INVALID_ARG_TYPE through 'error', as Node does. + if (fnNode.body === undefined || !completionDataIsChunk(lowerer, fnNode.body, symbol, param.name)) + return null; const data = unionOf(lowerer, [BYTES, STRING, { kind: "undefinedT" }]); lowerer.stageSourceTypes.set( symbol, @@ -740,6 +808,59 @@ function pinAnyDataCompletion( return symbol; } +/** Whether every reference to the completion callback `symbol` in `body` + * calls it with a `data` argument that is a Buffer, a string, null or + * undefined (or with no data). */ +function completionDataIsChunk( + lowerer: Lowerer, + body: ts.Node, + symbol: ts.Symbol, + declaration: ts.Node, +): boolean { + let ok = true; + const visit = (node: ts.Node): void => { + if (!ok) return; + if ( + ts.isIdentifier(node) && + node !== declaration && + lowerer.checker.getSymbolAtLocation(node) === symbol + ) { + const call = node.parent; + if (!call || !ts.isCallExpression(call) || call.expression !== node) { + ok = false; + return; + } + const data = call.arguments.nodes[1]; + if (data !== undefined) { + let t: IrType | null; + try { + t = lowerer.mapTypeOf(lowerer.typeOf(data)); + } catch { + t = null; + } + const arms = + t?.kind === "union" ? (lowerer.unions.get(t.unionId)?.arms ?? null) : t ? [t] : null; + if ( + arms === null || + !arms.every( + (a) => + (a.kind === "bytes" && a.elem === "u8") || + a.kind === "string" || + a.kind === "undefinedT" || + a.kind === "nullT", + ) + ) { + ok = false; + return; + } + } + } + ts.forEachChild(node, visit); + }; + visit(body); + return ok; +} + /** Parses the options object literal of a stream constructor. `which` * names the callbacks this class accepts, in canonical flag order. */ function parseStreamOptions( @@ -767,6 +888,9 @@ function parseStreamOptions( badEncoding: null, pushEncoding: null, signal: null, + valueDecls: [], + decodeStringsValue: null, + defaultEncodingValue: null, readableObject: false, writableObject: false, decodeStringsOff: false, @@ -802,6 +926,21 @@ function parseStreamOptions( optsLit !== undefined && ts.isObjectLiteralExpression(optsLit) && literalWritesValues(optsLit, ctorName !== "Readable" && ctorName !== "Writable"); + // In JavaScript a decodeStrings value known only at run time may let + // string writes through, so the write and transform callbacks' chunk + // parameters read as any (the declarations type it Buffer). + const runtimeDecodeStrings = + optsLit !== undefined && + isJsSourceFile(ts.getSourceFileOfNode(optsLit)) && + ts.isObjectLiteralExpression(optsLit) && + (optsLit.properties?.nodes ?? []).some( + (p) => + ts.isPropertyAssignment(p) && + (ts.isIdentifier(p.name) || ts.isStringLiteral(p.name)) && + p.name.text === "decodeStrings" && + p.initializer.kind !== ts.SyntaxKind.TrueKeyword && + p.initializer.kind !== ts.SyntaxKind.FalseKeyword, + ); if (out.readableObject) out.hwmR = numLit(16, loc); if (out.writableObject) out.hwmW = numLit(16, loc); const tuples = cbTuples(lowerer); @@ -851,6 +990,7 @@ function parseStreamOptions( } const full = tuples[name]!.map((t) => (t === "fn" ? null : t)); const pinned = out.readableObject ? null : pinAnyDataCompletion(lowerer, name, fnNode); + let anyChunk: ts.Symbol | undefined; try { // The completion-callback tail: typed by the checker contextually; // accept the user's declared func type as the tuple's entry so the @@ -867,12 +1007,22 @@ function parseStreamOptions( } return declared ?? funcOf([], VOID); }); + const chunkParam = + runtimeDecodeStrings && (name === "write" || name === "transform") + ? fnNode.parameters.nodes[0] + : undefined; + anyChunk = + chunkParam !== undefined && ts.isIdentifier(chunkParam.name) && !chunkParam.type + ? lowerer.checker.getSymbolAtLocation(chunkParam.name) + : undefined; + if (anyChunk) lowerer.anySymbols.add(anyChunk); present.set( name, lowerStreamCallback(lowerer, name, fnNode, thisType, tuple, writesValues), ); } finally { if (pinned) lowerer.stageSourceTypes.delete(pinned); + if (anyChunk) lowerer.anySymbols.delete(anyChunk); } continue; } @@ -925,11 +1075,25 @@ function parseStreamOptions( if (writableClass) out.decodeStringsOff = true; break; } - lowerer.noLowering( - `${ctorName} with a runtime 'decodeStrings' value`, - prop, - "write 'decodeStrings: true' or 'decodeStrings: false'", - ); + // a runtime value: WritableState's `options.decodeStrings !== false` + if ( + optsLit !== undefined && + ts.isObjectLiteralExpression(optsLit) && + !earlierOptionsPure(optsLit, prop, value) + ) + lowerer.noLowering( + `the ${ctorName} option 'decodeStrings' with a runtime value that an earlier option value could observe or change`, + prop, + "compute the option values into consts before the options object (the runtime value is read before the other options run)", + ); + const local = lowerer.declareHiddenLocal("%decodeStrings", DYN); + out.valueDecls.push({ + kind: "varDecl", + localId: local.id, + init: lowerer.lowerExprExpecting(value, DYN), + loc, + }); + if (writableClass) out.decodeStringsValue = varRef(local.id, DYN, loc); break; } case "encoding": { @@ -973,11 +1137,43 @@ function parseStreamOptions( break; } } - lowerer.noLowering( - `the ${ctorName} option 'defaultEncoding' with a runtime value`, - prop, - "write the encoding as a string literal", - ); + { + // a runtime value: null, undefined and utf8 are the default, + // another name Buffer.isEncoding knows is kept as given, and any + // other value throws ERR_UNKNOWN_ENCODING before construction + if ( + optsLit !== undefined && + ts.isObjectLiteralExpression(optsLit) && + !earlierOptionsPure(optsLit, prop, value) + ) + lowerer.noLowering( + `the ${ctorName} option 'defaultEncoding' with a runtime value that an earlier option value could observe or change`, + prop, + "compute the option values into consts before the options object (the runtime value is read before the other options run)", + ); + const local = lowerer.declareHiddenLocal("%defaultEncoding", DYN); + const ref = varRef(local.id, DYN, loc); + out.valueDecls.push( + { + kind: "varDecl", + localId: local.id, + init: lowerer.lowerExprExpecting(value, DYN), + loc, + }, + { + kind: "exprStmt", + expr: { + kind: "libCall", + fn: "stream.checkEncodingOption", + args: [ref], + type: VOID, + loc, + }, + loc, + }, + ); + out.defaultEncodingValue = ref; + } break; case "signal": { // `if (options.signal) addAbortSignal(options.signal, this)`: @@ -1041,9 +1237,7 @@ function parseStreamOptions( lowerer.noLowering( `the ${ctorName} option 'writev'`, prop, - ctorName === "Readable" - ? "a Readable has no writable side" - : "batched writes on a Transform would bypass its transform — writev is supported on Writable and Duplex", + "a Readable has no writable side", ); break; default: @@ -1083,6 +1277,9 @@ function streamCtorArgs( badEncoding: string | null; pushEncoding: string | null; signal: IrExpr | null; + valueDecls: IrStmt[]; + decodeStringsValue: IrExpr | null; + defaultEncodingValue: IrExpr | null; readableObject: boolean; writableObject: boolean; decodeStringsOff: boolean; @@ -1117,6 +1314,9 @@ function streamCtorArgs( badEncoding: o.badEncoding, pushEncoding: o.pushEncoding, signal: o.signal, + valueDecls: o.valueDecls, + decodeStringsValue: o.decodeStringsValue, + defaultEncodingValue: o.defaultEncodingValue, readableObject: o.readableObject, writableObject: o.writableObject, decodeStringsOff: o.decodeStringsOff, @@ -1237,6 +1437,18 @@ export function lowerStreamNew(lowerer: Lowerer, expr: ts.NewExpression, info: C loc, }; } + const values = optionValuesArgs(parsed, loc); + if (values) { + // runtime decodeStrings and defaultEncoding values (answers the + // receiver +1) + built = { + kind: "libCall", + fn: "stream.optionValues", + args: [built, ...values], + type: thisType, + loc, + }; + } if (parsed.pushEncoding !== null) { // { defaultEncoding }: chain the push-side decode default (the // setEncoding chaining shape — answers the receiver +1). @@ -1269,8 +1481,11 @@ export function lowerStreamNew(lowerer: Lowerer, expr: ts.NewExpression, info: C loc, }; } - if (!parsed.decodeStringsOff && parsed.writeEncoding === null && parsed.extras.length === 0) - return built; + if (!parsed.decodeStringsOff && parsed.writeEncoding === null && parsed.extras.length === 0) { + return parsed.valueDecls.length > 0 + ? { kind: "seqExpr", stmts: parsed.valueDecls, result: built, type: thisType, loc } + : built; + } const fresh = lowerer.declareHiddenLocal("%stream", thisType); const followUps = streamOptionFollowUps( parsed, @@ -1280,13 +1495,30 @@ export function lowerStreamNew(lowerer: Lowerer, expr: ts.NewExpression, info: C ); return { kind: "seqExpr", - stmts: [{ kind: "varDecl", localId: fresh.id, init: built, loc }, ...followUps], + stmts: [ + ...parsed.valueDecls, + { kind: "varDecl", localId: fresh.id, init: built, loc }, + ...followUps, + ], result: varRef(fresh.id, thisType, loc), type: thisType, loc, }; } +/** stream.optionValues' (decodeStrings, defaultEncoding) arguments, or + * null when both options are literals or absent. */ +function optionValuesArgs( + parsed: { decodeStringsValue: IrExpr | null; defaultEncodingValue: IrExpr | null }, + loc: SrcLoc, +): IrExpr[] | null { + if (parsed.decodeStringsValue === null && parsed.defaultEncodingValue === null) return null; + return [ + parsed.decodeStringsValue ?? dynUndefinedExpr(loc), + parsed.defaultEncodingValue ?? dynUndefinedExpr(loc), + ]; +} + /** True when a value of Node's argument type `passed` widens implicitly * into the declared parameter type `declared` (one of its union arms, or a * sub-union of it). */ @@ -1589,6 +1821,7 @@ export function lowerStreamSuperCall( ]; } const out: IrStmt[] = [ + ...parsed.valueDecls, { kind: "exprStmt", expr: { @@ -1614,6 +1847,20 @@ export function lowerStreamSuperCall( loc, }); } + const values = optionValuesArgs(parsed, loc); + if (values) { + out.push({ + kind: "exprStmt", + expr: { + kind: "libCall", + fn: "stream.optionValues", + args: [thisRef(), ...values], + type: thisType, + loc, + }, + loc, + }); + } if (parsed.pushEncoding !== null) { out.push({ kind: "exprStmt", @@ -1697,16 +1944,15 @@ export function lowerStreamUnderscoreAssign( const loc = locOf(expr); const cls = base.def.name; const clsName = cls.slice(1); - if (methodName === "_construct") { - // Node's constructor reads _construct once; a later assignment has no - // effect there - lowerer.noLowering( - `assigning '${methodName}' on a ${clsName}`, - expr.left, - "declare _construct on the class or pass the construct option", - ); - } - const which = methodName === "_writev" ? "writev" : optionByMethod.get(methodName)!; + // `_construct`: Node's constructNT reads it on the construction tick, so + // an assignment before that tick replaces a scheduled hook + // (stream.replaceConstruct) and has no effect after it or without one. + const which = + methodName === "_writev" + ? "writev" + : methodName === "_construct" + ? "construct" + : optionByMethod.get(methodName)!; const { accepted } = streamCtorShape(cls); if (!accepted.includes(which) && !streamExtraCallbacks(cls).includes(which)) { lowerer.noLowering( @@ -1745,6 +1991,7 @@ export function lowerStreamUnderscoreAssign( transform: "stream.setTransform", flush: "stream.setFlush", writev: "stream.setWritev", + construct: "stream.replaceConstruct", }; return { kind: "exprStmt", @@ -2699,6 +2946,22 @@ class StreamArgTypeThrow extends Error { } } +/** Whether a checked-dynamic pipeline callback can take undefined as its + * error: a callback that is already checked-dynamic, one without + * parameters, and one whose first parameter (or rest element) type holds + * undefined. */ +function callbackErrorAdmitsUndefined(lowerer: Lowerer, cb: IrExpr): boolean { + if (cb.kind !== "dynFrom" || cb.value.type.kind !== "func") return true; + const fnT = cb.value.type; + const first = fnT.params[0]; + if (first === undefined) return true; + const p0 = + fnT.rest === true && fnT.params.length === 1 && first.kind === "array" ? first.elem : first; + if (p0.kind === "dyn" || p0.kind === "jsval" || p0.kind === "undefinedT") return true; + if (p0.kind !== "union") return false; + return lowerer.unions.get(p0.unionId)?.arms.some((a) => a.kind === "undefinedT") ?? false; +} + /** The finished/pipeline completion callback: an inline function lowers * through the option-callback machinery (leading `this`, an `Error | * null` prefix — Node binds the stream and passes the error); any other @@ -3160,12 +3423,16 @@ export function lowerStreamModuleCall( // an async generator) reaches the callback through the // checked-dynamic path. const cb = lowerer.lowerExprExpecting(args[args.length - 1]!, DYN); - // SCR_PIPE_TYPED_CB: typed code's callback takes null for no error + // SCR_PIPE_TYPED_CB: a typed callback whose error parameter cannot + // hold undefined takes null for no error; others take undefined. const stagedArgs = [...staged.args]; const flagsArg = stagedArgs[3]!; if (flagsArg.kind !== "numLit") throw new InternalCompilerError("pipeline flags not a literal"); - if (!isJsSourceFile(ts.getSourceFileOfNode(call))) + if ( + !isJsSourceFile(ts.getSourceFileOfNode(call)) && + !callbackErrorAdmitsUndefined(lowerer, cb) + ) stagedArgs[3] = numLit(flagsArg.value | 128, loc); return staged.wrap({ kind: "libCall", diff --git a/packages/compiler/src/frontend/lowering/lowerer.ts b/packages/compiler/src/frontend/lowering/lowerer.ts index 0922f8928..f3360d561 100644 --- a/packages/compiler/src/frontend/lowering/lowerer.ts +++ b/packages/compiler/src/frontend/lowering/lowerer.ts @@ -150,6 +150,7 @@ import { defaultAfterUndefined, lowerStaticallyUndefinedArgument } from "./optio import { type ProtocolDispatch } from "./class-protocol-dispatch.js"; import { ClassDynamicDispatch, trackClassFieldCreation } from "./class-dynamic-dispatch.js"; import { guardClassAttributeWrites } from "./class-attributes.js"; +import { guardFrozenWrites, markSloppyRecordWrites } from "./frozen-records.js"; import { applyHandleProperties } from "./handle-properties.js"; import { ClassProtocols } from "./class-protocols.js"; import { earlyReadVarSymbols } from "./var-hoisting.js"; @@ -496,7 +497,7 @@ import { lowerBuiltinExtraProperty } from "./builtins/properties.js"; import { registerPromisifiedBuiltinDecl } from "./builtins/callable-aliases.js"; import { lowerExecFileAsyncCall } from "./builtins/child-process-async.js"; import { lowerStringDecoderMethodCall, lowerTextCodecCall } from "./builtins/text-codecs.js"; -import { lowerReadlineMethodCall } from "./builtins/readline.js"; +import { isOptionalReadlineCursorRead, lowerReadlineMethodCall } from "./builtins/readline.js"; import { lowerDiagnosticsChannelMethodCall, lowerDiagnosticsChannelProperty, @@ -558,6 +559,7 @@ import { lowerStaticReadableStreamCancelCall, lowerStaticReadableStreamControllerCall, lowerStaticReadableStreamNew, + lowerStaticTransformStreamNew, lowerStaticReadableStreamReaderCall, lowerStaticResponseCall, lowerStdlibMethodCall, @@ -3258,6 +3260,12 @@ export class Lowerer { // Stack of function contexts (bottom = the function being declared at // top level, top = the innermost nested function currently lowering). fnStack: FnCtx[] = []; + /** Source ranges of record writes in sloppy-mode code, and of the + * absent-state writes of `delete r.f` (markSloppyRecordWrites). */ + readonly sloppyRecordWrites = new Set(); + readonly recordFieldDeletes = new Set(); + /** typeKeys of the array types the program may freeze (IrModule.freezableArrays). */ + readonly freezableArrays = new Set(); readonly liftedFns: IrFunction[] = []; /** The first lifted function of each name (liftedFnNamed), for the * liftedFns indexed so far. liftedFns only grows. */ @@ -5572,6 +5580,7 @@ export class Lowerer { if (isIndexedRead(e)) return true; if (ts.isPropertyAccessExpression(e) && isRecordIndexPropertyRead(e)) return true; if (ts.isPropertyAccessExpression(e) && isOptionalProcessStreamProperty(this, e)) return true; + if (ts.isPropertyAccessExpression(e) && isOptionalReadlineCursorRead(this, e)) return true; if (isOptionalProcessVersionsRead(this, e)) return true; if (ts.isPropertyAccessExpression(e) && isSettledIteratorValueRead(this, e)) return true; if (ts.isPropertyAccessExpression(e) && isOsConstantsSymbol(symbolOf(e.expression), this)) @@ -7714,6 +7723,7 @@ export class Lowerer { finalizePrivateBrands(this, functions); trackClassFieldCreation(this, functions); guardClassAttributeWrites(this, functions); + guardFrozenWrites(this, functions); applyHandleProperties(this, functions); // Globals typed by a class that never REGISTERED (a JS class whose @@ -7822,6 +7832,9 @@ export class Lowerer { ...(this.threadSlotKeys.size > 0 ? { threadSlotKeys: [...this.threadSlotKeys].sort() } : {}), + ...(this.freezableArrays.size > 0 + ? { freezableArrays: [...this.freezableArrays].sort() } + : {}), ...(isNodeEsmFile(this.entry, this.program) ? {} : { commonJsEntry: true }), ...(this.ffiImports.length > 0 ? { ffiImports: [...this.ffiImports] } : {}), }; @@ -13003,7 +13016,7 @@ export class Lowerer { } lowerElementWrite(expr: ts.BinaryExpression): IrStmt { - return lowerElementWrite(this, expr); + return markSloppyRecordWrites(this, lowerElementWrite(this, expr), expr); } lowerLiveWebValue(node: ts.Expression): IrExpr { @@ -13400,7 +13413,7 @@ export class Lowerer { } fieldSetStmt(target: FieldTarget, value: IrExpr, loc: SrcLoc, blame: ts.Node): IrStmt { - return fieldSetStmt(this, target, value, loc, blame); + return markSloppyRecordWrites(this, fieldSetStmt(this, target, value, loc, blame), blame); } lowerFieldCompound( @@ -13440,6 +13453,7 @@ export class Lowerer { lowerBlobNew(this, expr) ?? lowerNetAddressNew(this, expr) ?? lowerStaticReadableStreamNew(this, expr) ?? + lowerStaticTransformStreamNew(this, expr) ?? lowerConsoleNew(this, expr); return ( builtin ?? this.withRuntimeOptionalClassValue(expr.expression, () => lowerNew(this, expr)) diff --git a/packages/compiler/src/frontend/lowering/surfaces.ts b/packages/compiler/src/frontend/lowering/surfaces.ts index 8b826304b..44564f2f1 100644 --- a/packages/compiler/src/frontend/lowering/surfaces.ts +++ b/packages/compiler/src/frontend/lowering/surfaces.ts @@ -1593,6 +1593,9 @@ export const BUILTIN_MODULE_FNS: Record< // Prime checks and generation (lowerPrimeCall): checked-dynamic // arguments, results converted to the call site's type. checkPrimeSync: { fn: "crypto.checkPrimeSync", params: [], result: DYN }, + // Checked-dynamic arguments, the result converted to the call site's + // type (lowerPrimeCall). + getCipherInfo: { fn: "crypto.getCipherInfo", params: [], result: DYN }, checkPrime: { fn: "crypto.checkPrimeCb", params: [], result: VOID }, generatePrimeSync: { fn: "crypto.generatePrimeSync", params: [], result: DYN }, generatePrime: { fn: "crypto.generatePrimeCb", params: [], result: VOID }, @@ -3101,14 +3104,6 @@ export const BUILTIN_MODULE_FENCE_HINTS: Record< diffieHellman: "crypto.diffieHellman takes DH, EC or X25519 KeyObjects and has no lowering; " + "the objects of createDiffieHellman, getDiffieHellman and createECDH compute shared secrets", - ...Object.fromEntries( - ["getCipherInfo"].map((m) => [ - m, - "cipher metadata has no lowering — createCipheriv and createDecipheriv support " + - "aes-128/192/256 in cbc, ctr, gcm, ecb, cfb, cfb8, cfb1, and ofb modes, des-ede3 and " + - "des-ede, chacha20, and chacha20-poly1305", - ]), - ), setFips: "a compiled binary has no FIPS provider to enable, and Node itself throws on setFips(true) " + "in a non-FIPS build — getFips() answers 0 here", diff --git a/packages/compiler/src/frontend/lowering/var-hoisting.ts b/packages/compiler/src/frontend/lowering/var-hoisting.ts index 3f950f3ca..0b415bb3d 100644 --- a/packages/compiler/src/frontend/lowering/var-hoisting.ts +++ b/packages/compiler/src/frontend/lowering/var-hoisting.ts @@ -6,9 +6,11 @@ * * A binding settles at the first statement of its hoisting scope (the * function body or the source file) that stores it unconditionally: a - * direct `var x = e`, `x = e;`, or a loop declaring it (`for (var i = 0; - * ...)`, `for (var k in o)`), whose own condition, update and body run - * after the store. A read settles when it runs after that statement: + * direct `var x = e`, `x = e;`, or a `for` loop declaring it (`for (var + * i = 0; ...)`), whose own condition, update and body run after the store. + * A `for (var k in o)` or `for (var v of xs)` loop stores the binding only + * when it iterates, so it settles the reads in its body and no others. A + * read settles when it runs after the settling statement: * - a read in the scope's own code, in a TypeScript file: tsc's * definite-assignment analysis rejects any earlier one; * - a read in the scope's own code, in a JavaScript file: its statement @@ -194,6 +196,18 @@ function referencesBySymbol( return out; } +/** Where a binding is first stored unconditionally: the scope statement's + * index, and for a declarator of a `var` statement or a `for` initializer, + * the list and the declarator's end. */ +interface Settle { + index: number; + list: ts.VariableDeclarationList | null; + end: number; + /** The first for-in/of loop declaring the binding before `index`, whose + * body runs after its store, or -1. */ + loop: number; +} + class ScopeAnalysis { private readonly js: boolean; /** Whether a function declaration's references all settle, by scope @@ -214,10 +228,13 @@ class ScopeAnalysis { } /** The index of the scope statement that first stores `symbol` - * unconditionally, or Infinity. */ - settleIndex(scope: Scope, symbol: ts.Symbol): number { + * unconditionally (Infinity when none does), and, when that statement + * declares it in a `var` statement or a `for` initializer, the end of the + * storing declarator: later declarators of the list run after it. */ + settleIndex(scope: Scope, symbol: ts.Symbol): Settle { const statements = statementsOf(scope); const checker = this.lowerer.checker; + let loopIndex = -1; for (let i = 0; i < statements.length; i++) { const s = statements[i]!; const list = ts.isVariableStatement(s) @@ -233,8 +250,10 @@ class ScopeAnalysis { if ( (d.initializer !== undefined || loop) && boundIdentifiersOf(d.name).some((n) => checker.getSymbolAtLocation(n) === symbol) - ) - return i; + ) { + if (!loop) return { index: i, list, end: d.end, loop: loopIndex }; + if (loopIndex < 0) loopIndex = i; + } } } else if ( ts.isExpressionStatement(s) && @@ -243,20 +262,21 @@ class ScopeAnalysis { ts.isIdentifier(s.expression.left) && checker.getSymbolAtLocation(s.expression.left) === symbol ) { - return i; + return { index: i, list: null, end: -1, loop: loopIndex }; } } - return Infinity; + return { index: Infinity, list: null, end: -1, loop: loopIndex }; } /** Whether `ref` (inside `scope`, or in another module) runs only after - * the statement at index `settle`. `read`: `ref` reads the binding - * itself, so tsc vouches for one in the scope's own code. */ - settles(scope: Scope, settle: number, ref: ts.Identifier, read = true): boolean { + * the store `settle` describes. `read`: `ref` reads the binding itself, + * so tsc vouches for one in the scope's own code. */ + settles(scope: Scope, settle: Settle, ref: ts.Identifier, read = true): boolean { if (ts.getSourceFileOfNode(ref) !== this.sf) return true; const statements = statementsOf(scope); let deferred: ts.Node | null = null; let top: ts.Node | null = null; + const container: ts.Node | undefined = ts.isSourceFile(scope) ? scope : scope.body; for (let n: ts.Node = ref; n.parent !== undefined && n !== scope; n = n.parent) { if ( ts.isFunctionLike(n) || @@ -265,18 +285,26 @@ class ScopeAnalysis { ts.isClassStaticBlockDeclaration(n) ) deferred = n; - if (n.parent === scope || (!ts.isSourceFile(scope) && n.parent === scope.body)) top = n; + // The scope's own statement, not the function body that holds it. + if (n.parent === container) top = n; } if (top === null) return false; if (deferred === null && read && !this.js) return true; if (deferred !== null && ts.isFunctionDeclaration(deferred) && deferred.name !== undefined) return this.functionSettles(scope, settle, deferred); const index = statements.indexOf(top as ts.Statement); - return index > settle || (index === settle && settlesWithin(top, ref)); + return ( + index > settle.index || + (index === settle.index && + (settlesWithin(top, ref) || + // A later declarator of the storing list (`var a = {}, b = { ...a }`). + (settle.list !== null && isInside(ref, settle.list) && ref.pos >= settle.end))) || + (index === settle.loop && settlesWithin(top, ref)) + ); } - private functionSettles(scope: Scope, settle: number, fn: ts.FunctionDeclaration): boolean { - const key = `${fn.pos}:${settle}`; + private functionSettles(scope: Scope, settle: Settle, fn: ts.FunctionDeclaration): boolean { + const key = `${fn.pos}:${settle.index}:${settle.end}:${settle.loop}`; const known = this.functionMemo.get(key); if (known !== undefined) return known; this.functionMemo.set(key, true); diff --git a/packages/compiler/src/frontend/npm-static-bundled-chunks.test.ts b/packages/compiler/src/frontend/npm-static-bundled-chunks.test.ts index 0779db3ce..dc238c622 100644 --- a/packages/compiler/src/frontend/npm-static-bundled-chunks.test.ts +++ b/packages/compiler/src/frontend/npm-static-bundled-chunks.test.ts @@ -276,13 +276,14 @@ export default require_f(); ).toBeNull(); }); -test("an identifier-minified bundle names its lost helper names", () => { - // esbuild --minify output: the helper shapes survive, their names do not. - const result = split( +test("an identifier-minified bundle splits through its helper's shape", () => { + // esbuild --minify output: the helper shape survives, its name does not. + const answer = expectSameBehavior( `var c=(r,e)=>()=>(e||r((e={exports:{}}).exports,e),e.exports);` + - `var x=c((t,m)=>{m.exports={v:1}});console.log(x().v);`, + `var x=c((t,m)=>{m.exports={v:1}});var y=c(t=>{t.w=x().v+1});` + + `console.log(x().v,y().w,x()===x());`, ); - expect(result).toEqual({ degrade: expect.stringContaining("identifier-minified") as string }); + expect(answer.chunks.map((c) => basename(c.path))).toEqual(["index.js.x.cjs", "index.js.y.cjs"]); }); test("an ES-module bundle of CommonJS chunks names the chunk interop", () => { diff --git a/packages/compiler/src/frontend/npm-static-bundled-chunks.ts b/packages/compiler/src/frontend/npm-static-bundled-chunks.ts index 37a74f2be..66c25403b 100644 --- a/packages/compiler/src/frontend/npm-static-bundled-chunks.ts +++ b/packages/compiler/src/frontend/npm-static-bundled-chunks.ts @@ -525,7 +525,7 @@ export function splitBundledChunks( } } } - const helperNames = [...top.keys()].filter((n) => helperBase(n) === "__commonJS"); + let helperNames = [...top.keys()].filter((n) => helperBase(n) === "__commonJS"); // Every name a recognizer relies on is written nowhere and declared // once at the top level; `Object` itself is never rebound. @@ -552,18 +552,14 @@ export function splitBundledChunks( ); }; if (helperNames.length === 0) { - // The helper's shape under another name: an identifier-minified bundle, - // whose other interop helpers lost their names as well. - for (const [name, t] of top) { + // An identifier-minified bundle: the helper's shape under another name. + helperNames = [...top].flatMap(([name, t]) => { const init = ts.isVariableDeclaration(t.node) ? t.node.initializer : undefined; - if (init !== undefined && stable(name) && helperForm(init, sf, ownNames) !== null) - return { - degrade: - "its shipped JS is an identifier-minified bundle whose bundler-interop helpers " + - "(__commonJS and the rest) have lost the names the recognizers verify", - }; - } - return null; + return init !== undefined && stable(name) && helperForm(init, sf, ownNames) !== null + ? [name] + : []; + }); + if (helperNames.length === 0) return null; } const forms = new Map(); for (const name of helperNames) { diff --git a/packages/compiler/src/frontend/npm-static-rewrite-syntax.ts b/packages/compiler/src/frontend/npm-static-rewrite-syntax.ts index c48420625..917041603 100644 --- a/packages/compiler/src/frontend/npm-static-rewrite-syntax.ts +++ b/packages/compiler/src/frontend/npm-static-rewrite-syntax.ts @@ -260,11 +260,21 @@ function definePropertyExportOf( value = getterReturnOf(p); } } - const esModuleStamp = - name === "__esModule" && value !== null && value.kind === ts.SyntaxKind.TrueKeyword; + const esModuleStamp = name === "__esModule" && value !== null && isTrueLiteral(value); return { name, value, esModuleStamp }; } +/** `true`, or `!0`, its minified spelling. */ +function isTrueLiteral(expr: ts.Expression): boolean { + return ( + expr.kind === ts.SyntaxKind.TrueKeyword || + (ts.isPrefixUnaryExpression(expr) && + expr.operator === ts.SyntaxKind.ExclamationToken && + ts.isNumericLiteral(expr.operand) && + expr.operand.text === "0") + ); +} + /** Space-pads [start, end) of `text` preserving newlines — the recognized * statement disappears while every original offset survives. */ function padSpan(text: string, start: number, end: number): string { diff --git a/packages/compiler/src/frontend/npm-static.ts b/packages/compiler/src/frontend/npm-static.ts index 800d02eff..eed02c95c 100644 --- a/packages/compiler/src/frontend/npm-static.ts +++ b/packages/compiler/src/frontend/npm-static.ts @@ -614,27 +614,12 @@ export function clearProgramBuiltinAliasCache(): void { * but the bundler-emitted CJS export rewrite (npm-static-rewrite.ts) * types the getter-table shapes and the offender attribution degrades a * package whose surface still breaks its consumers — so those bundles are - * worth ATTEMPTING: eligible when the .d.ts and unminified criteria hold, - * gracefully per-package-degraded when the attempt fails. */ + * worth ATTEMPTING: eligible when the .d.ts criteria hold, gracefully + * per-package-degraded when the attempt fails. Minified JS is attempted + * too: a minified dist is ordinary JavaScript, and the bundle recognizers + * know esbuild's __commonJS helper by its shape under any name. */ const TRANSFORM_MARKERS = ["__webpack_require__"]; -/** Unminified-JS heuristic over an entry source: at least two lines and - * an average line length under 200 characters (minified dists are one - * enormous line; readable code averages well under 100). */ -function looksUnminified(source: string): boolean { - const lines = source.split("\n"); - if (lines.length < 2) return false; - return source.length / lines.length <= 200; -} - -/** A whitespace/syntax-minified bundler dist whose interop helpers keep - * their names (`var __toESM=…,__toCommonJS=mod=>…`): the recognizers - * verify those shapes and refuse the package precisely when they cannot, - * so the minification heuristic need not refuse it up front. */ -function namedBundlerHelpers(source: string): boolean { - return /\b__(?:toCommonJS|commonJS|toESM)\d*\s*=\s*(?:\(|[A-Za-z_$][\w$]*\s*=>)/.test(source); -} - function hasTransformMarkers(source: string): boolean { return TRANSFORM_MARKERS.some((m) => source.includes(m)); } @@ -676,8 +661,6 @@ export function npmStaticIneligibleReason( if (jsEntry === null) return "no runtime JS entry resolves"; const source = trackedReadFile(jsEntry); if (source === null) return `its runtime entry ${jsEntry} cannot be read`; - if (!looksUnminified(source) && !namedBundlerHelpers(source)) - return "its shipped JS looks minified"; if (hasTransformMarkers(source)) { return "its shipped JS carries build-transform markers (bundled/transpiled dist)"; } diff --git a/packages/compiler/src/frontend/package-shims.ts b/packages/compiler/src/frontend/package-shims.ts index a0c2e4cb9..10beeaae8 100644 --- a/packages/compiler/src/frontend/package-shims.ts +++ b/packages/compiler/src/frontend/package-shims.ts @@ -63,7 +63,7 @@ interface ImplementationShim { * and this file) is the package itself. */ mainFile: string; /** The package's declaration entry relative to its directory, when it - * is not `index.d.ts`. */ + * is not `index.d.ts` (a `.d.mts` or `.d.cts` entry included). */ typesFile?: string; /** An alternative entry for this installation (relative to the shim * root), or undefined for the version's own entry. */ @@ -117,6 +117,15 @@ const IMPLEMENTATION_SHIMS: ReadonlyMap = new Map< variants: ["hot-shots/v8-unix-dgram.ts"], }, ], + [ + "@hono/node-server", + { + name: "@hono/node-server", + mainFile: "dist/index.mjs", + typesFile: "dist/index.d.mts", + versions: new Map([[2, "hono-node-server/v2.ts"]]), + }, + ], [ "undici", { @@ -324,7 +333,7 @@ export function isShimmedImport(fromFile: string, specifier: string): boolean { /** The shimmed package whose declaration entry `path` is, with the * package directory: `/index.d.ts`, or the entry's own `typesFile`. */ function shimOfDeclarationPath(path: string): { pkgDir: string; shim: ImplementationShim } | null { - if (!path.endsWith(".d.ts")) return null; + if (!/\.d\.[cm]?ts$/.test(path)) return null; if (path.endsWith("/index.d.ts")) { const pkgDir = dirname(path); const shim = IMPLEMENTATION_SHIMS.get(pkgDir.slice(pkgDir.lastIndexOf("/") + 1)); @@ -377,7 +386,7 @@ export function isPackageShimFile(fileName: string): boolean { /** True for a name a shipped shim may declare as a bodiless runtime * intrinsic (lower-calls.ts lowers its calls to runtime calls). */ export function isShimIntrinsicName(name: string): boolean { - return name.startsWith("scriptcUnixDgram"); + return name.startsWith("scriptcUnixDgram") || name === "scriptcResponseLight"; } /** Resets per-compile caches. */ diff --git a/packages/compiler/src/frontend/program.test.ts b/packages/compiler/src/frontend/program.test.ts index bdd6260ed..6a068353a 100644 --- a/packages/compiler/src/frontend/program.test.ts +++ b/packages/compiler/src/frontend/program.test.ts @@ -371,15 +371,60 @@ test("repeated cycle edges reuse a file's binding index", () => { test.for([ [ - // A shared instance built by a user constructor: the reads of it in - // the partner's function can run early; the class cannot. + // A shared instance built by a user constructor that calls the + // partner's function: its read of the instance can run early. + { + "main.ts": 'import { shared } from "./a.ts"; console.log(shared.n);', + "a.ts": + 'import { f } from "./b.ts"; export class C { n = 1; constructor() { console.log(f()); } } export const shared = new C();', + "b.ts": 'import { shared } from "./a.ts"; export function f(): number { return shared.n; }', + }, + ["shared"], + ], + [ + // User code runs while the instance is built, but nothing it reaches + // calls the partner's function before the declaration completed. { "main.ts": 'import { shared } from "./a.ts"; console.log(shared.n);', "a.ts": 'import { f } from "./b.ts"; export class C { n = 1; } export const shared = new C(); console.log(f());', "b.ts": 'import { shared } from "./a.ts"; export function f(): number { return shared.n; }', }, - ["shared"], + [], + ], + [ + // The partner's top level runs first and calls a hoisted function of + // the importer that reads a const declared later; a second function + // reads another const but nothing calls it early. + { + "main.ts": 'import { late } from "./a.ts"; console.log(late());', + "a.ts": + 'import { probe } from "./b.ts"; export function read(): string { return name; } export function late(): string { return other + probe; } export const name = "n"; export const other = "o";', + "b.ts": + 'import { read } from "./a.ts"; function attempt(): string { try { return read(); } catch { return "tdz"; } } export const probe = attempt();', + }, + ["name"], + ], + [ + // The partner's class calls a method through `this` in a static + // initializer; the method reads a const of the importer declared later. + { + "main.ts": 'import { name } from "./a.ts"; console.log(name);', + "a.ts": 'import { Probe } from "./b.ts"; export const name = "n"; console.log(Probe.value);', + "b.ts": + 'import { name } from "./a.ts"; export class Probe { static read(): string { return name; } static value = this.read(); }', + }, + ["name"], + ], + [ + // The same through a static block of a class stored by a const. + { + "main.ts": 'import { name } from "./a.ts"; console.log(name);', + "a.ts": 'import { Box } from "./b.ts"; export const name = "n"; console.log(Box.value);', + "b.ts": + 'import { name } from "./a.ts"; export const Box = class { static value = ""; static read(): string { return name; } static { this.value = this.read(); } };', + }, + ["name"], ], [ // The partner's top level reads the importer's const before it ran. diff --git a/packages/compiler/src/frontend/program.ts b/packages/compiler/src/frontend/program.ts index 68b649b07..6d8ffe4c0 100644 --- a/packages/compiler/src/frontend/program.ts +++ b/packages/compiler/src/frontend/program.ts @@ -1436,8 +1436,20 @@ function requiresOf7( return out; } +/** `true`, or `!0`, its minified spelling. */ +function isTrueLiteral(expr: ts.Expression): boolean { + return ( + expr.kind === ts.SyntaxKind.TrueKeyword || + (ts.isPrefixUnaryExpression(expr) && + expr.operator === ts.SyntaxKind.ExclamationToken && + ts.isNumericLiteral(expr.operand) && + expr.operand.text === "0") + ); +} + /** The exact `Object.defineProperty(exports|module.exports, "__esModule", - * { value: true })` interop stamp (shared by preflight and the no-op lowering). */ + * { value: true })` interop stamp (shared by preflight and the no-op lowering); + * `{ value: !0 }` is the same stamp. */ export function isEsModuleStamp(expr: ts.Expression): boolean { if (!ts.isCallExpression(expr) || expr.questionDotToken !== undefined) return false; const callee = expr.expression; @@ -1471,7 +1483,7 @@ export function isEsModuleStamp(expr: ts.Expression): boolean { ts.isPropertyAssignment(p) && ts.isIdentifier(p.name) && p.name.text === "value" && - p.initializer.kind === ts.SyntaxKind.TrueKeyword + isTrueLiteral(p.initializer) ); } @@ -3891,22 +3903,7 @@ function computeCycleEarlyBindings7( const evaluation = known ? comp.slice().sort((a, b) => position.get(a)! - position.get(b)!) : comp.slice(); - interface Candidate { - symbol: ts.Symbol; - name: ts.Identifier; - kind: CycleEarlyBinding["kind"]; - module: ts.SourceFile; - /** User code can run between the cycle's start and the end of the - * declaration (always true when the order is unknown). */ - exposed: boolean; - rank: number; - start: number; - end: number; - reference: ts.Identifier | null; - folded: ts.Identifier | null; - localNames: Map; - } - const candidates = new Map(); + const candidates = new Map(); const names = new Set(); let dirty = !known; evaluation.forEach((m, rank) => { @@ -3943,6 +3940,7 @@ function computeCycleEarlyBindings7( start: ts.tokenStartOfNode(stmt), end: stmt.end, reference: null, + deferredReference: null, folded: null, localNames: new Map(), }); @@ -4011,15 +4009,27 @@ function computeCycleEarlyBindings7( ref.end <= c.end ) continue; + if (inDeferredPosition7(ref)) { + if (c.exposed && c.deferredReference === null) c.deferredReference = ref; + continue; + } let early: boolean; - if (inDeferredPosition7(ref)) early = c.exposed; - else if (sf === c.module) early = ts.tokenStartOfNode(ref) < c.start; + if (sf === c.module) early = ts.tokenStartOfNode(ref) < c.start; else if (!memberSet.has(sf) || !rankOf.has(sf)) early = false; else early = !known || rankOf.get(sf)! < c.rank; if (early) c.reference = ref; } } + // With a static evaluation order, a deferred reference only runs early + // when the code that runs before the declaration completes can reach + // the function holding it. + const reached = + known && [...candidates.values()].some((c) => c.reference === null && c.deferredReference) + ? cycleReachedDeferredReferences7(program, evaluation, candidates) + : null; for (const c of candidates.values()) { + if (c.reference === null && c.deferredReference !== null) + c.reference = reached === null ? c.deferredReference : (reached.get(c.symbol) ?? null); if (c.reference === null) continue; const settled = new Set(order.filter((sf) => !rankOf.has(sf))); if (known) for (const m of evaluation.slice(c.rank + 1)) settled.add(m); @@ -4038,6 +4048,299 @@ function computeCycleEarlyBindings7( return out; } +/** Collects into `units` the callables an initializer expression stores + * (see moduleEarlyBindingsOf7), recording in `immediate` the parts of a + * stored node that run where it is evaluated. */ +function storedCallables7( + checker: ts.TypeChecker, + e: ts.Expression, + units: ts.Node[], + immediate: Map, +): void { + while ( + ts.isParenthesizedExpression(e) || + ts.isAsExpression(e) || + ts.isSatisfiesExpression(e) || + ts.isNonNullExpression(e) + ) + e = e.expression; + if (ts.isArrowFunction(e) || ts.isFunctionExpression(e)) { + units.push(e); + } else if (ts.isClassExpression(e)) { + units.push(e); + immediate.set(e, classSyncParts7(e)); + } else if (ts.isObjectLiteralExpression(e)) { + for (const p of e.properties.nodes) { + if ( + ts.isMethodDeclaration(p) || + ts.isGetAccessorDeclaration(p) || + ts.isSetAccessorDeclaration(p) + ) { + // A computed key evaluates where the literal does; a plain + // member-chain key (`[Equal.symbol]`) runs no user code. + if (!ts.isComputedPropertyName(p.name)) units.push(p); + else if (memberChainKey7(p.name.expression)) { + units.push(p); + immediate.set(p, [p.name]); + } + } else if (ts.isPropertyAssignment(p)) + storedCallables7(checker, p.initializer, units, immediate); + } + } else if (ts.isArrayLiteralExpression(e)) { + for (const el of e.elements.nodes) + if (!ts.isSpreadElement(el)) storedCallables7(checker, el, units, immediate); + } else if (ts.isCallExpression(e)) { + // `const f = wrap(callback)` where wrap only returns closures over + // the callback: it runs when the stored result is called. + e.arguments.nodes.forEach((arg, index) => { + let fn: ts.Expression = arg; + while (ts.isParenthesizedExpression(fn)) fn = fn.expression; + if ( + (ts.isArrowFunction(fn) || ts.isFunctionExpression(fn)) && + returnsOnlyClosuresOver7(checker, e.expression, index) + ) + units.push(fn); + }); + } +} + +/** A binding of an import cycle and what the reference scan found. */ +interface CycleCandidate7 { + symbol: ts.Symbol; + name: ts.Identifier; + kind: CycleEarlyBinding["kind"]; + module: ts.SourceFile; + /** User code can run between the cycle's start and the end of the + * declaration (always true when the order is unknown). */ + exposed: boolean; + rank: number; + start: number; + end: number; + /** A reference read at top level before the declaration ran. */ + reference: ts.Identifier | null; + /** A deferred reference (function body, parameter default, instance + * field) when user code may run before the declaration completes. */ + deferredReference: ts.Identifier | null; + folded: ts.Identifier | null; + localNames: Map; +} + +/** For one import cycle with a static evaluation order: the first deferred + * reference to each candidate that code running before the candidate's + * declaration completes can reach, or null when the walk cannot be precise + * (top-level await, a default-exported function or class declaration). + * + * The cycle's top-level code runs in evaluation order. A stored callable + * (a function or class declaration, or a function, class or object-literal + * method stored by a variable initializer) becomes reachable once + * reachable code names its binding, through an import alias or a + * namespace member too; every other function-like in reachable code counts + * as reachable, since it may be handed to a callee. A namespace import of + * a cycle member used other than for a member read makes every stored + * callable reachable. moduleEarlyBindingsOf7 is the single-module form. */ +function cycleReachedDeferredReferences7( + program: ts.Program, + evaluation: readonly ts.SourceFile[], + candidates: ReadonlyMap, +): Map | null { + const checker = program.getTypeChecker(); + const members = new Set(evaluation); + for (const sf of evaluation) { + if (hasTopLevelAwait7(sf)) return null; + for (const stmt of sf.statements.nodes) { + if ( + (ts.isFunctionDeclaration(stmt) || ts.isClassDeclaration(stmt)) && + (ts.getCombinedModifierFlags(stmt as unknown as ts.Declaration) & + ts.ModifierFlags.Default) !== + 0 + ) + return null; + } + } + const stored = new Set(); + const immediate = new Map(); + const storedNames: ts.Identifier[] = []; + const unitsOfName = new Map(); + const store = (name: ts.Identifier, units: ts.Node[]): void => { + if (units.length === 0) return; + for (const u of units) stored.add(u); + storedNames.push(name); + unitsOfName.set(name, units); + }; + for (const sf of evaluation) { + for (const stmt of sf.statements.nodes) { + if ( + (ts.getCombinedModifierFlags(stmt as unknown as ts.Declaration) & + ts.ModifierFlags.Ambient) !== + 0 + ) + continue; + if (ts.isFunctionDeclaration(stmt) && stmt.name !== undefined && stmt.body !== undefined) { + store(stmt.name, [stmt]); + } else if (ts.isClassDeclaration(stmt) && stmt.name !== undefined) { + store(stmt.name, [stmt]); + } else if (ts.isVariableStatement(stmt)) { + for (const d of stmt.declarationList.declarations.nodes) { + if (ts.isIdentifier(d.name) && d.initializer !== undefined) { + const units: ts.Node[] = []; + storedCallables7(checker, d.initializer, units, immediate); + store(d.name, units); + } + } + } + } + } + const unitsBySymbol = new Map(); + const allUnits: ts.Node[] = []; + for (const name of storedNames) { + const symbol = checker.getSymbolAtLocation(name); + if (symbol === undefined) continue; + const units = unitsOfName.get(name)!; + unitsBySymbol.set(symbol, [...(unitsBySymbol.get(symbol) ?? []), ...units]); + allUnits.push(...units); + } + // A named class expression's inner name (`class Impl { static x = + // Impl.m(); }`) names the class inside its own body, as the binding it + // is stored in names it outside. + const innerNames: ts.Identifier[] = []; + for (const unit of allUnits) { + if (!ts.isClassExpression(unit) || unit.name === undefined) continue; + const symbol = checker.getSymbolAtLocation(unit.name); + if (symbol === undefined) continue; + innerNames.push(unit.name); + unitsBySymbol.set(symbol, [...(unitsBySymbol.get(symbol) ?? []), unit]); + } + // The identifier texts worth resolving: declared names, and the local + // names of imports (named, default and namespace) in cycle members. + const names = new Set(); + for (const c of candidates.values()) names.add(c.name.text); + for (const name of storedNames) names.add(name.text); + for (const name of innerNames) names.add(name.text); + for (const sf of evaluation) { + for (const stmt of sf.statements.nodes) { + if (!ts.isImportDeclaration(stmt) || stmt.importClause === undefined) continue; + const clause = stmt.importClause; + if (clause.name !== undefined) names.add(clause.name.text); + const bindings = clause.namedBindings; + if (bindings === undefined) continue; + if (ts.isNamedImports(bindings)) + for (const element of bindings.elements.nodes) names.add(element.name.text); + else names.add(bindings.name.text); + } + } + const idents: ts.Identifier[] = []; + for (const sf of evaluation) { + ts.walkPreorder(sf, (node) => { + if (ts.isImportDeclaration(node)) return "skip"; + if (ts.isIdentifier(node) && names.has(node.text)) idents.push(node); + return undefined; + }); + } + checker.prefetchSymbolNodesExact(idents); + const symbolOf = (id: ts.Identifier): ts.Symbol | undefined => { + const parent = id.parent; + let symbol = checker.getSymbolAtLocation(id); + if (parent !== undefined && ts.isShorthandPropertyAssignment(parent) && parent.name === id) { + symbol = checker.getShorthandAssignmentValueSymbol(parent) ?? symbol; + } + if (symbol !== undefined && symbol.flags & ts.SymbolFlags.Alias) { + symbol = checker.getAliasedSymbol(symbol); + } + return symbol; + }; + const isMemberNamespace = (symbol: ts.Symbol): boolean => + (symbol.flags & ts.SymbolFlags.ValueModule) !== 0 && + symbol.declarations.some((d) => ts.isSourceFile(d) && members.has(d)); + const declared = new Set(); + const reachedUnits = new Set(); + const out = new Map(); + const work: ts.Node[] = []; + const reach = (unit: ts.Node): void => { + if (reachedUnits.has(unit)) return; + reachedUnits.add(unit); + work.push(unit); + }; + const visit = (root: ts.Node): void => { + ts.walkPreorder(root, (node) => { + if (node !== root && stored.has(node)) { + for (const part of immediate.get(node) ?? []) visit(part); + return "skip"; + } + if (ts.isImportDeclaration(node)) return "skip"; + if (node.kind === ts.SyntaxKind.ThisKeyword) { + // `this` in a static initializer or block names the class, so it + // reaches the class's methods as naming the class does. + const owner = staticThisClass7(node); + if (owner !== undefined && stored.has(owner)) reach(owner); + return undefined; + } + if (!ts.isIdentifier(node) || !names.has(node.text)) return undefined; + if (inTypePosition7(node)) return undefined; + const parent = node.parent; + if (parent !== undefined && ts.isExportSpecifier(parent)) return undefined; + if (parent !== undefined && declaredName7(parent) === node) return undefined; + const symbol = symbolOf(node); + if (symbol === undefined) return undefined; + if (isMemberNamespace(symbol)) { + // `ns.member` resolves through the member's own identifier; any + // other use can reach every export. + if ( + !( + parent !== undefined && + ts.isPropertyAccessExpression(parent) && + parent.expression === node + ) + ) + for (const unit of allUnits) reach(unit); + return undefined; + } + const c = candidates.get(symbol); + if ( + c !== undefined && + node !== c.name && + !declared.has(symbol) && + !out.has(symbol) && + inDeferredPosition7(node) && + !( + c.kind === "class" && + ts.getSourceFileOfNode(node) === c.module && + ts.tokenStartOfNode(node) >= c.start && + node.end <= c.end + ) + ) + out.set(symbol, node); + for (const unit of unitsBySymbol.get(symbol) ?? []) reach(unit); + return undefined; + }); + }; + const drain = (): void => { + while (work.length > 0) visit(work.pop()!); + }; + const declare = (name: ts.BindingName): void => { + for (const id of boundNames7(name)) { + const symbol = checker.getSymbolAtLocation(id); + if (symbol !== undefined) declared.add(symbol); + } + }; + for (const sf of evaluation) { + for (const stmt of sf.statements.nodes) { + if (ts.isClassDeclaration(stmt)) { + for (const part of classSyncParts7(stmt)) visit(part); + drain(); + if (stmt.name !== undefined) declare(stmt.name); + } else if (ts.isVariableStatement(stmt)) { + for (const d of stmt.declarationList.declarations.nodes) { + visit(d); + drain(); + declare(d.name); + } + } else if (!ts.isFunctionDeclaration(stmt)) visit(stmt); + drain(); + } + } + return out; +} + /** Top-level let/const and class bindings, in modules outside the * exactly-initialized cycles, that code running before their declaration * completes can read. @@ -4102,51 +4405,8 @@ function moduleEarlyBindingsOf7( storedBy.set(name.text, list); }; const checker = program.getTypeChecker(); - const storedIn = (e: ts.Expression, units: ts.Node[]): void => { - while ( - ts.isParenthesizedExpression(e) || - ts.isAsExpression(e) || - ts.isSatisfiesExpression(e) || - ts.isNonNullExpression(e) - ) - e = e.expression; - if (ts.isArrowFunction(e) || ts.isFunctionExpression(e)) { - units.push(e); - } else if (ts.isClassExpression(e)) { - units.push(e); - immediate.set(e, classSyncParts7(e)); - } else if (ts.isObjectLiteralExpression(e)) { - for (const p of e.properties.nodes) { - if ( - ts.isMethodDeclaration(p) || - ts.isGetAccessorDeclaration(p) || - ts.isSetAccessorDeclaration(p) - ) { - // A computed key evaluates where the literal does; a plain - // member-chain key (`[Equal.symbol]`) runs no user code. - if (!ts.isComputedPropertyName(p.name)) units.push(p); - else if (memberChainKey7(p.name.expression)) { - units.push(p); - immediate.set(p, [p.name]); - } - } else if (ts.isPropertyAssignment(p)) storedIn(p.initializer, units); - } - } else if (ts.isArrayLiteralExpression(e)) { - for (const el of e.elements.nodes) if (!ts.isSpreadElement(el)) storedIn(el, units); - } else if (ts.isCallExpression(e)) { - // `const f = wrap(callback)` where wrap only returns closures over - // the callback: it runs when the stored result is called. - e.arguments.nodes.forEach((arg, index) => { - let fn: ts.Expression = arg; - while (ts.isParenthesizedExpression(fn)) fn = fn.expression; - if ( - (ts.isArrowFunction(fn) || ts.isFunctionExpression(fn)) && - returnsOnlyClosuresOver7(checker, e.expression, index) - ) - units.push(fn); - }); - } - }; + const storedIn = (e: ts.Expression, units: ts.Node[]): void => + storedCallables7(checker, e, units, immediate); sf.statements.nodes.forEach((stmt, index) => { if (ambient(stmt)) return; if (ts.isFunctionDeclaration(stmt) && stmt.name !== undefined && stmt.body !== undefined) { @@ -4399,6 +4659,40 @@ function returnsOnlyClosuresOver7( * expressions, decorators, computed member keys, static field * initializers and static blocks. Methods and instance fields wait for a * call or a construction. */ +/** The class whose static field initializer or static block evaluates `this` + * (through arrow functions, which keep the enclosing `this`), or undefined + * when another function or a class's instance side binds it. */ +function staticThisClass7(node: ts.Node): ts.ClassLikeDeclaration | undefined { + let child = node; + for (let n = node.parent; n !== undefined; child = n, n = n.parent) { + if (ts.isArrowFunction(n)) continue; + if (ts.isClassStaticBlockDeclaration(n)) return n.parent as ts.ClassLikeDeclaration; + if (ts.isPropertyDeclaration(n)) { + if (n.initializer !== child) return undefined; + return (ts.getCombinedModifierFlags(n) & ts.ModifierFlags.Static) !== 0 + ? (n.parent as ts.ClassLikeDeclaration) + : undefined; + } + if (ts.isComputedPropertyName(n)) { + // A computed key evaluates in the scope around the class. + const member = n.parent; + const cls = member?.parent; + if (cls === undefined) return undefined; + child = cls; + n = cls; + continue; + } + if ( + ts.isFunctionLike(n) || + ts.isClassDeclaration(n) || + ts.isClassExpression(n) || + ts.isSourceFile(n) + ) + return undefined; + } + return undefined; +} + function classSyncParts7(c: ts.ClassLikeDeclaration): ts.Node[] { const decorators = (n: ts.Node): ts.Node[] => ((n as { modifiers?: ts.NodeArray }).modifiers?.nodes ?? []).filter( diff --git a/packages/compiler/src/frontend/ts7/lib-diagnostics.test.ts b/packages/compiler/src/frontend/ts7/lib-diagnostics.test.ts new file mode 100644 index 000000000..f041a6cb5 --- /dev/null +++ b/packages/compiler/src/frontend/ts7/lib-diagnostics.test.ts @@ -0,0 +1,221 @@ +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterAll, afterEach, expect, test } from "vitest"; +import type { SourceFile } from "./ast-types.js"; +import { + reusedLibDiagnostics, + setLibDiagnosticsStore, + type LibDiagnosticsStore, +} from "./lib-diagnostics.js"; +import type { Ts7DiagnosticData } from "./session-schema.generated.js"; +import type { Ts7CheckedFileDiagnosticData, Ts7SessionProgram } from "./session.js"; +import { closeSourceParser, parseSourceFile } from "./source-parser-node.js"; + +const dir = mkdtempSync(join(tmpdir(), "scriptc-lib-diagnostics-")); +const lib = join(dir, "lib.es5.d.ts"); +const fallback = join(dir, "scriptc-node-fallback.d.ts"); +const libText = "declare var NaN: number;\ninterface Array { length: number }\n"; +const fallbackText = "export {};\ndeclare global {\n var process: { argv: string[] };\n}\n"; +writeFileSync(lib, libText); +writeFileSync(fallback, fallbackText); +const main = join(dir, "main.ts"); +const util = join(dir, "\u{1F600}.ts"); +const other = join(dir, "�.ts"); + +afterEach(() => setLibDiagnosticsStore(null)); +afterAll(() => { + closeSourceParser(); + rmSync(dir, { recursive: true, force: true }); +}); + +function memoryStore(): LibDiagnosticsStore & { entries: Map } { + const entries = new Map(); + return { + entries, + read: (key) => entries.get(key) ?? null, + write: (key, text) => void entries.set(key, text), + }; +} + +function diagnostic(fileName: string | undefined, pos: number, code = 2322): Ts7DiagnosticData { + return { fileName, pos, end: pos + 1, code, category: 1, text: `at ${pos}` }; +} + +/** A session program whose full-check and checked-file answers are given; it + * records which requests it was asked. */ +function sessionProgram(answers: { + semantic: Map; + global?: Ts7DiagnosticData[]; + checked?: boolean; +}): Ts7SessionProgram & { asked: string[] } { + const asked: string[] = []; + const all = [...answers.semantic.values()].flat(); + return { + asked, + getCompilerOptions: () => ({ strict: true }), + diagnosticData: (method: string, file?: string) => { + asked.push(file === undefined ? method : `${method} ${file}`); + if (method === "getGlobalDiagnostics") return answers.global ?? []; + return file === undefined ? all : (answers.semantic.get(file) ?? []); + }, + checkedFileDiagnosticData: (files: readonly string[]): Ts7CheckedFileDiagnosticData | null => { + asked.push(`checked ${files.join(",")}`); + if (answers.checked === false) return null; + return { files: files.map((file) => answers.semantic.get(file) ?? []), global: [] }; + }, + } as unknown as Ts7SessionProgram & { asked: string[] }; +} + +/** The program's files, parsed: the supplied two and the given others. */ +function files(texts: Record): (name: string) => SourceFile | undefined { + const parsed = new Map([ + [lib, parseSourceFile(lib, libText, "ts")], + [fallback, parseSourceFile(fallback, fallbackText, "ts")], + ]); + for (const [name, text] of Object.entries(texts)) + parsed.set(name, parseSourceFile(name, text, "ts")); + return (name) => parsed.get(name); +} + +const names = [lib, fallback, main]; +const supplied = (name: string): boolean => name === lib || name === fallback; +const moduleFile = files({ [main]: "export const n: number = 'x';\n" }); + +test("without a store the full check answers", () => { + const program = sessionProgram({ semantic: new Map() }); + expect(reusedLibDiagnostics(program, names, supplied, moduleFile)).toBeNull(); + expect(program.asked).toEqual([]); +}); + +test("a stored answer stands in for checking the supplied files, merged by file name", () => { + const store = memoryStore(); + setLibDiagnosticsStore(store, "checker"); + const semantic = new Map([ + [lib, [diagnostic(lib, 1)]], + [fallback, []], + [main, [diagnostic(main, 5), diagnostic(main, 9)]], + ]); + const first = sessionProgram({ semantic }); + const full = reusedLibDiagnostics(first, names, supplied, moduleFile); + // The first build runs the full check and stores the supplied files' answers. + expect(first.asked).toEqual([ + "getGlobalDiagnostics", + "getSemanticDiagnostics", + "checked ", + `getSemanticDiagnostics ${lib}`, + `getSemanticDiagnostics ${fallback}`, + ]); + expect(store.entries.size).toBe(1); + const second = sessionProgram({ semantic }); + const reused = reusedLibDiagnostics(second, names, supplied, moduleFile); + expect(second.asked).toEqual([`checked ${main}`]); + // The files' diagnostics in the order of their names (lib.es5.d.ts, main.ts), as the full answer. + expect(reused).toEqual({ + semantic: [diagnostic(lib, 1), diagnostic(main, 5), diagnostic(main, 9)], + global: [], + }); + expect(full?.semantic).toEqual(reused?.semantic); +}); + +test("files merge in code point order, which differs from UTF-16 order", () => { + const store = memoryStore(); + setLibDiagnosticsStore(store, "checker"); + const all = [lib, fallback, util, other]; + const sources = files({ [util]: "export {};\n", [other]: "export {};\n" }); + const semantic = new Map([ + [util, [diagnostic(util, 1)]], + [other, [diagnostic(other, 2)]], + ]); + reusedLibDiagnostics(sessionProgram({ semantic }), all, supplied, sources); + const reused = reusedLibDiagnostics(sessionProgram({ semantic }), all, supplied, sources); + // U+FFFD sorts before U+1F600 by code point (its surrogates sort after it by UTF-16 unit). + expect(reused?.semantic).toEqual([diagnostic(other, 2), diagnostic(util, 1)]); +}); + +test("a full check with global diagnostics is not stored", () => { + const store = memoryStore(); + setLibDiagnosticsStore(store, "checker"); + const program = sessionProgram({ + semantic: new Map([[main, [diagnostic(main, 1)]]]), + global: [diagnostic(undefined, 0, 2318)], + }); + expect(reusedLibDiagnostics(program, names, supplied, moduleFile)?.global).toEqual([ + diagnostic(undefined, 0, 2318), + ]); + expect(store.entries.size).toBe(0); +}); + +test("scripts reuse the answer unless they declare a global the supplied files declare", () => { + const store = memoryStore(); + setLibDiagnosticsStore(store, "checker"); + const semantic = new Map([[main, []]]); + // A script declaring only new names: it stores the answer, and a later one reuses it. + const fresh = files({ + [main]: "const total = 1;\nfunction run() {}\nif (total) { var later = 2; }\n", + }); + reusedLibDiagnostics(sessionProgram({ semantic }), names, supplied, fresh); + expect(store.entries.size).toBe(1); + const again = sessionProgram({ semantic }); + expect(reusedLibDiagnostics(again, names, supplied, fresh)).not.toBeNull(); + expect(again.asked).toEqual([`checked ${main}`]); + // Scripts declaring a supplied global, at the top level, in a block, in a + // `declare global` of a supplied module, or an ambient module: the full check. + for (const text of [ + "interface Array { extra: T }\n", + "if (true) { var NaN = 1; }\n", + "declare var process: unknown;\n", + 'declare module "fs" {}\n', + ]) { + const program = sessionProgram({ semantic }); + const result = reusedLibDiagnostics(program, names, supplied, files({ [main]: text })); + expect(program.asked[0], text).not.toBe(`checked ${main}`); + if (result !== null) expect(program.asked[0]).toBe("getGlobalDiagnostics"); + } + expect(store.entries.size).toBe(1); +}); + +test("a program whose script declares a supplied global does not store its answer", () => { + const store = memoryStore(); + setLibDiagnosticsStore(store, "checker"); + const program = sessionProgram({ semantic: new Map([[main, []]]) }); + reusedLibDiagnostics(program, names, supplied, files({ [main]: "declare var NaN: number;\n" })); + expect(store.entries.size).toBe(0); +}); + +test("modules that can declare something global, and servers without the method, ask as before", () => { + const store = memoryStore(); + setLibDiagnosticsStore(store, "checker"); + const semantic = new Map([[main, []]]); + reusedLibDiagnostics(sessionProgram({ semantic }), names, supplied, moduleFile); + expect(store.entries.size).toBe(1); + for (const text of [ + "export {};\ndeclare global { var extra: number }\n", + 'export {};\ndeclare module "./other" { const extra: number }\n', + "export as namespace Lib;\nexport const n = 1;\n", + ]) { + const program = sessionProgram({ semantic }); + expect(reusedLibDiagnostics(program, names, supplied, files({ [main]: text }))).toBeNull(); + expect(program.asked).toEqual([]); + } + const old = sessionProgram({ semantic, checked: false }); + expect(reusedLibDiagnostics(old, names, supplied, moduleFile)).toBeNull(); + // Nor does a server without the method store an answer it could not use. + const fresh = memoryStore(); + setLibDiagnosticsStore(fresh, "checker"); + reusedLibDiagnostics(sessionProgram({ semantic, checked: false }), names, supplied, moduleFile); + expect(fresh.entries.size).toBe(0); +}); + +test("a supplied file that changes is a different entry", () => { + const store = memoryStore(); + setLibDiagnosticsStore(store, "checker"); + const semantic = new Map([[main, []]]); + reusedLibDiagnostics(sessionProgram({ semantic }), names, supplied, moduleFile); + writeFileSync(fallback, fallbackText + "// edited\n"); + const program = sessionProgram({ semantic }); + reusedLibDiagnostics(program, names, supplied, moduleFile); + expect(program.asked[0]).toBe("getGlobalDiagnostics"); + expect(store.entries.size).toBe(2); + writeFileSync(fallback, fallbackText); +}); diff --git a/packages/compiler/src/frontend/ts7/lib-diagnostics.ts b/packages/compiler/src/frontend/ts7/lib-diagnostics.ts new file mode 100644 index 000000000..827b337bd --- /dev/null +++ b/packages/compiler/src/frontend/ts7/lib-diagnostics.ts @@ -0,0 +1,324 @@ +/* The semantic diagnostics of the declaration files scriptc supplies (the + * TypeScript libs and scriptc's own declaration files), reused across + * builds. Checking them is most of the full check of a small program. When + * no other file of a program can change what checking those files reports + * (every one is a module without global or module augmentations, or a + * script declaring no global name the supplied files declare), the answer + * depends only on the checker, the compiler options and the files + * themselves, so a stored answer for the same three stands in for checking + * them again: the server checks only the program's other files + * (getDiagnosticsOfCheckedFiles), and the semantic diagnostics merge in the + * order the full check sorts them, by file name. An answer is stored only + * when the full check reports no global diagnostics: the global diagnostics + * of checking the other files are then the full check's. */ +import { createHash } from "node:crypto"; +import { statSync } from "node:fs"; +import type { Node, SourceFile } from "./ast-types.js"; +import type { Ts7DiagnosticData } from "./session-schema.generated.js"; +import type { Ts7SessionProgram } from "./session.js"; +import { NodeFlags, SyntaxKind } from "./enums.js"; +import { walkPreorder } from "./ast.js"; + +/** Key-value storage for reused diagnostics (best effort: failures miss). */ +export interface LibDiagnosticsStore { + read(key: string): string | null; + write(key: string, text: string): void; +} + +const SCHEMA = "scriptc.lib-diagnostics.v1"; + +let configured: { store: LibDiagnosticsStore; checker: string } | null = null; + +/** Has later builds of this process reuse the diagnostics of the supplied + * declaration files through `store`, for the checker identified by + * `checker` (its executable's identity), or stop reusing them (null). */ +export function setLibDiagnosticsStore(store: LibDiagnosticsStore | null, checker = ""): void { + configured = store === null ? null : { store, checker }; +} + +/** A checker executable's identity for the stored diagnostics' key: its + * path and file identity, or the path alone when it cannot be read. */ +export function checkerExecutableIdentity(path: string): string { + try { + const info = statSync(path); + return JSON.stringify([path, info.size, info.mtimeMs, info.ino]); + } catch { + return JSON.stringify([path]); + } +} + +/** The program's semantic and global diagnostics as the full check reports + * them, using stored diagnostics for the supplied declaration files; null + * when they cannot be reused for this program (then the caller asks for + * them as before). `supplied` tells whether a program file is one of the + * supplied declaration files. */ +export function reusedLibDiagnostics( + program: Ts7SessionProgram, + fileNames: readonly string[], + supplied: (fileName: string) => boolean, + sourceFile: (fileName: string) => SourceFile | undefined, +): { semantic: Ts7DiagnosticData[]; global: Ts7DiagnosticData[] } | null { + if (configured === null || process.env["SCRIPTC_LIB_DIAGNOSTICS"] === "0") return null; + const { store, checker } = configured; + const libs: string[] = []; + const own: string[] = []; + for (const name of fileNames) (supplied(name) ? libs : own).push(name); + if (libs.length === 0) return null; + // The global names each of the program's scripts declares; a module that + // declares nothing global contributes none. + const scripts: Set[] = []; + for (const name of own) { + const file = sourceFile(name); + if (file === undefined) return null; + if (file.externalModuleIndicator !== undefined) { + if (!declaresNothingGlobal(file)) return null; + continue; + } + const names = isJsFileName(name) ? null : scriptGlobalNames(file); + if (names === null) return null; + scripts.push(names); + } + // A script's globals cannot change what the supplied files report unless + // one of them is also a global the supplied files declare. + const collides = (globals: ReadonlySet): boolean => + scripts.some((names) => [...names].some((name) => globals.has(name))); + // The options without the configuration file's path: it names where the + // project is, which checking the supplied files does not read (the files a + // project's location brings in are part of the key). + const options = { ...program.getCompilerOptions(), configFilePath: undefined }; + const key = entryKey(checker, options, libs); + if (key === null) return null; + const stored = readEntry(store, key, libs); + if (stored !== null && !collides(stored.globals)) { + const checked = program.checkedFileDiagnosticData(own); + if (checked === null) return null; + const byFile = new Map(stored.files); + own.forEach((name, index) => byFile.set(name, checked.files[index] ?? [])); + return { semantic: inFileOrder(byFile), global: checked.global }; + } + // The full check, as before. Its answers for the supplied files are stored + // when it reports no global diagnostics and the program's scripts declare + // none of the supplied files' globals (so nothing of the program reached + // what those files report). + const global = program.diagnosticData("getGlobalDiagnostics"); + const semantic = program.diagnosticData("getSemanticDiagnostics"); + // A server that cannot check only some files could never use the answer. + if (global.length === 0 && stored === null && program.checkedFileDiagnosticData([]) !== null) { + const globals = suppliedGlobalNames(libs, sourceFile); + if (globals !== null && !collides(globals)) { + const files: Record = {}; + for (const name of libs) files[name] = program.diagnosticData("getSemanticDiagnostics", name); + try { + store.write(key, JSON.stringify({ schema: SCHEMA, globals: [...globals], files })); + } catch { + // Best effort: a failed write is a later miss. + } + } + } + return { semantic, global }; +} + +/** Whether a file cannot add to or change the global declarations the + * supplied files declare: a module with no `declare global`, no module + * augmentation and no UMD `export as namespace` declaration. */ +function declaresNothingGlobal(file: SourceFile): boolean { + if (file.externalModuleIndicator === undefined) return false; + if (file.moduleAugmentations.length > 0) return false; + for (const statement of file.statements.nodes) { + if (statement.kind === SyntaxKind.NamespaceExportDeclaration) return false; + // `declare global` (a module declaration named by the identifier `global`). + if ( + statement.kind === SyntaxKind.ModuleDeclaration && + statement.name?.kind === SyntaxKind.Identifier && + statement.name.text === "global" + ) + return false; + } + return true; +} + +function isJsFileName(name: string): boolean { + return /\.[cm]?jsx?$/.test(name); +} + +/** The names a script file declares in the global scope: its top-level + * declarations, and the `var` and function declarations of its top-level + * blocks; null when it declares an ambient module (`declare module "x"`), + * which can merge with one the supplied files declare. Over-approximates: + * a name more only makes the stored answer apply less often. */ +function scriptGlobalNames(file: SourceFile): Set | null { + const names = new Set(); + let ambientModule = false; + walkPreorder(file, (node) => { + if (node === file) return undefined; + if (node.kind === SyntaxKind.ModuleDeclaration && node.name?.kind === SyntaxKind.StringLiteral) + ambientModule = true; + const topLevel = node.parent === file; + if (topLevel || node.kind === SyntaxKind.FunctionDeclaration) addDeclaredNames(node, names); + else if ( + node.kind === SyntaxKind.VariableDeclarationList && + (node.flags & NodeFlags.BlockScoped) === 0 + ) + for (const declaration of node.declarations?.nodes ?? []) + addBindingNames(declaration.name, names); + // Functions, classes and namespaces have scopes of their own. + return GLOBAL_SCOPE_BOUNDARIES.has(node.kind) ? "skip" : undefined; + }); + return ambientModule ? null : names; +} + +/** The global names the supplied files declare: the declarations of their + * scripts (as scriptGlobalNames finds them, with ambient modules as their + * quoted names) and those of the `declare global` blocks of their modules; + * null when one of them is not available. */ +function suppliedGlobalNames( + libs: readonly string[], + sourceFile: (fileName: string) => SourceFile | undefined, +): Set | null { + const names = new Set(); + for (const name of libs) { + const file = sourceFile(name); + if (file === undefined) return null; + if (file.externalModuleIndicator === undefined) { + walkPreorder(file, (node) => { + if (node === file) return undefined; + if ( + node.kind === SyntaxKind.ModuleDeclaration && + node.name?.kind === SyntaxKind.StringLiteral + ) + names.add(JSON.stringify(node.name.text ?? "")); + if (node.parent === file || node.kind === SyntaxKind.FunctionDeclaration) + addDeclaredNames(node, names); + else if ( + node.kind === SyntaxKind.VariableDeclarationList && + (node.flags & NodeFlags.BlockScoped) === 0 + ) + for (const declaration of node.declarations?.nodes ?? []) + addBindingNames(declaration.name, names); + return GLOBAL_SCOPE_BOUNDARIES.has(node.kind) ? "skip" : undefined; + }); + continue; + } + for (const statement of file.statements.nodes) { + if ( + statement.kind !== SyntaxKind.ModuleDeclaration || + statement.name?.kind !== SyntaxKind.Identifier || + statement.name.text !== "global" + ) + continue; + for (const inner of statement.body?.statements?.nodes ?? []) addDeclaredNames(inner, names); + } + } + return names; +} + +const GLOBAL_SCOPE_BOUNDARIES = new Set([ + SyntaxKind.FunctionDeclaration, + SyntaxKind.FunctionExpression, + SyntaxKind.ArrowFunction, + SyntaxKind.MethodDeclaration, + SyntaxKind.Constructor, + SyntaxKind.GetAccessor, + SyntaxKind.SetAccessor, + SyntaxKind.ClassDeclaration, + SyntaxKind.ClassExpression, + SyntaxKind.ModuleDeclaration, +]); + +/** The names a statement declares in the scope it is in. */ +function addDeclaredNames(node: Node, names: Set): void { + switch (node.kind) { + case SyntaxKind.VariableStatement: + for (const declaration of node.declarationList?.declarations?.nodes ?? []) + addBindingNames(declaration.name, names); + return; + case SyntaxKind.FunctionDeclaration: + case SyntaxKind.ClassDeclaration: + case SyntaxKind.InterfaceDeclaration: + case SyntaxKind.TypeAliasDeclaration: + case SyntaxKind.EnumDeclaration: + case SyntaxKind.ModuleDeclaration: + case SyntaxKind.ImportEqualsDeclaration: + if (node.name?.kind === SyntaxKind.Identifier) names.add(node.name.text ?? ""); + return; + } +} + +function addBindingNames(name: Node | undefined, names: Set): void { + if (name === undefined) return; + if (name.kind === SyntaxKind.Identifier) { + names.add(name.text ?? ""); + return; + } + // Object and array binding patterns. + for (const element of name.elements?.nodes ?? []) addBindingNames(element.name, names); +} + +/** The stored diagnostics' key: the checker, the compiler options and the + * supplied files in program order with each one's file identity; null when + * one of them cannot be read. */ +function entryKey(checker: string, options: unknown, libs: readonly string[]): string | null { + const files: unknown[] = []; + for (const name of libs) { + try { + const info = statSync(name); + files.push([name, info.size, info.mtimeMs, info.ino]); + } catch { + return null; + } + } + return createHash("sha256") + .update(JSON.stringify({ schema: SCHEMA, checker, options, files })) + .digest("hex"); +} + +function readEntry( + store: LibDiagnosticsStore, + key: string, + libs: readonly string[], +): { files: Map; globals: Set } | null { + let text: string | null; + try { + text = store.read(key); + } catch { + return null; + } + if (text === null) return null; + let entry: { schema?: unknown; globals?: unknown; files?: Record }; + try { + entry = JSON.parse(text) as typeof entry; + } catch { + return null; + } + if (entry.schema !== SCHEMA || entry.files === undefined || !Array.isArray(entry.globals)) + return null; + const files = new Map(); + for (const name of libs) { + const list = entry.files[name]; + if (!Array.isArray(list)) return null; + files.set(name, list); + } + return { files, globals: new Set(entry.globals.filter((name) => typeof name === "string")) }; +} + +/** Each file's diagnostics (already in the checker's order) in the order the + * checker sorts files: by file name, compared by code point. */ +function inFileOrder(byFile: ReadonlyMap): Ts7DiagnosticData[] { + const names = [...byFile.keys()].sort(compareCodePoints); + const out: Ts7DiagnosticData[] = []; + for (const name of names) for (const d of byFile.get(name)!) out.push(d); + return out; +} + +function compareCodePoints(a: string, b: string): number { + let i = 0; + let j = 0; + while (i < a.length && j < b.length) { + const x = a.codePointAt(i)!; + const y = b.codePointAt(j)!; + if (x !== y) return x < y ? -1 : 1; + i += x > 0xffff ? 2 : 1; + j += y > 0xffff ? 2 : 1; + } + return i < a.length ? 1 : j < b.length ? -1 : 0; +} diff --git a/packages/compiler/src/frontend/ts7/program-host.ts b/packages/compiler/src/frontend/ts7/program-host.ts index 639086e94..ec143e401 100644 --- a/packages/compiler/src/frontend/ts7/program-host.ts +++ b/packages/compiler/src/frontend/ts7/program-host.ts @@ -46,6 +46,10 @@ import { trackedRealpath, } from "../input-tracker.js"; import { packageShimDeclarationText } from "../package-shims.js"; +import { ambientDtsPath, fallbackDtsPath, overridesDtsPath } from "../dts-paths.js"; +import { reusedLibDiagnostics } from "./lib-diagnostics.js"; +import { ts7Diagnostic } from "./diagnostic.js"; +import type { Ts7DiagnosticData } from "./session-schema.generated.js"; /** The compiler options our createProgram accepts: TS7's CompilerOptions * shape (numeric enums for target/module/moduleResolution — the enums module @@ -393,6 +397,22 @@ export class Ts7Program { return this.project.program.getSourceFileMetadata(resolve(fileName)); } + /** The semantic and global diagnostics with the stored diagnostics of the + * declaration files scriptc supplies (lib-diagnostics.ts), or null. */ + reusedLibDiagnostics(): { semantic: Ts7DiagnosticData[]; global: Ts7DiagnosticData[] } | null { + this.snapshot.ensureActive(); + const program = this.project.program; + const declarations = new Set([ambientDtsPath(), overridesDtsPath(), fallbackDtsPath()]); + return reusedLibDiagnostics( + program, + this.getSourceFileNames(), + (name) => + declarations.has(tsgoPath(name)) || + program.getSourceFileMetadataByPath(name)?.isDefaultLibrary === true, + (name) => this.getSourceFile(name), + ); + } + isSourceFileDefaultLibrary(file: SourceFile): boolean { return this.project.program.isSourceFileDefaultLibrary(file); } @@ -423,9 +443,20 @@ export class Ts7Program { * package dropped the aggregate). Order mirrors 5.9.3: config, options/ * program, global, syntactic, semantic. */ export function getPreEmitDiagnostics(program: Ts7Program): readonly Diagnostic[] { + const config = program.getConfigFileParsingDiagnostics(); + const options = program.getProgramDiagnostics(); + const reused = program.reusedLibDiagnostics(); + if (reused !== null) + return [ + ...config, + ...options, + ...reused.global.map(ts7Diagnostic), + ...program.getSyntacticDiagnostics(), + ...reused.semantic.map(ts7Diagnostic), + ]; return [ - ...program.getConfigFileParsingDiagnostics(), - ...program.getProgramDiagnostics(), + ...config, + ...options, ...program.getGlobalDiagnostics(), ...program.getSyntacticDiagnostics(), ...program.getSemanticDiagnostics(), diff --git a/packages/compiler/src/frontend/ts7/session.ts b/packages/compiler/src/frontend/ts7/session.ts index d3f2ae9fe..082bb79a6 100644 --- a/packages/compiler/src/frontend/ts7/session.ts +++ b/packages/compiler/src/frontend/ts7/session.ts @@ -55,6 +55,13 @@ interface ProgramQuery { snapshot?: number; project?: string; file?: SemanticDocument; + files?: string[]; +} + +/** The answer of getDiagnosticsOfCheckedFiles, as the server sends it. */ +export interface Ts7CheckedFileDiagnosticData { + files: Ts7DiagnosticData[][]; + global: Ts7DiagnosticData[]; } /** Platform-independent session over an already connected transport. The @@ -344,6 +351,29 @@ export class Ts7SessionProgram { result.push(ts7Diagnostic(data)); return result; } + /** The diagnostics of `method` for `file`, or for the whole program, as + * the server sends them. */ + diagnosticData(method: string, file?: SemanticDocument): Ts7DiagnosticData[] { + return this.session.request(method, this.query(file)) ?? []; + } + /** The semantic diagnostics of each of `files` and the global diagnostics, + * after the server checks only those files (getDiagnosticsOfCheckedFiles), + * or null when the server does not answer that method. */ + checkedFileDiagnosticData(files: readonly string[]): Ts7CheckedFileDiagnosticData | null { + try { + return this.session.request("getDiagnosticsOfCheckedFiles", { + ...this.query(), + files: [...files], + }); + } catch (error) { + if ( + error instanceof Error && + error.message.includes('unknown API method "getDiagnosticsOfCheckedFiles"') + ) + return null; + throw error; + } + } getSyntacticDiagnostics(file?: SemanticDocument): Ts7Diagnostic[] { return this.diagnostics("getSyntacticDiagnostics", file); } diff --git a/packages/compiler/src/frontend/type-mapper.ts b/packages/compiler/src/frontend/type-mapper.ts index 24a684078..e26a4aa19 100644 --- a/packages/compiler/src/frontend/type-mapper.ts +++ b/packages/compiler/src/frontend/type-mapper.ts @@ -213,6 +213,9 @@ export const FETCH_AMBIENT_TYPES = [ "ReadableStreamReadResult", "ReadableStreamReadValueResult", "ReadableStreamReadDoneResult", + "TransformStream", + "WritableStream", + "WritableStreamDefaultWriter", "Blob", "File", "BlobPropertyBag", @@ -3109,7 +3112,7 @@ function mapTypeInner(type: ts.Type, ctx: TypeMapperCtx): IrType | null { psym && (FETCH_AMBIENT_TYPES as readonly string[]).includes(ts.symbolName(psym)) && !( - ts.symbolName(psym) === "ReadableStream" && + (ts.symbolName(psym) === "ReadableStream" || ts.symbolName(psym) === "WritableStream") && psym.declarations.some( (d) => ts.isInterfaceDeclaration(d) && isDeclaredInAmbientNamespace(d, "NodeJS"), ) @@ -4128,6 +4131,17 @@ function mapTypeInner(type: ts.Type, ctx: TypeMapperCtx): IrType | null { fields.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)); return { kind: "record", shapeId: ctx.shapes.intern(fields, false, undefined, order) }; } + // crypto.getCipherInfo's result is Node's info object itself, read as a + // checked-dynamic value: Node leaves out the fields a cipher lacks, + // including the nid the declaration requires, so a record cannot hold + // every result. + if ( + isStdlibInterface("CipherInfo") && + psym!.declarations.some( + (d) => ts.isInterfaceDeclaration(d) && isDeclaredInAmbientModule(d, "crypto"), + ) + ) + return DYN; // Filesystem options are data records. Keep their declared optional fields // instead of allowing arbitrary standard-library interfaces into records. if ( diff --git a/packages/compiler/src/index.ts b/packages/compiler/src/index.ts index 4f7b93cfd..58dc39c7b 100644 --- a/packages/compiler/src/index.ts +++ b/packages/compiler/src/index.ts @@ -9,6 +9,7 @@ import { } from "./library/prepare.js"; import { analyzeWithFrontend } from "./frontend/analysis.js"; import { checkerCacheIdentity } from "./frontend/ts7/checker-backend.js"; +import { setLibDiagnosticsStore } from "./frontend/ts7/lib-diagnostics.js"; import { analyzeWithVerdictCache, fileVerdictStore } from "./coverage/verdict-cache.js"; import { coverageCompilerIdentity } from "./coverage/verdict-startup.js"; import { commentOnlyReplay } from "./coverage/verdict-semantic.js"; @@ -450,6 +451,7 @@ export async function compile( ); return withCompileWarnings(result, warnings); } finally { + setLibDiagnosticsStore(null); frontendProcess?.release(); } } @@ -623,6 +625,7 @@ async function compileExecutableNative( dgram: features.dgram, dnsPromises: features.dnsPromises, dgramDyn: features.dgramDyn, + textCodecDyn: features.textCodecDyn, watch: features.watch, foreignFfi: features.foreignFfi, ...(features.workers ? { workers: true } : {}), @@ -1161,10 +1164,36 @@ async function prepareExecutableInputInChild( * `requestPath` as `compileTracked` would in-process, and writes what it prepared, without the LLVM * text, to `resultPath`. */ +/** Has the build reuse the diagnostics of the declaration files scriptc + * supplies (lib-diagnostics.ts) through the build cache at `root`, when the + * build uses one (executables; source outputs create no cache) and the + * checker is tsc-ts. */ +function configureLibDiagnostics(root: string | null): void { + let checker: string | undefined; + try { + checker = root === null ? undefined : checkerCacheIdentity(); + } catch { + // A missing checker fails where the build starts it. + checker = undefined; + } + setLibDiagnosticsStore( + root === null || checker === undefined + ? null + : fileVerdictStore(join(root, "lib-diagnostics-v1")), + checker ?? "", + ); +} + export async function runFrontendProcess(requestPath: string, resultPath: string): Promise { const request = JSON.parse(await readFile(requestPath, "utf8")) as FrontendProcessRequest; clearCompileSessionCaches(); setProvenanceSources(request.provenance); + // The compile that started this process prepared the cache root. + configureLibDiagnostics( + (request.opts.outputKind ?? "exe") === "exe" && provenanceSources() === null + ? await prepareBuildCacheRoot(buildCacheRoot()) + : null, + ); const frontendInputs = new FrontendInputTracker(); const warnings: CompileWarningSink = { warnings: [], sourceTexts: null }; let result: FrontendProcessResult = { kind: "failed" }; @@ -1212,6 +1241,8 @@ export async function runFrontendProcess(requestPath: string, resultPath: string } } catch { result = { kind: "failed" }; + } finally { + setLibDiagnosticsStore(null); } await writeFile(resultPath, JSON.stringify(result), { mode: 0o600 }); } @@ -1449,6 +1480,7 @@ async function compileTracked( outputKind === "exe" && provenanceSources() === null ? await prepareBuildCacheRoot(buildCacheRoot()) : null; + configureLibDiagnostics(cacheRoot); let earlyCacheOptions: EarlyExecutableCacheOptions | null = null; if (outputKind === "exe") { const helperObjectRoute = diff --git a/packages/compiler/src/ir/analysis.ts b/packages/compiler/src/ir/analysis.ts index 70ec381a1..ee1cee6be 100644 --- a/packages/compiler/src/ir/analysis.ts +++ b/packages/compiler/src/ir/analysis.ts @@ -4,6 +4,7 @@ import { BYTES_ELEMENT_NAME, DYN_HANDLE_KINDS, RUNTIME_STREAM_CLASSES, + recordTextCodecClass, typeEquals, type IrExpr, type IrLibFn, @@ -141,8 +142,10 @@ export function dynDesc( return "string"; case "bool": return "boolean"; - case "record": - return recordsById.get(t.shapeId)?.tuple ? "array" : "object"; + case "record": { + const shape = recordsById.get(t.shapeId); + return shape?.tuple ? "array" : ((shape && recordTextCodecClass(shape)) ?? "object"); + } case "array": return "array"; case "nullT": diff --git a/packages/compiler/src/ir/array-length-stability.test.ts b/packages/compiler/src/ir/array-length-stability.test.ts index f9b3cc9c2..97270e059 100644 --- a/packages/compiler/src/ir/array-length-stability.test.ts +++ b/packages/compiler/src/ir/array-length-stability.test.ts @@ -193,3 +193,34 @@ test("number and boolean writes to class fields run no user code", () => { // A reference stored into a field releases the old one. expect(preserves([fieldSet(arr("b"))], "i", new Set(), scope)).toBe(false); }); + +test("optional element reads, their assertion errors and present-undefined stores at an in-range index keep the length", () => { + const optional = (receiver: string, index: IrExpr): IrExpr => ({ + kind: "call", + callee: "%arr.idxOr.f64", + args: [arr(receiver), index], + type: F64, + loc, + }); + const undefinedAt = (receiver: string, index: IrExpr): IrStmt => ({ + kind: "arraySetUndefined", + arr: arr(receiver), + index, + loc, + }); + const thrown: IrExpr = { kind: "libCall", fn: "error.nodeThrow", args: [num(1)], type: F64, loc }; + expect(preserves([stmt(optional("a", ref("i"))), stmt(thrown)])).toBe(true); + expect(preserves([undefinedAt("a", ref("i"))])).toBe(true); + // Past the counter, a present undefined extends the array. + expect(preserves([undefinedAt("a", plus(ref("i"), num(1)))])).toBe(false); + // The high counter of a two-counter loop is in range as well. + const paired = (body: IrStmt[]): boolean => + preservesArrayLength( + [cond, ...body], + { receiver: "a", inRange: "i", inRangeHigh: "j" }, + locals, + new Set(), + ); + expect(paired([undefinedAt("a", ref("j")), set("a", ref("j"), num(0))])).toBe(true); + expect(preserves([set("a", ref("j"), num(0))])).toBe(false); +}); diff --git a/packages/compiler/src/ir/array-length-stability.ts b/packages/compiler/src/ir/array-length-stability.ts index ed57972b6..7715ea1e1 100644 --- a/packages/compiler/src/ir/array-length-stability.ts +++ b/packages/compiler/src/ir/array-length-stability.ts @@ -162,6 +162,10 @@ export interface ArrayLengthLoop { * long as the length itself has not changed). An element write at this * index replaces an existing element or sets a non-index property. */ inRange: string | null; + /** A second such binding: the high counter of a two-counter loop + * (`for (let i = lo, j = hi; i < j; i++, j--)`), whose values stay + * between the low counter's start and its own. */ + inRangeHigh?: string | null; } /** Whether running `nodes` (a loop's condition, body and update) can never @@ -191,7 +195,10 @@ export function preservesArrayLength( // escaping. const receiver = (e: IrExpr): boolean => (e.kind === "varRef" && e.localId === loop.receiver) || expr(e); - const written = (id: string): boolean => id === loop.receiver || id === loop.inRange; + const inRange = (id: string): boolean => + (loop.inRange !== null && id === loop.inRange) || + (loop.inRangeHigh != null && id === loop.inRangeHigh); + const written = (id: string): boolean => id === loop.receiver || inRange(id); const expr = (e: IrExpr): boolean => { switch (e.kind) { case "varRef": @@ -224,12 +231,21 @@ export function preservesArrayLength( case "call": // A checked union extraction (`x!`, coercions/unions.ts): the // asserted arm's payload or a TypeError, and no user code. - return e.callee.startsWith("%union.narrow.") && e.args.every(expr); + if (e.callee.startsWith("%union.narrow.")) return e.args.every(expr); + // An element read as `T | undefined` (int32-arrays.ts isElementReader). + return ( + e.callee.startsWith("%arr.idxOr.") && + e.args.length === 2 && + receiver(e.args[0]!) && + expr(e.args[1]!) + ); case "mapIntrinsic": return READING_MAP_METHODS.has(e.method) && expr(e.receiver) && e.args.every(expr); case "setIntrinsic": return READING_SET_METHODS.has(e.method) && expr(e.receiver) && e.args.every(expr); case "libCall": + // The TypeError of an assertion past a union arm leaves the loop. + if (e.fn === "error.nodeThrow") return e.args.every(expr); return ( isBorrowSafeMath(e.fn) && e.args.every((arg) => arg.type.kind === "f64") && @@ -254,14 +270,18 @@ export function preservesArrayLength( break; case "arraySet": return ( - (separate(s.arr) || - (s.index.kind === "varRef" && - loop.inRange !== null && - s.index.localId === loop.inRange)) && + (separate(s.arr) || (s.index.kind === "varRef" && inRange(s.index.localId))) && receiver(s.arr) && expr(s.index) && expr(s.value) ); + case "arraySetUndefined": + // A present undefined replacing an element below the length. + return ( + (separate(s.arr) || (s.index.kind === "varRef" && inRange(s.index.localId))) && + receiver(s.arr) && + expr(s.index) + ); case "return": return s.value === null || expr(s.value); case "fieldSet": diff --git a/packages/compiler/src/ir/builtin-effects.ts b/packages/compiler/src/ir/builtin-effects.ts index 0239c1624..e671c3da7 100644 --- a/packages/compiler/src/ir/builtin-effects.ts +++ b/packages/compiler/src/ir/builtin-effects.ts @@ -16,6 +16,8 @@ export const MAY_THROW_LIB_FNS: ReadonlySet = new Set([ // Node's prime, generator, size, encoding and group-name errors. "crypto.dhNew", "crypto.dhGroupNew", + // Node's nameOrNid and options errors. + "crypto.getCipherInfo", // Node's candidate, size, options, add and rem errors, and OpenSSL's // generation errors. "crypto.checkPrimeSync", @@ -362,6 +364,7 @@ export const MAY_THROW_LIB_FNS: ReadonlySet = new Set([ "transform.newDyn", "passthrough.newDyn", "readable.initDyn", + "stream.checkEncodingOption", "writable.initDyn", "duplex.initDyn", "transform.initDyn", @@ -691,6 +694,7 @@ export const MAY_THROW_LIB_FNS: ReadonlySet = new Set([ "class.defineField", "class.throwReadOnly", "class.throwNotExtensible", + "record.throwFrozen", "dyn.toString", "dyn.valueOf", "string.fromCodePoint", diff --git a/packages/compiler/src/ir/builtin-signatures.ts b/packages/compiler/src/ir/builtin-signatures.ts index 9ea65d47d..8a78eb6c9 100644 --- a/packages/compiler/src/ir/builtin-signatures.ts +++ b/packages/compiler/src/ir/builtin-signatures.ts @@ -66,6 +66,7 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ "fetch.start": { argTypes: [STRING, DYN], result: { kind: "promise", inner: DYN } }, "fetch.responseNew": { argTypes: [DYN, DYN], result: DYN }, "fetch.responseJsonStatic": { argTypes: [DYN, DYN], result: DYN }, + "fetch.responseLight": { argTypes: [DYN], result: arrayOf(STRING) }, "fetch.input": { argTypes: [DYN, DYN], result: { kind: "promise", inner: DYN } }, "abort.controllerNew": { argTypes: [], result: DYN }, /** EventTarget / Event / CustomEvent construction (scr_event_target.c): @@ -162,6 +163,7 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ "fetch.abortNow": { argTypes: [DYN], result: DYN }, "fetch.abortAny": { argTypes: [DYN], result: DYN }, "fetch.streamNew": { argTypes: [DYN], result: DYN }, + "fetch.transformNew": { argTypes: [], result: DYN }, // Program-dependent iterable: typed arrays/bytes/string stay intact so // the native stream can pull lazily; checked-dynamic values are the // fallback. The libCall validator in validate.ts checks the closed set. @@ -271,6 +273,28 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ /** Throws Node's TypeError for a class field initializer on a * non-extensible instance: the field name. */ "class.throwNotExtensible": { argTypes: [STRING], result: VOID }, + /** Object.freeze of a record whose shape is freezable (frozen-records.ts): + * sets the frozen bit in its rc word (scr_record_freeze). The receiver is + * any record type. */ + "record.freeze": { argTypes: [null], result: VOID }, + /** Whether a record is frozen: its rc word's frozen bit, set by + * record.freeze and dyn.freeze of a capsule. Emitted inline. The receiver + * is any record type. */ + "record.isFrozen": { argTypes: [null], result: BOOL }, + /** Throws Node's TypeError for a write to a frozen record: the key, the + * object's text in the message ("#") and the operation (0 assign, + * 1 add, 2 delete). */ + "record.throwFrozen": { argTypes: [STRING, STRING, F64], result: VOID }, + /** Object.freeze of an array (scr_arr_freeze): sets its frozen bit, which + * the write guards of IrModule.freezableArrays test, and isFrozen reads. + * The receiver is any array type. */ + "array.freeze": { argTypes: [null], result: VOID }, + "array.isFrozen": { argTypes: [null], result: BOOL }, + /** Object.freeze, seal or preventExtensions of a Map or Set + * (scr_map_prevent_extensions), which only makes it not extensible: its + * entries stay writable. The receiver is any map or set type. */ + "collection.preventExtensions": { argTypes: [null], result: VOID }, + "collection.isExtensible": { argTypes: [null], result: BOOL }, "dyn.nativeSetNew": { argTypes: [DYN], result: DYN }, "dyn.nativeSetIs": { argTypes: [DYN], result: BOOL }, "dyn.nativeMapIs": { argTypes: [DYN], result: BOOL }, @@ -1835,7 +1859,7 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ * connection (+1, the same handle net.connect would give); * sockRemoteAddress answers `string | undefined` (NULL after the fd * closed, Node's destroyed-socket undefined; a dual-stack accept of an - * IPv4 peer reads "::ffff:a.b.c.d" like Node). reqResume/reqDestroy and + * IPv4 peer reads "::ffff:a.b.c.d" like Node). reqResume/reqPause/reqDestroy and * the req error/close listener slots complete the IncomingMessage * surface portless's client responses use; resDestroy/resOnClose and * resWriteHeadPairs ([k0,v0,k1,v1,...] — the env.pairs helper's flat @@ -1902,6 +1926,7 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ "http.reqPipeClient": { argTypes: [HTTPREQ_T, HTTPCLIENTREQ_T], result: VOID }, "http.reqPipeSock": { argTypes: [HTTPREQ_T, NETSOCKET_T], result: VOID }, "http.reqResume": { argTypes: [HTTPREQ_T], result: VOID }, + "http.reqPause": { argTypes: [HTTPREQ_T], result: VOID }, "http.reqDestroy": { argTypes: [HTTPREQ_T], result: VOID }, "http.reqOnError": { argTypes: [HTTPREQ_T, null, BOOL], result: VOID }, "http.reqOnClose": { @@ -2844,6 +2869,9 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ * undefined, and call it in a later macrotask; Node's argument errors * throw (may-throw). */ "crypto.checkPrimeSync": { argTypes: [DYN, DYN], result: DYN }, + /** getCipherInfo(nameOrNid, options) over checked-dynamic arguments + * (scr_cipher_info.c): the info object or undefined. */ + "crypto.getCipherInfo": { argTypes: [DYN, DYN], result: DYN }, "crypto.checkPrimeCb": { argTypes: [DYN, DYN, DYN, BOOL], result: VOID }, "crypto.generatePrimeSync": { argTypes: [DYN, DYN], result: DYN }, "crypto.generatePrimeCb": { argTypes: [DYN, DYN, DYN, BOOL], result: VOID }, @@ -3990,6 +4018,7 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ "stream.setTransform": { argTypes: [null, null], result: VOID }, "stream.setFlush": { argTypes: [null, null], result: VOID }, "stream.setConstruct": { argTypes: [null, null], result: VOID }, + "stream.replaceConstruct": { argTypes: [null, null], result: VOID }, "stream.setWritev": { argTypes: [null, null], result: VOID }, "stream.setWritevDefault": { argTypes: [null, null], result: VOID }, /** The dyn-options twins (a checked-dynamic options record — the JS @@ -4126,6 +4155,11 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ * after construction; answers the receiver (+1, the chaining shape). */ "stream.objectMode": { argTypes: [null, BOOL, BOOL], result: VOID }, "stream.decodeStrings": { argTypes: [null, BOOL], result: VOID }, + /** Runtime decodeStrings and defaultEncoding option values, applied + * after construction (answers the receiver +1); the encoding was + * validated by stream.checkEncodingOption before construction. */ + "stream.optionValues": { argTypes: [null, DYN, DYN], result: VOID }, + "stream.checkEncodingOption": { argTypes: [DYN], result: VOID }, "stream.duplexFromGen": { argTypes: [null, null, null, null, null, null], result: VOID }, "stream.duplexFromDyn": { argTypes: [DYN], result: VOID }, /** Duplex.from over a lone Readable or Writable, or a { readable, @@ -4846,6 +4880,10 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ "text.decoderEncoding": { argTypes: [DYN], result: F64 }, "text.decoderName": { argTypes: [F64], result: STRING }, "text.encodeInto": { argTypes: [DYN, DYN], result: DYN }, + /** `value instanceof TextDecoder` (true) or `TextEncoder` (false) on a + * checked-dynamic value: a codec record's handle box (scr_text_codec_dyn.c). + * Borrowed; never throws. */ + "text.codecIs": { argTypes: [DYN, BOOL], result: BOOL }, "buffer.isAscii": { argTypes: [DYN], result: BOOL }, "buffer.isUtf8": { argTypes: [DYN], result: BOOL }, "buffer.transcode": { argTypes: [DYN, DYN, DYN], result: BYTES_U8 }, diff --git a/packages/compiler/src/ir/callback-specialization.ts b/packages/compiler/src/ir/callback-specialization.ts index 00432621d..6038ccaa5 100644 --- a/packages/compiler/src/ir/callback-specialization.ts +++ b/packages/compiler/src/ir/callback-specialization.ts @@ -2,6 +2,7 @@ import type { IrContracts, IrExpr, IrFunction, IrModule, IrStmt } from "./ir.js" import { exprFunctionReference, moduleFunctionReferences } from "./function-references.js"; import { reparseExpr, reparseStmt } from "./serialize.js"; import { everyStmtList, mapExprChildren, mapStmtChildren } from "./traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "./node-index.js"; /** Largest body (IR nodes) that is copied per callback target. */ const MAX_FUNCTION_NODES = 300; @@ -15,11 +16,24 @@ const COPIED_NODE_FRACTION = 0.25; * closure it creates writes through its environment (and transitively the * closures those create). A parameter or capture absent from its function's * set holds the value it entered with for the whole call. */ -export function writtenLocals(mod: IrModule): Map> { - return rebindings(mod).written; +export function writtenLocals(mod: IrModule, nodes?: IrNodeIndex): Map> { + return rebindings(mod, nodes).written; } -function rebindings(mod: IrModule): { +/** The nodes that rebind a local or capture bindings. */ +const REBINDING_KINDS: readonly IrNodeKind[] = [ + "assign", + "forOf", + "tryCatch", + "assignExpr", + "incDec", + "closure", +]; + +function rebindings( + mod: IrModule, + nodes?: IrNodeIndex, +): { written: Map>; /** Bindings of each function that a closure it creates writes. */ closureWritten: Map>; @@ -27,20 +41,15 @@ function rebindings(mod: IrModule): { const written = new Map>(); const closureWritten = new Map>(); const sites: { owner: string; fnName: string; captures: string[] }[] = []; + const scan = nodes?.scan(REBINDING_KINDS); for (const fn of mod.functions) { const ids = new Set(); - everyStmtList(fn.body, { - stmt: (stmt) => { - if (stmt.kind === "assign" || stmt.kind === "forOf") ids.add(stmt.localId); - if (stmt.kind === "tryCatch" && stmt.catchLocalId !== null) ids.add(stmt.catchLocalId); - return true; - }, - expr: (expr) => { - if (expr.kind === "assignExpr" || expr.kind === "incDec") ids.add(expr.localId); - if (expr.kind === "closure" && expr.captures.length) - sites.push({ owner: fn.name, fnName: expr.fnName, captures: expr.captures }); - return true; - }, + eachNodeOfKinds(fn, scan, (node) => { + if (node.kind === "assign" || node.kind === "forOf") ids.add(node.localId); + else if (node.kind === "tryCatch" && node.catchLocalId !== null) ids.add(node.catchLocalId); + else if (node.kind === "assignExpr" || node.kind === "incDec") ids.add(node.localId); + else if (node.kind === "closure" && node.captures.length) + sites.push({ owner: fn.name, fnName: node.fnName, captures: node.captures }); }); written.set(fn.name, ids); } diff --git a/packages/compiler/src/ir/final-fields.ts b/packages/compiler/src/ir/final-fields.ts index b0340cbaa..991339c6c 100644 --- a/packages/compiler/src/ir/final-fields.ts +++ b/packages/compiler/src/ir/final-fields.ts @@ -1,6 +1,7 @@ import { isRefCounted, type IrClassDef, type IrExpr, type IrModule, type IrStmt } from "./ir.js"; import { dynamicFieldStores, type DynamicFieldStores } from "./int32-slots.js"; import { everyExprChild, everyStmtChild } from "./traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "./node-index.js"; /** Whole-program final reference fields. * @@ -75,6 +76,9 @@ export function constructorClass(fnName: string): string | null { : null; } +/** The nodes that write a class field. */ +const FIELD_WRITE_KINDS: readonly IrNodeKind[] = ["fieldSet", "fieldIncDec"]; + /** `scalars` extends the analysis from reference-counted fields to fields of * every type, for consumers that read a final field's value once instead of * borrowing it (field-values.ts). */ @@ -82,6 +86,7 @@ export function analyzeFinalFields( mod: IrModule, scalars = false, stores?: DynamicFieldStores, + nodes?: IrNodeIndex, ): FinalFields { const classes = new Map((mod.classes ?? []).map((c) => [c.name, c])); const finals = new FinalFields(classes); @@ -121,30 +126,25 @@ export function analyzeFinalFields( } if (finals.size === 0) return finals; + const scan = nodes?.scan(FIELD_WRITE_KINDS); for (const fn of mod.functions) { const ctor = constructorClass(fn.name); const self = ctor !== null ? fn.params[0]?.localId : undefined; const selfLocal = self !== undefined ? fn.locals.find((l) => l.id === self) : undefined; // `this` must name the object under construction for the whole body. const constructing = selfLocal !== undefined && !selfLocal.boxed && !reassigns(fn.body, self!); - const stmt = (s: IrStmt): boolean => { - if (s.kind === "fieldSet") { - const family = finals.family(s.className, s.field); - if (family !== null && finals.isFinal(s.className, s.field)) { + eachNodeOfKinds(fn, scan, (node) => { + if (node.kind === "fieldSet") { + const family = finals.family(node.className, node.field); + if (family !== null && finals.isFinal(node.className, node.field)) { const own = - constructing && ctor !== null && inFamily(ctor, family) && isSelf(s.obj, self!); + constructing && ctor !== null && inFamily(ctor, family) && isSelf(node.obj, self!); if (!own) finals.drop(family, `written in ${fn.name}`); } - } - return everyStmtChild(s, expr, stmt); - }; - // `o.n++` updates a numeric field in place; reference fields never see it. - const expr = (e: IrExpr): boolean => { - if (e.kind === "fieldIncDec" && finals.isFinal(e.className, e.field)) - finals.drop(finals.family(e.className, e.field)!, `updated in ${fn.name}`); - return everyExprChild(e, expr, stmt); - }; - for (const s of fn.body) stmt(s); + } else if (node.kind === "fieldIncDec" && finals.isFinal(node.className, node.field)) + // `o.n++` updates a numeric field in place; reference fields never see it. + finals.drop(finals.family(node.className, node.field)!, `updated in ${fn.name}`); + }); } return finals; } diff --git a/packages/compiler/src/ir/function-references.test.ts b/packages/compiler/src/ir/function-references.test.ts index 26539b17a..91e1d5f05 100644 --- a/packages/compiler/src/ir/function-references.test.ts +++ b/packages/compiler/src/ir/function-references.test.ts @@ -219,6 +219,7 @@ const MODULE_FIELDS: Record>, Role> = { "wasmUntranslated[].sha256": "other", "wasmUntranslated[].reason": "other", "threadSlotKeys[]": "other", + "freezableArrays[]": "other", }; const loc = { file: "references.ts", start: 0, end: 0 }; diff --git a/packages/compiler/src/ir/function-references.ts b/packages/compiler/src/ir/function-references.ts index 0342172b6..db474e9bb 100644 --- a/packages/compiler/src/ir/function-references.ts +++ b/packages/compiler/src/ir/function-references.ts @@ -11,6 +11,9 @@ export function exprFunctionReference(expr: IrExpr): string | undefined { return undefined; } +/** The expression kinds exprFunctionReference answers for. */ +export const FUNCTION_REFERENCE_KINDS = ["closure", "orDefault"] as const; + /** Functions the module's metadata names: the entry, library exports and * the class prototype helpers the runtime calls. Class methods and * constructors follow the `%.` convention instead of being diff --git a/packages/compiler/src/ir/initializer-bindings.ts b/packages/compiler/src/ir/initializer-bindings.ts index cbda4241e..cb2cebc16 100644 --- a/packages/compiler/src/ir/initializer-bindings.ts +++ b/packages/compiler/src/ir/initializer-bindings.ts @@ -1,11 +1,27 @@ import type { IrExpr, IrFunction, IrGlobal, IrModule, IrStmt } from "./ir.js"; -import { everyExprChild, everyStmtChild, everyStmtList } from "./traverse.js"; +import { everyExprChild, everyStmtChild } from "./traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex, type IrNodeKind } from "./node-index.js"; + +/** The nodes findInitializerBindings reads: binding uses and captures. */ +const USE_KINDS: readonly IrNodeKind[] = [ + "varRef", + "assignExpr", + "incDec", + "closure", + "assign", + "varDecl", + "forOf", + "tryCatch", +]; /** Globals private to a guarded, synchronous initializer have local numeric * dataflow: no other function can replace their slots, and reentrant module * evaluation returns before reaching them. Keep the actual global storage * and lifetime; these facts are only inputs to numeric and extent analysis. */ -export function findInitializerBindings(mod: IrModule): ReadonlyMap { +export function findInitializerBindings( + mod: IrModule, + nodes?: IrNodeIndex, +): ReadonlyMap { const result = new Map(); // Embedders can reset library state between entries. Async initialization // and lifted captures also remain outside this synchronous proof. @@ -17,6 +33,7 @@ export function findInitializerBindings(mod: IrModule): ReadonlyMap { - if (e.kind === "varRef") use(e.localId, false); - if (e.kind === "assignExpr" || e.kind === "incDec") use(e.localId, true); - if (e.kind === "closure") for (const id of e.captures) excluded.add(id); - return true; - }, - stmt: (s) => { - if (s.kind === "assign" || s.kind === "varDecl" || s.kind === "forOf") use(s.localId, true); - if (s.kind === "tryCatch" && s.catchLocalId) excluded.add(s.catchLocalId); - return true; - }, + eachNodeOfKinds(fn, scan, (node) => { + if (node.kind === "varRef") use(node.localId, false); + else if (node.kind === "assignExpr" || node.kind === "incDec") use(node.localId, true); + else if (node.kind === "closure") for (const id of node.captures) excluded.add(id); + else if (node.kind === "assign" || node.kind === "varDecl" || node.kind === "forOf") + use(node.localId, true); + else if (node.kind === "tryCatch" && node.catchLocalId) excluded.add(node.catchLocalId); }); } const guarded = new Set(); diff --git a/packages/compiler/src/ir/int32-arrays.ts b/packages/compiler/src/ir/int32-arrays.ts index 02de7268f..353755c05 100644 --- a/packages/compiler/src/ir/int32-arrays.ts +++ b/packages/compiler/src/ir/int32-arrays.ts @@ -1,3 +1,4 @@ +import { exprFunctionReference, moduleFunctionReferences } from "./function-references.js"; import { F64, type IrExpr, type IrFunction, type IrModule, type IrStmt } from "./ir.js"; type SwitchCase = Extract["cases"][number]; @@ -6,22 +7,29 @@ import { everyExprChild, everyStmtChild, everyStmtList, + mapExprChildren, + mapStmtChildren, transformExpr, transformStmt, transformStmtList, type IrTransform, } from "./traverse.js"; -/** Local number arrays the backend may store as int32 slots. +/** Number arrays the backend may store as int32 slots. * - * A candidate is a local `number[]` of a synchronous function that no - * other binding can hold and only compiled element operations reach: - * every write of the local stores a fresh array (an array literal, or - * `new Array(n).fill(v)`), and every read of it is the receiver of an - * element read, write, state test, `length`, `push` or `pop` (a discarded - * `pop` included), or the array of an element-reader helper call. The - * array never reaches a runtime function that reads its slots, a library - * call, a closure, a return or untyped code. + * A candidate is a `number[]` binding that only compiled element + * operations reach: a local of a synchronous function, a module global, or + * a parameter. Every write of a local or a global stores a fresh array (an + * array literal, or `new Array(n).fill(v)`); a parameter is never written. + * Every read of a candidate is the receiver of an element read, write, + * state test, `length`, `push` or `pop` (a discarded `pop` included), the + * array of an element-reader helper call, or the argument of a direct call + * whose parameter at that position is a candidate itself. A parameter is a + * candidate only when its function is reached by direct calls alone and + * every call passes a candidate there, so every binding that can hold one + * of these arrays is a candidate: the array never reaches a runtime + * function that reads its slots, a library call, a closure, a return or + * untyped code. * * Candidates are only proposed here. The int32 slot analysis * (int32-slots.ts) keeps the ones whose every stored element is a proven @@ -33,6 +41,18 @@ import { * calls are inlined first (rewriteCandidates), so no shared helper ever * receives an int32 array. */ +/** The candidate key of the binding `localId` as `fnName` reads it: + * `function\0local` for a local or parameter, `\0global` for a module + * global (`%g.` ids), which every function reads alike. */ +export function int32ArrayKey(fnName: string, localId: string): string { + return localId.startsWith("%g.") ? `\0${localId}` : `${fnName}\0${localId}`; +} + +/** The int32 element slot (int32-slots.ts) of a candidate key. */ +export function int32ArraySlot(key: string): string { + return key.startsWith("\0") ? `g:${key.slice(1)}` : `l:${key}`; +} + /** Array methods a candidate may be the receiver of: the runtime paths * behind them read and write int32 slots through the double-valued slot * accessors. */ @@ -202,10 +222,70 @@ function writesLocal(e: IrExpr, id: string): boolean { return !expr(e); } -/** The candidate locals of `fn` (see the module comment). */ -export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet): Set { +/** Expression kinds an index can be built from without running any code. */ +const PURE_INDEX_KINDS: ReadonlySet = new Set([ + "varRef", + "numLit", + "boolLit", + "bin", + "unary", + "ternary", + "logical", + "arrayGet", + "arrayState", +]); + +/** Whether evaluating an index runs no code (no call, accessor or + * conversion): plain reads, literals and their operators only. */ +function pureIndex(e: IrExpr): boolean { + return PURE_INDEX_KINDS.has(e.kind) && everyExprChild(e, pureIndex, () => false); +} + +/** The parameters and globals a module's functions share as candidates, + * still assumed to qualify. */ +export interface SharedCandidates { + /** Candidate parameters, as `function\0local` keys. */ + params: ReadonlySet; + /** Candidate globals, as `%g.` ids. */ + globals: ReadonlySet; + /** Whether the parameter at `index` of `callee` is a candidate. */ + paramAt: (callee: string, index: number) => boolean; +} + +const NO_SHARED: SharedCandidates = { + params: new Set(), + globals: new Set(), + paramAt: () => false, +}; + +/** The candidates of `fn` (see the module comment): its own locals, its + * parameters in `shared.params`, and the globals of `shared.globals` that + * it reads only in candidate positions. `written` receives the globals it + * assigns fresh arrays. */ +export function int32ArrayCandidates( + fn: IrFunction, + readers: ReadonlySet, + shared: SharedCandidates = NO_SHARED, + written: Set = new Set(), +): Set { const result = new Set(); - if (fn.async || fn.generator) return result; + if (fn.async || fn.generator) { + // Globals read here leave the candidates. + for (const id of shared.globals) result.add(id); + everyStmtList(fn.body, { + expr: (e) => { + if ((e.kind === "varRef" || e.kind === "assignExpr") && result.has(e.localId)) + result.delete(e.localId); + return true; + }, + stmt: (s) => { + if ((s.kind === "varDecl" || s.kind === "assign") && result.has(s.localId)) + result.delete(s.localId); + return true; + }, + }); + return result; + } const captured = new Set( [...(fn.captures ?? []), ...(fn.classCaptures ?? [])].map((c) => c.localId), ); @@ -217,11 +297,12 @@ export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet(); + const allocated = new Set(); // A receiver position: the candidate is read there without escaping. const receiver = (e: IrExpr): void => { if (e.kind !== "varRef") expr(e); @@ -255,13 +336,28 @@ export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet { + if (shared.paramAt(e.callee, index)) receiver(arg); + else expr(arg); + }); return true; } } @@ -272,7 +368,8 @@ export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet | undefined, live: ReadonlySet): Set { + const result = new Set(); + if (ids) for (const id of ids) if (live.has(id)) result.add(id); + return result; +} + +/** The candidates of every function of `mod` that has any, only those + * whose keys are in `keep` when given. Parameters and globals are shared: + * starting from every one that may qualify, each pass drops the ones a + * function reads outside candidate positions, the parameters some call + * passes anything but a candidate, and the globals never assigned a fresh + * array, until a pass drops nothing. Functions in `redirected` keep their + * parameters: some of their calls reach another function. */ +export function moduleInt32ArrayCandidates( + mod: IrModule, + readers: ReadonlySet, + keep: ReadonlySet | null = null, + redirected: ReadonlySet = new Set(), +): Map> { + const kept = (key: string): boolean => keep === null || keep.has(key); + const arrayParams = new Map(); + for (const fn of mod.functions) { + // Class methods, constructors and runtime helpers (`%` names) keep + // their parameters. + if (fn.name.startsWith("%") || fn.async || fn.generator || redirected.has(fn.name)) continue; + const indexes: number[] = []; + fn.params.forEach((p, i) => { + if (p.type.kind === "array" && p.type.elem.kind === "f64") indexes.push(i); + }); + if (indexes.length > 0) arrayParams.set(fn.name, indexes); + } + let globals = new Set(); + for (const g of mod.globals ?? []) + if ( + g.type.kind === "array" && + g.type.elem.kind === "f64" && + g.initFlag === undefined && + kept(`\0${g.id}`) + ) + globals.add(g.id); + const result = new Map>(); + if (arrayParams.size === 0 && globals.size === 0) { + for (const fn of mod.functions) { + const candidates = int32ArrayCandidates(fn, readers); + for (const id of candidates) if (!kept(`${fn.name}\0${id}`)) candidates.delete(id); + if (candidates.size > 0) result.set(fn.name, candidates); + } + return result; + } + // One walk: the functions reached other than by direct calls, the direct + // calls of functions with number[] parameters, the globals each function + // reads or writes, and whether a module namespace object exists (its + // member reads can reach any global by name). + const reached = new Set(moduleFunctionReferences(mod)); + const calls: { caller: string; call: IrExpr & { kind: "call" } }[] = []; + const globalsIn = new Map>(); + let namespaces = false; + for (const fn of mod.functions) { + const used = new Set(); + const note = (id: string): void => { + if (globals.has(id)) used.add(id); + }; + everyStmtList(fn.body, { + expr: (e) => { + const name = exprFunctionReference(e); + if (name !== undefined) reached.add(name); + if (e.kind === "moduleNsRef") namespaces = true; + if (e.kind === "call" && arrayParams.has(e.callee)) + calls.push({ caller: fn.name, call: e }); + if (e.kind === "varRef" || e.kind === "assignExpr") note(e.localId); + return true; + }, + stmt: (s) => { + if (s.kind === "varDecl" || s.kind === "assign") note(s.localId); + return true; + }, + }); + if (used.size > 0) globalsIn.set(fn.name, used); + } + if (namespaces) { + globals = new Set(); + globalsIn.clear(); + } + const paramIds = new Map(); + let params = new Set(); + for (const fn of mod.functions) { + const indexes = arrayParams.get(fn.name); + if (!indexes || reached.has(fn.name)) continue; + paramIds.set( + fn.name, + fn.params.map((p) => p.localId), + ); + for (const i of indexes) { + const key = `${fn.name}\0${fn.params[i]!.localId}`; + if (kept(key)) params.add(key); + } + } + for (;;) { + const current = params; + const written = new Set(); + const liveGlobals = new Set(globals); + result.clear(); + for (const fn of mod.functions) { + const shared: SharedCandidates = { + params: current, + globals: liveIn(globalsIn.get(fn.name), globals), + paramAt: (callee, index) => { + const id = paramIds.get(callee)?.[index]; + return id !== undefined && current.has(`${callee}\0${id}`); + }, + }; + const candidates = int32ArrayCandidates(fn, readers, shared, written); + for (const id of shared.globals) if (!candidates.has(id)) liveGlobals.delete(id); + for (const id of candidates) + if (!shared.globals.has(id) && !kept(`${fn.name}\0${id}`)) candidates.delete(id); + if (candidates.size > 0) result.set(fn.name, candidates); + } + for (const id of liveGlobals) if (!written.has(id)) liveGlobals.delete(id); + const live = new Set( + [...current].filter((key) => { + const cut = key.indexOf("\0"); + return result.get(key.slice(0, cut))?.has(key.slice(cut + 1)) === true; + }), + ); + // A parameter needs a candidate at that position of every call; a call + // with fewer arguments passes undefined. + for (const { caller, call } of calls) { + const ids = paramIds.get(call.callee); + if (!ids) continue; + ids.forEach((id, i) => { + const key = `${call.callee}\0${id}`; + if (!live.has(key)) return; + const arg = call.args[i]; + const ok = + arg?.kind === "varRef" && + (arg.localId.startsWith("%g.") + ? liveGlobals.has(arg.localId) + : result.get(caller)?.has(arg.localId) === true); + if (!ok) live.delete(key); + }); + } + if (live.size === current.size && liveGlobals.size === globals.size) break; + params = live; + globals = liveGlobals; + } + return result; +} + +/** The candidates one array can reach together: a direct call's argument + * and the parameter it binds hold the same array, so their element facts + * stand or fall together. `group` maps each of `keys` to its group's keys + * (itself alone for most), and `globalReaders` each candidate global's + * key to the functions that read or write it. */ +export function int32ArrayGroups( + mod: IrModule, + keys: ReadonlySet, +): { group: Map; globalReaders: Map> } { + const parent = new Map(); + const root = (key: string): string => { + let r = key; + while (parent.has(r)) r = parent.get(r)!; + if (r !== key) parent.set(key, r); + return r; + }; + const globalReaders = new Map>(); + const params = new Map(mod.functions.map((fn) => [fn.name, fn.params])); + const paramKeys = new Set( + mod.functions.flatMap((fn) => fn.params.map((p) => `${fn.name}\0${p.localId}`)), + ); + if ([...keys].some((key) => key.startsWith("\0") || paramKeys.has(key))) { + for (const fn of mod.functions) { + const touch = (id: string): void => { + const key = int32ArrayKey(fn.name, id); + if (!key.startsWith("\0") || !keys.has(key)) return; + const set = globalReaders.get(key) ?? new Set(); + set.add(fn.name); + globalReaders.set(key, set); + }; + everyStmtList(fn.body, { + expr: (e) => { + if (e.kind === "varRef" || e.kind === "assignExpr") touch(e.localId); + if (e.kind === "call") + e.args.forEach((arg, i) => { + const param = params.get(e.callee)?.[i]; + if (arg.kind !== "varRef" || !param) return; + const a = int32ArrayKey(fn.name, arg.localId), + p = `${e.callee}\0${param.localId}`; + if (!keys.has(a) || !keys.has(p)) return; + const ra = root(a), + rp = root(p); + if (ra !== rp) parent.set(ra, rp); + }); + return true; + }, + stmt: (s) => { + if (s.kind === "varDecl" || s.kind === "assign") touch(s.localId); + return true; + }, + }); + } + } + const members = new Map(); + for (const key of keys) { + const r = root(key); + const list = members.get(r); + if (list) list.push(key); + else members.set(r, [key]); + } + const group = new Map(); + for (const key of keys) group.set(key, members.get(root(key))!); + return { group, globalReaders }; +} + /** The element values stored into each candidate: element writes, pushed * values, literal elements and fill values. */ export function int32ArrayStores( @@ -341,16 +656,34 @@ export function int32ArrayStores( return stores; } +const REPEATED_INDEX_OPS = new Set(["+", "-", "*", "&", "|", "^", "<<", ">>", ">>>"]); + +/** An index the inlined read may evaluate at each of its uses: a plain read, + * a literal, or integer arithmetic over them. Repeating it keeps the forms + * the backend derives from its operands at every use, such as the exact i64 + * of an int32 product, where a bound local holds only the double. */ +function repeatableIndex(e: IrExpr): boolean { + if (e.kind === "varRef" || e.kind === "numLit") return true; + if (e.kind === "unary") return e.op === "-" && repeatableIndex(e.operand); + return ( + e.kind === "bin" && + REPEATED_INDEX_OPS.has(e.op) && + repeatableIndex(e.left) && + repeatableIndex(e.right) + ); +} + /** Rewrite a function's candidates for the backend. A discarded pop uses * the candidate in place of its hidden local. An element-reader helper call * whose array is a candidate is inlined: the helper's returned ternary, with * the candidate for its array and the index for its index, so the read means - * the same. An index that is not a plain read or literal is bound to a fresh - * local first: in its own declaration just before the statement when the - * call is the whole value of a declaration or assignment (the form lowering - * gives `a[i]!`, and the one the backend reads without a box), otherwise in a - * sequence around the ternary. Selection keeps out indexes that write the - * array, so evaluating the index before the array read is unobservable. */ + * the same. An index that is not repeatable (repeatableIndex) is bound to a + * fresh local first: in its own declaration just before the statement when + * the call is the whole value of a declaration or assignment (the form + * lowering gives `a[i]!`, and the one the backend reads without a box), + * otherwise in a sequence around the ternary. Selection keeps out indexes + * that write the array, so evaluating the index before the array read is + * unobservable. */ function rewriteCandidates( fn: IrFunction, candidates: ReadonlySet, @@ -374,14 +707,13 @@ function rewriteCandidates( candidates.has(e.args[0]!.localId) ? e : null; - const simple = (e: IrExpr): boolean => e.kind === "varRef" || e.kind === "numLit"; // The statements of a list, with complex read indexes declared first. const hoist = (list: IrStmt[]): IrStmt[] => list.flatMap((s): IrStmt[] => { if (s.kind !== "varDecl" && s.kind !== "assign") return [s]; const value = s.kind === "varDecl" ? s.init : s.value; const call = readerCall(value); - if (!call || simple(call.args[1]!)) return [s]; + if (!call || repeatableIndex(call.args[1]!)) return [s]; const { decl, ref } = bind(call.args[1]!); const read: IrExpr = { ...call, args: [call.args[0]!, ref] }; if (s.kind === "varDecl") return [decl, { ...s, init: read }]; @@ -444,21 +776,23 @@ function rewriteCandidates( if (ret.kind !== "return" || !ret.value) return e; let index = call.args[1]!; const stmts: IrStmt[] = []; - if (!simple(index)) { + if (!repeatableIndex(index)) { const { decl, ref } = bind(index); stmts.push(decl); index = ref; } const operand = index; - const body = transformExpr(reparseExpr(ret.value), { - expr: (x) => { - if (x.kind !== "varRef") return x; - if (x.localId === arrayParam!.localId) return reparseExpr(array); - if (x.localId === indexParam!.localId) return reparseExpr(operand); - return x; - }, - stmt: (x) => x, - }); + // The arguments' copies are not searched again: an index may read a + // local that has the id of one of the helper's parameters. + const substituteExpr = (x: IrExpr): IrExpr => { + if (x.kind !== "varRef") return mapExprChildren(x, substituteExpr, substituteStmt); + if (x.localId === arrayParam!.localId) return reparseExpr(array); + if (x.localId === indexParam!.localId) return reparseExpr(operand); + return x; + }; + const substituteStmt = (x: IrStmt): IrStmt => + mapStmtChildren(x, substituteExpr, substituteStmt); + const body = substituteExpr(reparseExpr(ret.value)); // The ternary is a fresh copy (reparseExpr): it takes the call's location. const typed = body; typed.loc = e.loc; @@ -471,40 +805,45 @@ function rewriteCandidates( return { ...fn, body, locals: locals.filter((local) => !popLocals.has(local.id)) }; } -/** Propose the module's int32 array candidates (`function\0local` keys in +/** Propose the module's int32 array candidates (int32ArrayKey keys in * `int32ArrayCandidates`), only those in `keep` when given, and rewrite * their functions. */ export function markInt32ArrayCandidates( mod: IrModule, keep: ReadonlySet | null = null, + redirected: ReadonlySet = new Set(), ): IrModule { if (mod.lib !== undefined) return mod; const readers = new Map(); for (const fn of mod.functions) if (isElementReader(fn)) readers.set(fn.name, fn); - const names = new Set(readers.keys()); - const keys: string[] = []; + const all = moduleInt32ArrayCandidates(mod, new Set(readers.keys()), keep, redirected); + if (all.size === 0) return mod; + const keys = new Set(); const functions = mod.functions.map((fn) => { - const candidates = int32ArrayCandidates(fn, names); - for (const id of candidates) - if (keep === null || keep.has(`${fn.name}\0${id}`)) keys.push(`${fn.name}\0${id}`); - else candidates.delete(id); - return candidates.size === 0 ? fn : rewriteCandidates(fn, candidates, readers); + const candidates = all.get(fn.name); + if (!candidates) return fn; + for (const id of candidates) keys.add(int32ArrayKey(fn.name, id)); + return rewriteCandidates(fn, candidates, readers); }); - return keys.length === 0 ? mod : { ...mod, functions, int32ArrayCandidates: keys }; + return { ...mod, functions, int32ArrayCandidates: [...keys] }; } /** The module the backend emits: rewritten for exactly the candidates the * int32 slot analysis (`prove`, returning its int32Arrays) keeps, so a * module without int32 arrays is `mod` itself. A dropped candidate's * rewrite is undone by marking again without it, until every candidate - * marked is kept. */ + * marked is kept. `redirected` names the functions some of whose calls the + * backend emits as calls of another function (a result-slot entry): their + * parameters keep doubles. */ export function selectInt32Arrays( mod: IrModule, prove: (marked: IrModule) => ReadonlySet, + redirected: (mod: IrModule) => ReadonlySet = () => new Set(), ): IrModule { let keep: ReadonlySet | null = null; + const others = redirected(mod); for (let round = 0; round < 4; round++) { - const marked = markInt32ArrayCandidates(mod, keep); + const marked = markInt32ArrayCandidates(mod, keep, others); if (marked === mod) return mod; const kept = prove(marked); const proposed = marked.int32ArrayCandidates ?? []; diff --git a/packages/compiler/src/ir/int32-slots.ts b/packages/compiler/src/ir/int32-slots.ts index df9e3c0a5..85f373c89 100644 --- a/packages/compiler/src/ir/int32-slots.ts +++ b/packages/compiler/src/ir/int32-slots.ts @@ -8,10 +8,26 @@ import { type IntegerRanges, type IntegerSlotFacts, } from "./integer-ranges.js"; -import { exprFunctionReference, moduleFunctionReferences } from "./function-references.js"; +import { + exprFunctionReference, + FUNCTION_REFERENCE_KINDS, + moduleFunctionReferences, +} from "./function-references.js"; +import { + eachNodeOfKinds, + type IrKindScan, + type IrNodeIndex, + type IrNodeKind, +} from "./node-index.js"; import { findInitializerBindings, withInitializerBindings } from "./initializer-bindings.js"; -import { checkableInt32Allocations, int32ArrayStores } from "./int32-arrays.js"; -import { everyExprChild, everyStmtChild, everyStmtList, everyTypeChild } from "./traverse.js"; +import { + checkableInt32Allocations, + int32ArrayGroups, + int32ArrayKey, + int32ArraySlot, + int32ArrayStores, +} from "./int32-arrays.js"; +import { everyExprChild, everyStmtChild, everyTypeChild } from "./traverse.js"; /** The storage slot an array-valued expression reads, as an * IrContracts.int32Elements key: a local or parameter @@ -247,9 +263,10 @@ export class Int32Slots { /** Whether the module has int32 array candidates (the narrowedReads * range fact). */ narrowedReads = false; - /** Local number arrays (`function\0local`) stored as int32 slots: the - * module's int32 array candidates whose every stored element the - * fixpoint proved an int32. */ + /** Number arrays stored as int32 slots (int32ArrayKey keys: locals and + * parameters as `function\0local`, globals as `\0global`): the module's + * int32 array candidates whose every stored element, through every + * binding that can hold the same array, the fixpoint proved an int32. */ readonly int32Arrays = new Set(); /** Local number arrays that may hold int32 slots with checked stores: the * candidates whose element stores the fixpoint proved exact integers but @@ -467,10 +484,61 @@ const DYN_READ_ONLY = new Set([ "text.decodeStream", "worker.new", ]); +/** The node kinds each walk below reads. */ +const LIB_CALL_KINDS: readonly IrNodeKind[] = ["libCall"]; +const CALL_FACT_KINDS: readonly IrNodeKind[] = [ + ...FUNCTION_REFERENCE_KINDS, + "virtualCall", + "instanceOfValue", + "varRef", + "assignExpr", + "incDec", + "classRef", + "varDecl", + "assign", + "forOf", +]; +const CENSUS_KINDS: readonly IrNodeKind[] = [ + "call", + "new", + "varRef", + "fieldGet", + "fieldIncDec", + "closure", + "classRef", + "incDec", + "forOf", + "tryCatch", +]; +const DECLARATION_KINDS: readonly IrNodeKind[] = ["varDecl"]; +const CANDIDATE_WRITE_KINDS: readonly IrNodeKind[] = [ + "assign", + "varDecl", + "fieldSet", + "return", + "call", + "new", + "assignExpr", +]; +const CALL_KINDS: readonly IrNodeKind[] = ["call", "new"]; +const SLOT_WRITE_KINDS: readonly IrNodeKind[] = ["return", "call", "new"]; +const NARROW_WRITE_KINDS: readonly IrNodeKind[] = ["assign", "varDecl", "fieldSet", "assignExpr"]; + +/** Visits the nodes of `fn` in walk order; with `scan`, only those of its + * kinds, read from the module's node index (the visitors must ignore the + * other kinds). */ function walkFunction( fn: IrFunction, + scan: IrKindScan | undefined, visit: { expr?: (e: IrExpr) => void; stmt?: (s: IrStmt) => void }, ): void { + if (scan !== undefined) { + eachNodeOfKinds(fn, scan, (node, statement) => { + if (statement) visit.stmt?.(node as IrStmt); + else visit.expr?.(node as IrExpr); + }); + return; + } const expr = (e: IrExpr): boolean => { visit.expr?.(e); return everyExprChild(e, expr, stmt); @@ -498,6 +566,7 @@ export function dynamicFieldStores( mod: IrModule, records = new Map((mod.records ?? []).map((r) => [r.id, r])), unions = new Map((mod.unions ?? []).map((u) => [u.id, u])), + nodes?: IrNodeIndex, ): DynamicFieldStores { const dynTypes = new Map(); const dynNamed = new Map(); @@ -537,9 +606,10 @@ export function dynamicFieldStores( let dynWritesAll = false; // A store whose key is not a literal names every field: later stores add // nothing, so the walk stops there. + const scan = nodes?.scan(LIB_CALL_KINDS); for (const fn of mod.functions) { if (dynWritesAll) break; - walkFunction(fn, { + walkFunction(fn, scan, { expr: (e) => { if (dynWritesAll) return; if ( @@ -642,7 +712,7 @@ export function dynamicallyStoredFamilies( } }; for (const fn of mod.functions) - walkFunction(fn, { + walkFunction(fn, undefined, { expr: (e) => { const dynResult = DYN_KINDS.has(e.type.kind); const boundary = BOUNDARY_KINDS.has(e.kind); @@ -682,6 +752,7 @@ export function functionCallFacts( mod: IrModule, classes: ReadonlyMap, subtree: (className: string) => string[], + nodes?: IrNodeIndex, ): { valueEscaped: Set; virtualTargets: Set; @@ -716,7 +787,7 @@ export function functionCallFacts( const count = (map: Map, id: string): void => { map.set(id, (map.get(id) ?? 0) + 1); }; - walkFunction(fn, { + walkFunction(fn, nodes?.scan(CALL_FACT_KINDS), { expr: (e) => { const referenced = exprFunctionReference(e); if (referenced !== undefined) escaped.add(referenced); @@ -752,9 +823,12 @@ export function functionCallFacts( export interface Int32SlotInputs { dynamicStores?: DynamicFieldStores; initializerBindings?: ReadonlyMap; + /** The module's node index, read instead of walking the bodies. */ + nodes?: IrNodeIndex; } export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): Int32Slots { + const nodes = inputs.nodes; const classes = new Map((mod.classes ?? []).map((c) => [c.name, c])); const slots = new Int32Slots(classes); if (mod.lib) return slots; @@ -780,17 +854,29 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): for (const key of contracts?.int32Locals ?? []) slots.addInt32Local(key); for (const key of contracts?.int32Elements ?? []) slots.addInt32Elements(key); // Int32 array candidates (ir/int32-arrays.ts): assumed to hold int32 - // elements until a store the fixpoint cannot prove drops one. + // elements until a store the fixpoint cannot prove drops one, with every + // candidate the same array can reach (its group). const arrayCandidates = new Map>(); - slots.narrowedReads = (mod.int32ArrayCandidates?.length ?? 0) > 0; - for (const key of mod.int32ArrayCandidates ?? []) { + const globalArrays = new Set(); + const arrayKeys = new Set(mod.int32ArrayCandidates ?? []); + slots.narrowedReads = arrayKeys.size > 0; + for (const key of arrayKeys) { const cut = key.indexOf("\0"); - const fnName = key.slice(0, cut); - const set = arrayCandidates.get(fnName) ?? new Set(); - set.add(key.slice(cut + 1)); - arrayCandidates.set(fnName, set); - slots.addInt32Elements(`l:${key}`); + if (cut === 0) globalArrays.add(key.slice(1)); + else { + const fnName = key.slice(0, cut); + const set = arrayCandidates.get(fnName) ?? new Set(); + set.add(key.slice(cut + 1)); + arrayCandidates.set(fnName, set); + } + slots.addInt32Elements(int32ArraySlot(key)); } + const arrayGroups = arrayKeys.size > 0 ? int32ArrayGroups(mod, arrayKeys) : null; + // Functions that read or write a candidate global. + const globalArrayFunctions = new Set(); + if (arrayGroups) + for (const names of arrayGroups.globalReaders.values()) + for (const name of names) globalArrayFunctions.add(name); /* ── candidate fields ── */ const children = new Map(); @@ -844,7 +930,7 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): /* ── functions: escapes, parameter and return candidates ── */ const functions = new Map(mod.functions.map((fn) => [fn.name, fn])); - const calls = functionCallFacts(mod, classes, subtree); + const calls = functionCallFacts(mod, classes, subtree, nodes); const escaped = new Set([...calls.valueEscaped, ...calls.virtualTargets]); const virtual = calls.runtimeVirtual; @@ -901,7 +987,7 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): for (const capture of fn.classCaptures ?? []) unstableBoxes.add(boxKey(fn.name, capture.localId)); const read = new Set(); - walkFunction(fn, { + walkFunction(fn, nodes?.scan(CENSUS_KINDS), { expr: (e) => { if (e.kind === "call") { called.add(e.callee); @@ -1001,7 +1087,7 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): let ids = initializedDeclarations.get(fn.name); if (ids === undefined) { const found = new Set(); - walkFunction(fn, { + walkFunction(fn, nodes?.scan(DECLARATION_KINDS), { stmt: (s) => { if (s.kind === "varDecl" && s.init) found.add(s.localId); }, @@ -1069,28 +1155,46 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): } return false; }; + const candidateWrites = nodes?.scan(CANDIDATE_WRITE_KINDS); const hasCandidateWrite = (fn: IrFunction): boolean => { if (arrayCandidates.has(fn.name)) return true; + if (globalArrayFunctions.has(fn.name)) return true; const box = (localId: string): boolean => slots.hasBox(slots.boxGroup(fn.name, localId)); const returns = slots.hasReturn(fn.name); - return !everyStmtList(fn.body, { - stmt: (s) => { - if (s.kind === "assign") return !box(s.localId); - if (s.kind === "varDecl") return s.init === null || !box(s.localId); - if (s.kind === "fieldSet") { - const family = slots.family(s.className, s.field); - return !family || !slots.hasFamily(family); + let found = false; + eachNodeOfKinds(fn, candidateWrites, (node) => { + switch (node.kind) { + case "assign": + found = box(node.localId); + break; + case "varDecl": + found = node.init !== null && box(node.localId); + break; + case "fieldSet": { + const family = slots.family(node.className, node.field); + found = !!family && slots.hasFamily(family); + break; } - return !(s.kind === "return" && returns); - }, - expr: (e) => { - if (e.kind === "call") return !candidateArgs(functions.get(e.callee), e.args.length, 0); - if (e.kind === "new") - return !candidateArgs(functions.get(`%${e.className}.constructor`), e.args.length, 1); - if (e.kind === "assignExpr") return !box(e.localId); - return true; - }, + case "return": + found = returns; + break; + case "call": + found = candidateArgs(functions.get(node.callee), node.args.length, 0); + break; + case "new": + found = candidateArgs( + functions.get(`%${node.className}.constructor`), + node.args.length, + 1, + ); + break; + case "assignExpr": + found = box(node.localId); + break; + } + return !found; }); + return found; }; const provenByFunction = new Map(); // Each function's parameter and result writes under its latest analysis: @@ -1107,7 +1211,7 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): const ranges: IntegerRanges = candidate ? analyzeIntegerRanges(numeric(fn), slots.facts(fn.name)) : new Map(); - if (ranges.size > 0) lastWrites.set(fn.name, slotWrites(fn, ranges, functions)); + if (ranges.size > 0) lastWrites.set(fn.name, slotWrites(fn, ranges, functions, nodes)); else lastWrites.delete(fn.name); const proven: IrExpr[] = []; // The walk below acts only at writes into candidate slots. @@ -1125,25 +1229,49 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): // (checkableInt32Allocations), it keeps int32 slots with checked stores // (checkedInt32Arrays), which widen it to doubles in place at the first // store that is not an int32. - const arrays = arrayCandidates.get(fn.name); - if (arrays) { - let dropped = false; + const locals = arrayCandidates.get(fn.name); + let arrays = locals; + if (globalArrayFunctions.has(fn.name)) { + const both = new Set(); + if (locals) for (const id of locals) both.add(id); + for (const id of globalArrays) both.add(id); + arrays = both; + } + if (arrays && arrayGroups) { + const requeue = (name: string): void => { + if (queued.has(name)) return; + queued.add(name); + queue.push(name); + }; + const params = new Set(fn.params.map((param) => param.localId)); for (const [id, values] of int32ArrayStores(fn, arrays)) { - const key = `l:${fn.name}\0${id}`; - if (!slots.hasInt32ElementSlot(key) || values.every(int32)) continue; - slots.dropInt32Elements(key); + const key = int32ArrayKey(fn.name, id); + if (!slots.hasInt32ElementSlot(int32ArraySlot(key)) || values.every(int32)) continue; + // The array's group drops together: every function reading a + // member is analyzed again without its facts. + const group = arrayGroups.group.get(key) ?? [key]; + for (const member of group) { + slots.dropInt32Elements(int32ArraySlot(member)); + if (member.startsWith("\0")) { + const names = arrayGroups.globalReaders.get(member); + if (names) for (const name of names) requeue(name); + } else requeue(member.slice(0, member.indexOf("\0"))); + } + // Checked stores serve a lone local: its allocations are the only + // arrays it holds. const checkable = (value: IrExpr): boolean => { const range = ranges.get(value); return int32(value) || range != null; }; - if (values.every(checkable) && checkableInt32Allocations(fn, id)) - slots.checkedInt32Arrays.add(`${fn.name}\0${id}`); - else reason(`array:${fn.name}\0${id}`, "an element store is not int32"); - dropped = true; - } - if (dropped && !queued.has(fn.name)) { - queued.add(fn.name); - queue.push(fn.name); + if ( + group.length === 1 && + !key.startsWith("\0") && + !params.has(id) && + values.every(checkable) && + checkableInt32Allocations(fn, id) + ) + slots.checkedInt32Arrays.add(key); + else for (const member of group) reason(`array:${member}`, "an element store is not int32"); } } // An int32 written into a slot still assumed non-negative: widen the @@ -1182,7 +1310,7 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): if (int32(value)) settle(`box:${group}`, value); else if (dropBox(group, `write in ${fn.name} is not int32`)) invalidate(`box:${group}`); }; - walkFunction(fn, { + walkFunction(fn, nodes?.scan(CANDIDATE_WRITE_KINDS), { stmt: (s) => { if (s.kind === "assign") boxWrite(s.localId, s.value); else if (s.kind === "varDecl" && s.init) boxWrite(s.localId, s.init); @@ -1214,14 +1342,25 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}): } for (const proven of provenByFunction.values()) for (const value of proven) slots.provenWrites.add(value); - for (const [fnName, ids] of arrayCandidates) - for (const id of ids) - if (slots.hasInt32ElementSlot(`l:${fnName}\0${id}`)) - slots.int32Arrays.add(`${fnName}\0${id}`); - const ranged = rangedParameters(slots, functions, retry, numeric, (callee) => - readers.get(`ret:${callee}`), + for (const key of arrayKeys) + if (slots.hasInt32ElementSlot(int32ArraySlot(key))) slots.int32Arrays.add(key); + const ranged = rangedParameters( + slots, + functions, + retry, + numeric, + (callee) => readers.get(`ret:${callee}`), + nodes, + ); + narrowSlotRanges( + slots, + functions, + numeric, + lastWrites, + ranged, + (slot) => readers.get(slot), + nodes, ); - narrowSlotRanges(slots, functions, numeric, lastWrites, ranged, (slot) => readers.get(slot)); return slots; } @@ -1256,6 +1395,7 @@ function rangedParameters( retry: ReadonlySet, numeric: (fn: IrFunction) => IrFunction, callersOf: (callee: string) => ReadonlySet | undefined, + nodes: IrNodeIndex | undefined, ): Set { const changed = new Set(); if (retry.size === 0) return changed; @@ -1325,7 +1465,7 @@ function rangedParameters( contribute(callee, param.localId, ranges.get(arg) ?? null); }); }; - walkFunction(fn, { + walkFunction(fn, nodes?.scan(CALL_KINDS), { expr: (e) => { if (e.kind === "call") args(functions.get(e.callee), e.args, 0); else if (e.kind === "new") args(functions.get(`%${e.className}.constructor`), e.args, 1); @@ -1371,6 +1511,7 @@ function slotWrites( fn: IrFunction, ranges: IntegerRanges, functions: ReadonlyMap, + nodes: IrNodeIndex | undefined, ): Map { const writes = new Map(); const write = (key: string, value: IrExpr | null): void => { @@ -1385,7 +1526,7 @@ function slotWrites( if (param) write(`param:${callee.name}\0${param.localId}`, arg); }); }; - walkFunction(fn, { + walkFunction(fn, nodes?.scan(SLOT_WRITE_KINDS), { stmt: (s) => { if (s.kind === "return") write(`ret:${fn.name}`, s.value); }, @@ -1428,6 +1569,7 @@ function narrowSlotRanges( lastWrites: ReadonlyMap>, changed: ReadonlySet, readersOf: (slot: string) => ReadonlySet | undefined, + nodes: IrNodeIndex | undefined, ): void { // Every parameter and result slot with a fact, and that fact. const proven = new Map(); @@ -1486,7 +1628,7 @@ function narrowSlotRanges( const level = slots.isNonNegative(`box:${group}`) ? NON_NEGATIVE_INT32_RANGE : INT32_RANGE; if (!rangeWithin(range, level)) sound = false; }; - walkFunction(fn, { + walkFunction(fn, nodes?.scan(NARROW_WRITE_KINDS), { stmt: (s) => { if (s.kind === "assign") box(s.localId, s.value); else if (s.kind === "varDecl" && s.init) box(s.localId, s.init); @@ -1497,7 +1639,7 @@ function narrowSlotRanges( if (e.kind === "assignExpr") box(e.localId, e.value); }, }); - return { writes: slotWrites(fn, ranges, functions), sound }; + return { writes: slotWrites(fn, ranges, functions, nodes), sound }; }; const writesByFunction = new Map(lastWrites); const join = (): Map => { diff --git a/packages/compiler/src/ir/integer-ranges.test.ts b/packages/compiler/src/ir/integer-ranges.test.ts index fcb040275..5ae04d28f 100644 --- a/packages/compiler/src/ir/integer-ranges.test.ts +++ b/packages/compiler/src/ir/integer-ranges.test.ts @@ -440,6 +440,37 @@ test("int32-closed locals keep the int32 range across loops and switches", () => expect(switched.get(afterSwitch)).toEqual(INT32); }); +test("loop-carried locals written from Int32[] element reads keep the int32 range", () => { + const arrType = { kind: "array", elem: F64 } as const; + const loop = (localId: string): { after: IrExpr; f: IrFunction } => { + const after = ref(); + // `x = a[x]!` in a loop: a cursor that follows indices stored in the array. + const read: IrExpr = { + loc, + kind: "arrayGet", + arr: { loc, kind: "varRef", localId, type: arrType }, + index: ref(), + type: F64, + }; + const f = fn([ + assign(num(0)), + { + loc, + kind: "while", + cond: condition("<", ref("unknown"), num(1)), + body: [assign(read)], + }, + observe(after), + ]); + return { after, f }; + }; + const elements = { elements: (arr: IrExpr) => arr.kind === "varRef" && arr.localId === "ints" }; + const ints = loop("ints"); + expect(analyzeIntegerRanges(ints.f, elements).get(ints.after)).toEqual(INT32); + const floats = loop("floats"); + expect(analyzeIntegerRanges(floats.f, elements).get(floats.after) ?? null).toBeNull(); +}); + test("uint32-closed locals keep the uint32 range across loop back edges", () => { // for (let i = 0; i < n; i++) { v = unknown >>> 0; acc = (acc + v) >>> 0; ... } const loop = (start: IrExpr | null, writes: IrExpr[]) => { diff --git a/packages/compiler/src/ir/integer-ranges.ts b/packages/compiler/src/ir/integer-ranges.ts index 0e5ca426d..683ceab75 100644 --- a/packages/compiler/src/ir/integer-ranges.ts +++ b/packages/compiler/src/ir/integer-ranges.ts @@ -480,6 +480,7 @@ export function analyzeIntegerRangeFacts( return boxedRange(e.localId); } case "fieldGet": + case "arrayGet": case "call": return slotRange(e); default: diff --git a/packages/compiler/src/ir/ir.ts b/packages/compiler/src/ir/ir.ts index 6d0760b81..0ec0043ce 100644 --- a/packages/compiler/src/ir/ir.ts +++ b/packages/compiler/src/ir/ir.ts @@ -1056,6 +1056,10 @@ export interface IrModule { * each one's hierarchy carries a slot word (the slots.* intrinsics; * backend/llvm/thread-slots.ts). */ threadSlotKeys?: string[]; + /** typeKeys of the array types whose values the program may freeze + * (frontend frozen-records.ts): their writes test the array's frozen bit + * (SCR_ARR_FROZEN) and throw Node's TypeError when it is set. */ + freezableArrays?: string[]; /** The entry module is CommonJS under Node's classification. Node runs a * CommonJS main synchronously and drains process.nextTick callbacks * before promise jobs at the first checkpoint; an ES-module main's @@ -1685,6 +1689,10 @@ export interface IrRecordShape { * dynCheck), and JSON serialization appends overflow entries after the * declared fields in insertion order. */ indexValue?: IrType; + /** Values of this shape may be frozen (frozen-records.ts): their rc word + * can carry the frozen bit, which their release ignores, and every write + * to them checks it. */ + freezable?: true; /** Field names in FIRST-SEEN declaration order (the checker's property * order of the first ts.Type interned to this shape) — metadata, NOT * part of the interned identity: a later structurally-equal type with a @@ -1745,6 +1753,18 @@ export function recordTextCodecClass(shape: IrRecordShape): "TextEncoder" | "Tex return null; } +/** The codec class of a TextEncoder or TextDecoder record type, else null. + * Such a record crosses an `unknown` slot as a native handle that holds the + * record itself (scr_text_codec_dyn.c), never as a copy or a field view. */ +export function textCodecRecordClass( + t: IrType, + getRecord: (shapeId: string) => IrRecordShape | undefined, +): "TextEncoder" | "TextDecoder" | null { + if (t.kind !== "record") return null; + const shape = getRecord(t.shapeId); + return shape ? recordTextCodecClass(shape) : null; +} + /** Object-literal ACCESSOR properties (`{ get x() {...}, set x(v) {...} }`) * live on the shape as reserved '%'-fields holding closures: `%get:x` a * `() => T` invoked once per property READ (side effects and all — JS's @@ -3898,8 +3918,10 @@ export type IrExpr = * not shape equality; missing or wrong-typed fields throw, EXCEPT that a * missing key for an undefined-armed union field — an optional field — * builds the interned undefined arm instead); arrays check every element; - * unions try arms in canonical order and the first FULL match wins (no - * match → throw; an undefined arm matches no dyn value); JSON null matches + * unions try arms in canonical order, except that record arms with more + * required fields and then more fields go first, and the first FULL match + * wins (no match → throw; an undefined arm matches no dyn value); JSON + * null matches * exactly the nullT arm of a union target (bare null targets cannot * exist). MAY THROW: * backends' may-throw analyses must treat it like a `throw` statement. @@ -4271,6 +4293,7 @@ export function canConvertToDyn( visiting: Set = new Set(), ): boolean { if (isJsonSafeType(t, getRecord, getUnion)) return true; + if (textCodecRecordClass(t, getRecord) !== null) return true; // numeric typed arrays and boxable functions are dyn kinds the walker boxes // ANYWHERE (bytes and functions held by identity), including nested // in records/arrays/unions. isJsonSafeType rejects them, but dynFrom @@ -4317,6 +4340,7 @@ export function canConvertToDyn( // either its result or a promise of it. canBoxDynComposite(a, getRecord, getUnion, visiting) || isDynTypedRefType(a) || + textCodecRecordClass(a, getRecord) !== null || a.kind === "classval" || DYN_HANDLE_KINDS.has(a.kind) || (a.kind === "func" && canBoxFuncIntoDyn(a, getRecord, getUnion, visiting)) || @@ -4396,7 +4420,8 @@ function canBoxDynCompositeAt( case "record": { const shape = getRecord(t.shapeId); if (!shape) return false; - if (recordTextCodecClass(shape) !== null) return false; + // A codec boxes as its own handle wherever it sits (toDynHelper). + if (recordTextCodecClass(shape) !== null) return true; // Recursive shapes answer coinductively, like isJsonSafeType. if (visiting.has(t.shapeId)) return true; visiting.add(t.shapeId); @@ -4497,6 +4522,8 @@ function canDynCheckToAt( visiting.delete(key); return result; } + // A codec handle unboxes to the record it holds. + if (textCodecRecordClass(t, getRecord) !== null) return true; if (t.kind === "record") { const key = typeKey(t); if (visiting.has(key)) return false; diff --git a/packages/compiler/src/ir/js-ic.test.ts b/packages/compiler/src/ir/js-ic.test.ts index f71a1e9db..f5391b83e 100644 --- a/packages/compiler/src/ir/js-ic.test.ts +++ b/packages/compiler/src/ir/js-ic.test.ts @@ -126,6 +126,140 @@ test("property helpers whose class branches only test capsules are cache sites", expect(sites.site("%dyn.class.property.1")).toEqual({ kind: "set", key: "y" }); }); +test("bags converted between dyn and jsval are still the receiver", () => { + // A program that declares a class returns its bags as jsvals, and the + // helpers read them through js.toDyn. + const toDyn = (value: IrExpr): IrExpr => ({ + kind: "libCall", + fn: "js.toDyn", + args: [value], + type: DYN, + loc, + }); + const fromDyn = (value: IrExpr): IrExpr => ({ + kind: "libCall", + fn: "js.fromDyn", + args: [value], + type: { kind: "jsval" }, + loc, + }); + const jsvalBag = helper( + "%dyn.class.readPropertiesOrEmpty.js", + ["v"], + [capsuleBranch("v"), ret(fromDyn(ref("v")))], + ); + const staticBag = helper("%dyn.class.staticProperties", ["v"], [ret(ref("v"))]); + const getter = getHelper( + "%dyn.class.property.0", + "x", + [capsuleBranch("p.0")], + call(staticBag.name, toDyn(call(jsvalBag.name, ref("p.0")))), + ); + const setter = setHelper("%dyn.class.property.1", "y"); + const write = setter.body[0] as Extract; + const keySet = write.expr as Extract; + keySet.args[0] = toDyn(call(jsvalBag.name, ref("p.0"))); + // A converted value that is not the receiver stays a helper call. + const foreign = getHelper( + "%dyn.class.property.2", + "x", + [], + toDyn(fromDyn({ kind: "dynObjLit", type: DYN, loc } as IrExpr)), + ); + const sites = sitesOf(jsvalBag, staticBag, getter, setter, foreign); + expect(sites.site("%dyn.class.property.0")).toEqual({ kind: "get", key: "x" }); + expect(sites.site("%dyn.class.property.1")).toEqual({ kind: "set", key: "y" }); + expect(sites.site("%dyn.class.property.2")).toBeNull(); +}); + +test("a bag helper may first rebind its parameter to a bag of it", () => { + // Programs with subclasses normalize the value before the class branches. + const normalize = helper("%dyn.class.normalize", ["v"], [capsuleBranch("v"), ret(ref("v"))]); + const rebind = (value: IrExpr): IrStmt => ({ kind: "assign", localId: "v", value, loc }); + const normalized = helper( + "%dyn.class.readPropertiesOrEmpty.normalized", + ["v"], + [rebind(call(normalize.name, ref("v"))), capsuleBranch("v"), ret(ref("v"))], + ); + const replaced = helper( + "%dyn.class.readPropertiesOrEmpty.replaced", + ["v"], + [rebind({ kind: "dynObjLit", type: DYN, loc } as IrExpr), ret(ref("v"))], + ); + const sites = sitesOf( + normalize, + normalized, + replaced, + getHelper("%dyn.class.property.0", "x", [], call(normalized.name, ref("p.0"))), + getHelper("%dyn.class.property.1", "x", [], call(replaced.name, ref("p.0"))), + // The property helper itself normalizes its receiver first, too. + getHelper("%dyn.class.property.2", "x", [ + { kind: "assign", localId: "p.0", value: call(normalize.name, ref("p.0")), loc }, + capsuleBranch("p.0"), + ]), + ); + expect(sites.site("%dyn.class.property.0")).toEqual({ kind: "get", key: "x" }); + expect(sites.site("%dyn.class.property.1")).toBeNull(); + expect(sites.site("%dyn.class.property.2")).toEqual({ kind: "get", key: "x" }); +}); + +test("keyed helpers may test capsules between the key's coercion and the read", () => { + const read = (object: IrExpr, key: string): IrExpr => ({ + kind: "libCall", + fn: "js.fromDyn", + args: [{ kind: "dynKeyGet", value: object, key: ref(key), type: DYN, loc }], + type: { kind: "jsval" }, + loc, + }); + const guard = (test: "symbol" | "nullish", tested: string, object: IrExpr): IrStmt => ({ + kind: "if", + cond: { kind: "dynTest", test, value: ref(tested), type: BOOL, loc }, + then: [ret(read(object, "p.key"))], + else_: null, + loc, + }); + const keyed = (name: string, between: IrStmt[]): IrFunction => + helper( + name, + ["p.0", "p.key"], + [ + capsuleBranch("p.0"), + guard("symbol", "p.key", call(identityBag.name, ref("p.0"))), + guard("nullish", "p.0", ref("p.0")), + { + kind: "varDecl", + localId: "key.string", + init: { + kind: "libCall", + fn: "dyn.toStringCoerce", + args: [ref("p.key")], + type: STRING, + loc, + }, + loc, + }, + ...between, + ret(read(call(identityBag.name, ref("p.0")), "key.string")), + ], + ["key.string"], + ); + const errorBranch: IrStmt = { + kind: "if", + cond: { kind: "dynTest", test: "error", value: ref("p.0"), type: BOOL, loc }, + then: [ret(ref("p.0"))], + else_: null, + loc, + }; + const sites = sitesOf( + keyed("%dyn.class.computed.0", []), + keyed("%dyn.class.computed.1", [capsuleBranch("p.0"), capsuleBranch("p.0", "dyn.classIs")]), + keyed("%dyn.class.computed.2", [errorBranch]), + ); + expect(sites.site("%dyn.class.computed.0")).toEqual({ kind: "keyed" }); + expect(sites.site("%dyn.class.computed.1")).toEqual({ kind: "keyed" }); + expect(sites.site("%dyn.class.computed.2")).toBeNull(); +}); + test("any other prefix or tail keeps the helper call", () => { const errorBranch: IrStmt = { kind: "if", @@ -237,3 +371,57 @@ test("bag helpers that exit early for values no class branch matches are still t expect(sites.site("%dyn.class.property.2")).toBeNull(); expect(sites.site("%dyn.class.property.3")).toBeNull(); }); + +test("helpers that answer plain values first with the same operation are cache sites", () => { + const negated = (test: "classCapsule" | "funcKind"): IrExpr => ({ + kind: "dynTest", + test, + negated: true, + value: ref("p.0"), + type: BOOL, + loc, + }); + const exit = (body: IrStmt[], test: IrExpr = negated("classCapsule")): IrStmt => ({ + kind: "if", + cond: test, + then: [{ kind: "if", cond: negated("funcKind"), then: body, else_: null, loc }], + else_: null, + loc, + }); + const store = (key: string): IrStmt[] => [ + { + kind: "exprStmt", + expr: { + kind: "libCall", + fn: "dyn.keySet", + args: [ref("p.0"), str(key), ref("p.1")], + type: VOID, + loc, + }, + loc, + }, + ret(null), + ]; + const load = (key: string): IrStmt[] => [ + ret({ kind: "dynKeyGet", value: ref("p.0"), key: str(key), type: DYN, loc }), + ]; + const sites = sitesOf( + setHelper("%dyn.class.property.0", "x", [exit(store("x")), capsuleBranch("p.0")]), + getHelper("%dyn.class.property.1", "x", [exit(load("x")), capsuleBranch("p.0")]), + // An exit for another key, or one plain values do not take. + setHelper("%dyn.class.property.2", "x", [exit(store("y"))]), + setHelper("%dyn.class.property.3", "x", [ + exit(store("x"), { + kind: "dynTest", + test: "classCapsule", + value: ref("p.0"), + type: BOOL, + loc, + }), + ]), + ); + expect(sites.site("%dyn.class.property.0")).toEqual({ kind: "set", key: "x" }); + expect(sites.site("%dyn.class.property.1")).toEqual({ kind: "get", key: "x" }); + expect(sites.site("%dyn.class.property.2")).toBeNull(); + expect(sites.site("%dyn.class.property.3")).toBeNull(); +}); diff --git a/packages/compiler/src/ir/js-ic.ts b/packages/compiler/src/ir/js-ic.ts index 752316db3..48019636a 100644 --- a/packages/compiler/src/ir/js-ic.ts +++ b/packages/compiler/src/ir/js-ic.ts @@ -61,13 +61,68 @@ export class JsIcSites { if (fn.params.some((p) => p.type.kind !== "dyn")) return null; const receiver = fn.params[0]?.localId; if (receiver === undefined) return null; + const second = fn.params[1]?.localId; + const exits: JsIcSite[] = []; let i = 0; - while (i < fn.body.length && this.capsuleBranch(fn.body[i]!, receiver)) i++; + for (; i < fn.body.length; i++) { + const s = fn.body[i]!; + if (this.capsuleBranch(s, receiver) || this.rebinding(s, receiver)) continue; + const exit = this.plainExit(s, receiver, second); + if (exit === null) break; + exits.push(exit); + } const tail = fn.body.slice(i); - if (fn.params.length === 1) return this.getTail(tail, receiver); - if (fn.params.length !== 2) return null; - const second = fn.params[1]!.localId; - return this.setTail(tail, receiver, second) ?? this.keyedTail(tail, receiver, second); + let site: JsIcSite | null; + if (fn.params.length === 1) site = this.getTail(tail, receiver); + else if (fn.params.length !== 2 || second === undefined) return null; + else site = this.setTail(tail, receiver, second) ?? this.keyedTail(tail, receiver, second); + if (site === null || site.kind === "keyed") return exits.length === 0 ? site : null; + // An exit answers plain values with the operation the site caches. + return exits.every((exit) => exit.kind === site.kind && exit.key === site.key) ? site : null; + } + + /** `if (!classCapsule(receiver)) { [if (!funcKind(receiver))] OP }`, no + * else: an exit that runs the helper's generic operation directly on a + * value no class table matches. OP is `return [js.fromDyn](dynKeyGet(R, + * K))` or `dyn.keySet(R, K, value); return`, with R the receiver (or a + * bag of it). */ + private plainExit(s: IrStmt, receiver: string, value: string | undefined): JsIcSite | null { + const negated = (e: IrExpr, test: "classCapsule" | "funcKind"): boolean => + e.kind === "dynTest" && e.test === test && e.negated === true && isVar(e.value, receiver); + if (s.kind !== "if" || s.else_ !== null || !negated(s.cond, "classCapsule")) return null; + let body = s.then; + const inner = body[0]; + if ( + body.length === 1 && + inner?.kind === "if" && + inner.else_ === null && + negated(inner.cond, "funcKind") + ) + body = inner.then; + const [first, ret] = body; + if (body.length === 1 && first?.kind === "return") { + const read = unwrapFromDyn(first.value); + if ( + read?.kind !== "dynKeyGet" || + read.key.kind !== "strLit" || + !this.bagOf(read.value, receiver) + ) + return null; + return { kind: "get", key: read.key.value }; + } + if ( + body.length !== 2 || + first?.kind !== "exprStmt" || + ret?.kind !== "return" || + ret.value !== null + ) + return null; + const call = first.expr; + if (call.kind !== "libCall" || call.fn !== "dyn.keySet" || call.args.length !== 3) return null; + const [bag, key, stored] = call.args; + if (!bag || !this.bagOf(bag, receiver) || key?.kind !== "strLit") return null; + if (value === undefined || !isVar(stored, value)) return null; + return { kind: "set", key: key.value }; } /** `varDecl bag = BAG(receiver); return [js.fromDyn](bag === receiver ? @@ -109,10 +164,14 @@ export class JsIcSites { /** `if (typeof key === "symbol") return dynKeyGet(BAG(receiver), key); * if (receiver == null) return dynKeyGet(receiver, key); * const s = String(key); return dynKeyGet(BAG(receiver), s)`, each read - * possibly under js.fromDyn. */ + * possibly under js.fromDyn. Capsule branches may precede the final read: + * a program that declares classes looks the string key up in each class's + * key table there. */ private keyedTail(tail: IrStmt[], receiver: string, key: string): JsIcSite | null { - const [symbol, nullish, decl, ret] = tail; - if (tail.length !== 4) return null; + const [symbol, nullish, decl] = tail; + const ret = tail[tail.length - 1]; + if (tail.length < 4) return null; + if (!tail.slice(3, -1).every((s) => this.capsuleBranch(s, receiver))) return null; if (!this.guard(symbol, "symbol", key, (read) => this.bagOf(read, receiver), key)) return null; if (!this.guard(nullish, "nullish", receiver, (read) => isVar(read, receiver), key)) return null; @@ -159,9 +218,12 @@ export class JsIcSites { } /** Whether `e` is the receiver itself for every ordinary object or array: - * the receiver, or a chain of property-bag helpers applied to it. */ + * the receiver, or a chain of property-bag helpers applied to it. A + * conversion between the dyn and jsval forms keeps the object it holds. */ private bagOf(e: IrExpr, receiver: string): boolean { if (isVar(e, receiver)) return true; + if (e.kind === "libCall" && (e.fn === "js.toDyn" || e.fn === "js.fromDyn")) + return e.args.length === 1 && this.bagOf(e.args[0]!, receiver); return ( e.kind === "call" && e.args.length === 1 && @@ -182,6 +244,13 @@ export class JsIcSites { return ret.kind === "return" && ret.value !== null && this.bagOf(ret.value, param); } + /** `param = `, such as the subclass normalization + * `value = %dyn.class.normalize(value)`, which leaves an ordinary object + * or array in `param`. */ + private rebinding(s: IrStmt, param: string): boolean { + return s.kind === "assign" && s.localId === param && this.bagOf(s.value, param); + } + /** A property-bag helper that returns its argument unless a capsule * branch is taken. */ private identityHelper(name: string): boolean { @@ -204,7 +273,10 @@ export class JsIcSites { identity = fn.body .slice(0, -1) - .every((s) => this.capsuleBranch(s, param) || this.kindExit(s, param)) && + .every( + (s) => + this.capsuleBranch(s, param) || this.kindExit(s, param) || this.rebinding(s, param), + ) && last?.kind === "return" && last.value !== null && this.bagOf(last.value, param); diff --git a/packages/compiler/src/ir/node-index.ts b/packages/compiler/src/ir/node-index.ts new file mode 100644 index 000000000..2aac7df92 --- /dev/null +++ b/packages/compiler/src/ir/node-index.ts @@ -0,0 +1,214 @@ +/* One walk over a module's function bodies, recorded for analyses that + * only collect facts from nodes of a few kinds. Each such analysis used to + * walk every body itself; reading the recorded kind codes instead visits + * the same nodes in the same order without recursing through the tree. */ +import { InternalCompilerError } from "../errors.js"; +import type { IrExpr, IrFunction, IrModule, IrStmt, IrType } from "./ir.js"; +import { everyStmtList, everyTypeChild } from "./traverse.js"; + +/** An expression or statement kind. The two sets of kinds are disjoint. */ +export type IrNodeKind = IrExpr["kind"] | IrStmt["kind"]; + +/** The code of a kind no node has: it matches no recorded node. */ +const ABSENT = 255; + +/** Every expression and statement of each function body of a module, in + * the order everyStmtList visits them (a node before its children), with + * a kind code per node and, per function, the set of kinds present. + * + * The bodies must not change while the index is in use. Its owner (the + * LLVM emitter) builds it before its analyses run; no later pass rewrites + * a recorded body in place, and a rewritten copy of a function is walked. */ +export class IrNodeIndex { + /** Every type object the typed module traversal (everyModuleNode) + * reaches, nested ones included, each once. */ + readonly types: readonly IrType[]; + private readonly nodes: (IrExpr | IrStmt)[] = []; + private readonly codes: Uint8Array; + private readonly kindCodes = new Map(); + /** By function object: an analysis may see a rewritten copy of a + * function under the same name, whose nodes are not recorded here. */ + private readonly functionIndex = new Map(); + /** nodes[starts[i]] .. nodes[starts[i + 1] - 1] belong to function i. */ + private readonly starts: Int32Array; + /** Per function, eight words of kind-present bits. */ + private readonly present: Uint32Array; + /** By the kind list (or excluded set) a scan was made from. */ + private readonly scans = new Map(); + /** By kind code: 1 for statement kinds. */ + private readonly statements = new Uint8Array(256); + + constructor(mod: IrModule) { + const nodes = this.nodes; + const types: IrType[] = []; + const seenTypes = new Set(); + // A type object's children never change: one visit covers its tree. + const addType = (type: IrType): boolean => { + if (seenTypes.has(type)) return true; + seenTypes.add(type); + types.push(type); + return everyTypeChild(type, addType); + }; + const statementKinds = new Set(); + const visitor = { + expr: (node: IrExpr): boolean => { + nodes.push(node); + addType(node.type); + return true; + }, + stmt: (node: IrStmt): boolean => { + nodes.push(node); + statementKinds.add(node.kind); + return true; + }, + }; + const functions = mod.functions; + this.starts = new Int32Array(functions.length + 1); + for (let f = 0; f < functions.length; f++) { + this.functionIndex.set(functions[f]!, f); + this.starts[f] = nodes.length; + everyStmtList(functions[f]!.body, visitor); + } + this.starts[functions.length] = nodes.length; + // The other type slots everyModuleNode visits. + for (const fn of functions) { + for (const param of fn.params) addType(param.type); + addType(fn.returnType); + for (const local of fn.locals) addType(local.type); + for (const capture of fn.captures ?? []) addType(capture.type); + for (const capture of fn.classCaptures ?? []) addType(capture.type); + if (fn.generator !== undefined) { + addType(fn.generator.yieldT); + addType(fn.generator.nextT); + addType(fn.generator.resultType); + } + } + for (const cls of mod.classes ?? []) { + for (const capture of cls.localCaptures ?? []) addType(capture.type); + for (const field of cls.fields) addType(field.type); + } + for (const global of mod.globals ?? []) addType(global.type); + for (const record of mod.records ?? []) { + for (const field of record.fields) addType(field.type); + if (record.indexValue !== undefined) addType(record.indexValue); + } + for (const union of mod.unions ?? []) for (const arm of union.arms) addType(arm); + this.types = types; + const codes = new Uint8Array(nodes.length); + const present = new Uint32Array(functions.length * 8); + let previousKind = ""; + let previousCode = ABSENT; + for (let f = 0; f < functions.length; f++) { + const end = this.starts[f + 1]!; + for (let i = this.starts[f]!; i < end; i++) { + const kind = nodes[i]!.kind; + if (kind !== previousKind) { + let code = this.kindCodes.get(kind); + if (code === undefined) { + code = this.kindCodes.size; + if (code >= ABSENT) throw new InternalCompilerError("more IR node kinds than codes"); + this.kindCodes.set(kind, code); + } + previousKind = kind; + previousCode = code; + } + codes[i] = previousCode; + present[f * 8 + (previousCode >>> 5)]! |= 1 << (previousCode & 31); + } + } + this.codes = codes; + this.present = present; + for (const [kind, code] of this.kindCodes) + if (statementKinds.has(kind)) this.statements[code] = 1; + } + + /** A scan for the nodes of `kinds`, for eachNodeOfKinds. Pass a + * constant list: scans are kept by the list's identity. */ + scan(kinds: readonly IrNodeKind[]): IrKindScan { + const known = this.scans.get(kinds); + if (known !== undefined) return known; + const codes: number[] = []; + for (const kind of kinds) { + const code = this.kindCodes.get(kind); + if (code !== undefined) codes.push(code); + } + return this.scanOf(kinds, codes); + } + + /** A scan for every expression. */ + expressions(): IrKindScan { + const known = this.scans.get(this.statements); + if (known !== undefined) return known; + const codes: number[] = []; + for (const code of this.kindCodes.values()) if (this.statements[code] === 0) codes.push(code); + return this.scanOf(this.statements, codes); + } + + /** A scan for the nodes of every kind but those in `excluded`. Pass a + * constant set: scans are kept by the set's identity. */ + scanExcept(excluded: ReadonlySet): IrKindScan { + const known = this.scans.get(excluded); + if (known !== undefined) return known; + const codes: number[] = []; + for (const [kind, code] of this.kindCodes) if (!excluded.has(kind)) codes.push(code); + return this.scanOf(excluded, codes); + } + + private scanOf(key: object, codes: readonly number[]): IrKindScan { + const table = new Uint8Array(256); + const words = new Uint32Array(8); + for (const code of codes) { + table[code] = 1; + words[code >>> 5]! |= 1 << (code & 31); + } + const scan = { index: this, table, words }; + this.scans.set(key, scan); + return scan; + } + + /** Calls `note` with each recorded node of `fn` whose kind `scan` names, + * in walk order, until it answers false. False (nothing called) when + * `fn` is not recorded. */ + visit(fn: IrFunction, scan: IrKindScan, note: IrNodeNote): boolean { + const f = this.functionIndex.get(fn); + if (f === undefined) return false; + let any = 0; + for (let w = 0; w < 8; w++) any |= this.present[f * 8 + w]! & scan.words[w]!; + if (any === 0) return true; + const { nodes, codes, statements } = this; + const table = scan.table; + const end = this.starts[f + 1]!; + for (let i = this.starts[f]!; i < end; i++) { + const code = codes[i]!; + if (table[code] !== 0 && note(nodes[i]!, statements[code] !== 0) === false) break; + } + return true; + } +} + +/** Receives a node and whether it is a statement; false stops the visit. */ +export type IrNodeNote = (node: IrExpr | IrStmt, statement: boolean) => boolean | void; + +/** The kinds one analysis reads, resolved against one index. */ +export interface IrKindScan { + readonly index: IrNodeIndex; + readonly table: Uint8Array; + readonly words: Uint32Array; +} + +/** Calls `note` for the nodes of `fn` in walk order (everyStmtList's) + * until it answers false: with `scan`, only those of its kinds, read from + * the index; without one (or for a function the index did not record), + * every node, by walking. `note` must ignore the kinds the scan does not + * name, so both answer the same. */ +export function eachNodeOfKinds( + fn: IrFunction, + scan: IrKindScan | undefined, + note: IrNodeNote, +): void { + if (scan !== undefined && scan.index.visit(fn, scan, note)) return; + everyStmtList(fn.body, { + expr: (node) => note(node, false) !== false, + stmt: (node) => note(node, true) !== false, + }); +} diff --git a/packages/compiler/src/ir/runtime-features.ts b/packages/compiler/src/ir/runtime-features.ts index 819cfbee0..e1e7e28be 100644 --- a/packages/compiler/src/ir/runtime-features.ts +++ b/packages/compiler/src/ir/runtime-features.ts @@ -2,12 +2,14 @@ import type { IrClassDef, IrExpr, IrModule, IrRecordShape, IrType, IrUnionDef } import type { IrLibFn } from "./builtin-signatures.js"; import { isDirentShape, + recordTextCodecClass, RUNTIME_EMITTER_CLASS, RUNTIME_EVENT_CLASS, RUNTIME_EVENT_TARGET_CLASS, RUNTIME_STREAM_CLASSES, } from "./ir.js"; import { everyModuleNode } from "./traverse.js"; +import { eachNodeOfKinds, type IrNodeIndex } from "./node-index.js"; import { JsIcSites, hasJsIcSite } from "./js-ic.js"; const DYN_BUFFER_NUM_METHODS: ReadonlySet = new Set( @@ -219,6 +221,8 @@ const STREAM_LIB_FNS: ReadonlySet = new Set([ "stream.chunkDone", "stream.objectMode", "stream.decodeStrings", + "stream.optionValues", + "stream.checkEncodingOption", "stream.duplexFromGen", "stream.duplexFromDyn", "stream.duplexFromPromise", @@ -400,8 +404,10 @@ export function isDynUndefined(e: IrExpr): boolean { return e.kind === "dynFrom" && e.value.kind === "unitLit" && e.value.unit === "undefined"; } -export function moduleRuntimeFeatures(mod: IrModule): RuntimeFeatures { - return scanRuntimeFeatures(mod); +/** With `nodes` (the module's node index), the scan reads the recorded + * expressions and types instead of walking the module. */ +export function moduleRuntimeFeatures(mod: IrModule, nodes?: IrNodeIndex): RuntimeFeatures { + return scanRuntimeFeatures(mod, undefined, nodes); } /** The module's record shapes, unions and class layouts by id. */ @@ -410,7 +416,10 @@ interface TypeTables { unions: Map; classes: Map; /** mentionsDgramSocket's carriers, computed on first use. */ - dgramCarriers?: DgramCarriers; + dgramCarriers?: TypeCarriers; + /** mentionsTextCodec's and mentionsTextDecoder's carriers. */ + textCodecCarriers?: TypeCarriers; + textDecoderCarriers?: TypeCarriers; } /** Whether boxing a value of type `t` as dyn boxes a promise, which goes @@ -474,30 +483,42 @@ function boxesPromise(t: IrType, tables: TypeTables, seen: Set): boolean } } -/** The record shapes, unions and classes whose values carry a dgram socket - * (directly or through other members), computed once per module. */ -interface DgramCarriers { +/** The record shapes, unions and classes whose values carry a leaf type + * (directly or through other members), computed once per module and leaf. */ +interface TypeCarriers { records: Set; unions: Set; anyClass: boolean; } -/** Whether `t` carries a dgram socket, given the carrier sets so far: - * structural through functions (parameters and result), promises and - * collections; named records, unions and class instances by the sets. */ -function carriesDgram(t: IrType, carriers: DgramCarriers): boolean { +/** The leaf a carrier set tracks: a dgram socket, a codec record. */ +type CarrierLeaf = (t: IrType, tables: TypeTables) => boolean; + +/** Whether `t` carries a leaf, given the carrier sets so far: structural + * through functions (parameters and result), promises and collections; + * named records, unions and class instances by the sets. */ +function carriesLeaf( + t: IrType, + carriers: TypeCarriers, + leaf: CarrierLeaf, + tables: TypeTables, +): boolean { + if (leaf(t, tables)) return true; switch (t.kind) { - case "dgramSocket": - return true; case "func": - return t.params.some((p) => carriesDgram(p, carriers)) || carriesDgram(t.ret, carriers); + return ( + t.params.some((p) => carriesLeaf(p, carriers, leaf, tables)) || + carriesLeaf(t.ret, carriers, leaf, tables) + ); case "promise": - return carriesDgram(t.inner, carriers); + return carriesLeaf(t.inner, carriers, leaf, tables); case "array": case "set": - return carriesDgram(t.elem, carriers); + return carriesLeaf(t.elem, carriers, leaf, tables); case "map": - return carriesDgram(t.key, carriers) || carriesDgram(t.value, carriers); + return ( + carriesLeaf(t.key, carriers, leaf, tables) || carriesLeaf(t.value, carriers, leaf, tables) + ); case "record": return carriers.records.has(t.shapeId); case "union": @@ -510,25 +531,29 @@ function carriesDgram(t: IrType, carriers: DgramCarriers): boolean { } } -/** The module's dgram carriers: a fixpoint over its records, unions and +/** The module's carriers of `leaf`: a fixpoint over its records, unions and * class fields. */ -function dgramCarriers(tables: TypeTables): DgramCarriers { - const carriers: DgramCarriers = { records: new Set(), unions: new Set(), anyClass: false }; +function leafCarriers(tables: TypeTables, leaf: CarrierLeaf): TypeCarriers { + const carriers: TypeCarriers = { records: new Set(), unions: new Set(), anyClass: false }; + const carries = (t: IrType): boolean => carriesLeaf(t, carriers, leaf, tables); for (let changed = true; changed;) { changed = false; for (const [id, shape] of tables.records) - if (!carriers.records.has(id) && shape.fields.some((f) => carriesDgram(f.type, carriers))) { + if ( + !carriers.records.has(id) && + (leaf({ kind: "record", shapeId: id }, tables) || shape.fields.some((f) => carries(f.type))) + ) { carriers.records.add(id); changed = true; } for (const [id, def] of tables.unions) - if (!carriers.unions.has(id) && def.arms.some((a) => carriesDgram(a, carriers))) { + if (!carriers.unions.has(id) && def.arms.some(carries)) { carriers.unions.add(id); changed = true; } if ( !carriers.anyClass && - [...tables.classes.values()].some((c) => c.fields.some((f) => carriesDgram(f.type, carriers))) + [...tables.classes.values()].some((c) => c.fields.some((f) => carries(f.type))) ) { carriers.anyClass = true; changed = true; @@ -537,13 +562,42 @@ function dgramCarriers(tables: TypeTables): DgramCarriers { return carriers; } +const dgramLeaf: CarrierLeaf = (t) => t.kind === "dgramSocket"; + /** Whether converting a value of type `t` to or from dyn converts a dgram * socket, which goes through scr_dgram_dyn.c: the socket itself, or a * function, collection, record, union, promise or class instance that * carries one (boxed functions convert their parameters too). */ function mentionsDgramSocket(t: IrType, tables: TypeTables): boolean { - tables.dgramCarriers ??= dgramCarriers(tables); - return carriesDgram(t, tables.dgramCarriers); + tables.dgramCarriers ??= leafCarriers(tables, dgramLeaf); + return carriesLeaf(t, tables.dgramCarriers, dgramLeaf, tables); +} + +const textCodecLeaf: CarrierLeaf = (t, tables) => { + if (t.kind !== "record") return false; + const shape = tables.records.get(t.shapeId); + return shape !== undefined && recordTextCodecClass(shape) !== null; +}; + +const textDecoderLeaf: CarrierLeaf = (t, tables) => { + if (t.kind !== "record") return false; + const shape = tables.records.get(t.shapeId); + return shape !== undefined && recordTextCodecClass(shape) === "TextDecoder"; +}; + +/** Whether converting a value of type `t` to or from dyn converts a + * TextEncoder or TextDecoder record, which boxes it as a handle of + * scr_text_codec_dyn.c (the same carrier walk as dgram sockets). */ +function mentionsTextCodec(t: IrType, tables: TypeTables): boolean { + tables.textCodecCarriers ??= leafCarriers(tables, textCodecLeaf); + return carriesLeaf(t, tables.textCodecCarriers, textCodecLeaf, tables); +} + +/** The TextDecoder half: its boxed decode runs scr_text_decode_stream, + * which reaches the legacy decoders. */ +function mentionsTextDecoder(t: IrType, tables: TypeTables): boolean { + tables.textDecoderCarriers ??= leafCarriers(tables, textDecoderLeaf); + return carriesLeaf(t, tables.textDecoderCarriers, textDecoderLeaf, tables); } /** The libCall fns served by the OPTIONAL events unit (scr_events.c): @@ -789,6 +843,15 @@ const RUNTIME_FEATURE_RULES = { (node.kind === "dynFrom" && mentionsDgramSocket(node.value.type, tables())) || (node.kind === "dynCheck" && mentionsDgramSocket(node.type, tables())), }), + /** A TextEncoder or TextDecoder record converts to or from an untyped + * value (its handle box, or a boxed function, record or collection that + * carries one): scr_text_codec_dyn.c. */ + textCodecDyn: rule({ + fns: ["text.codecIs"], + expr: (node, tables) => + (node.kind === "dynFrom" && mentionsTextCodec(node.value.type, tables())) || + (node.kind === "dynCheck" && mentionsTextCodec(node.type, tables())), + }), /** node:dns/promises: scr_dns_promises.c over the dgram unit. */ dnsPromises: rule({ fns: ["dns.promisesLookup", "dns.promisesLookupService"] }), dynAsync: rule({ @@ -937,7 +1000,12 @@ const RUNTIME_FEATURE_RULES = { prefixes: ["insp.", "console.", "test."], libCall: (node) => node.fn === "rl.terminal" && !isDynUndefined(node.args[3]!), }), - legacyTextDecoder: rule({ fns: [...LEGACY_TEXT_DECODER_LIB_FNS] }), + legacyTextDecoder: rule({ + fns: [...LEGACY_TEXT_DECODER_LIB_FNS], + expr: (node, tables) => + (node.kind === "dynFrom" && mentionsTextDecoder(node.value.type, tables())) || + (node.kind === "dynCheck" && mentionsTextDecoder(node.type, tables())), + }), /** The program calls mock.timers members: its timer calls go through * scr_test.c's mock-aware entry points. */ mockTimers: rule({ prefixes: ["test.mockTimers", "test.ctxMockTimers"] }), @@ -1051,6 +1119,7 @@ function noRuntimeFeatures(): RuntimeFeatures { dc: false, dgram: false, dgramDyn: false, + textCodecDyn: false, dnsPromises: false, dynAsync: false, dynBuiltinClass: false, @@ -1126,7 +1195,11 @@ for (const name of RUNTIME_FEATURE_NAMES) { if (rule.expr) FEATURE_EXPR_TESTS.push(name); } -function scanRuntimeFeatures(mod: IrModule, stopAt?: RuntimeFeatureName): RuntimeFeatures { +function scanRuntimeFeatures( + mod: IrModule, + stopAt?: RuntimeFeatureName, + nodes?: IrNodeIndex, +): RuntimeFeatures { const features = noRuntimeFeatures(); const scans: { name: RuntimeFeatureName; scan: RuleScan }[] = []; for (const name of RUNTIME_FEATURE_NAMES) { @@ -1161,7 +1234,18 @@ function scanRuntimeFeatures(mod: IrModule, stopAt?: RuntimeFeatureName): Runtim for (const name of FEATURES_BY_TYPE.get(node.kind) ?? []) features[name] = true; return keepGoing(); }; - if (keepGoing()) everyModuleNode(mod, { expr, stmt: keepGoing, type }); + // Each condition is a fact about one expression or one type (the rule + // scans combine such facts in no particular order), so the index's + // expressions and distinct types answer as the walk does. An early stop + // (stopAt) keeps the walk. + if (nodes !== undefined && stopAt === undefined) { + const expressions = nodes.expressions(); + for (const fn of mod.functions) + eachNodeOfKinds(fn, expressions, (node) => { + expr(node as IrExpr); + }); + for (const t of nodes.types) type(t); + } else if (keepGoing()) everyModuleNode(mod, { expr, stmt: keepGoing, type }); for (const { name, scan } of scans) if (scan.result()) features[name] = true; // Inline-cache sites of dynamic-tier functions use the tier's shapes // (scr_js_object.c, with the SCR_JS_TIER build of scr_json.c). diff --git a/packages/compiler/src/ir/validate.ts b/packages/compiler/src/ir/validate.ts index 02825f0de..a404c918a 100644 --- a/packages/compiler/src/ir/validate.ts +++ b/packages/compiler/src/ir/validate.ts @@ -5500,6 +5500,7 @@ function validateFunction( e.fn === "stream.destroyErr" || e.fn === "stream.objectMode" || e.fn === "stream.decodeStrings" || + e.fn === "stream.optionValues" || e.fn === "stream.duplexFromGen" || e.fn === "stream.duplexFromDyn" || e.fn === "stream.duplexifyReadable" || @@ -5510,6 +5511,7 @@ function validateFunction( e.fn === "stream.duplexFromGenSignal" || e.fn === "stream.duplexFromAsyncFnSignal" || e.fn === "stream.setConstruct" || + e.fn === "stream.replaceConstruct" || e.fn === "stream.setWritev" || e.fn === "stream.setWritevDefault" || e.fn === "stream.iteratorClose" || @@ -5872,6 +5874,7 @@ function validateFunction( e.fn === "readable.pushEncoding" || e.fn === "stream.objectMode" || e.fn === "stream.decodeStrings" || + e.fn === "stream.optionValues" || e.fn === "writable.writeEncoding" || e.fn === "writable.setDefaultEncoding" || e.fn === "stream.destroy" || diff --git a/packages/compiler/src/native/cli.ts b/packages/compiler/src/native/cli.ts index 21047b929..aa46944f5 100644 --- a/packages/compiler/src/native/cli.ts +++ b/packages/compiler/src/native/cli.ts @@ -16,7 +16,11 @@ import { createNativeTs7Api } from "../frontend/ts7/native-api.js"; import { resolveProvenanceSourcesWithParser } from "../frontend/provenance-core.js"; import { setCompilerReleaseVersion } from "../library/sidecar.js"; import { NativeCompiler } from "./compiler.js"; -import { openNativeCache } from "./cache.js"; +import { NATIVE_STRING_BYTES, openNativeCache } from "./cache.js"; +import { + checkerExecutableIdentity, + setLibDiagnosticsStore, +} from "../frontend/ts7/lib-diagnostics.js"; import { loadNativeToolchain } from "./toolchain.js"; import { buildSanitizedRuntime } from "./sanitizer.js"; import { runCompilerTask } from "./task.js"; @@ -53,6 +57,17 @@ async function main(): Promise { if (toolchain.declarationsRoot !== undefined) setDeclarationRoot(toolchain.declarationsRoot); setCompilerReleaseVersion(toolchain.compilerVersion); cache = openNativeCache((path, harden) => compilerNativePrivateDirectory(path, harden)); + if (cache !== null) { + const store = cache; + setLibDiagnosticsStore( + { + read: (key) => + store.read("lib-diagnostics", key, NATIVE_STRING_BYTES)?.toString("utf8") ?? null, + write: (key, text) => store.write("lib-diagnostics", key, text), + }, + checkerExecutableIdentity(toolchain.ts7Executable), + ); + } compiler = new NativeCompiler(toolchain, cache); return compiler; }; @@ -122,6 +137,7 @@ async function main(): Promise { dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, diff --git a/packages/compiler/src/native/prepare.ts b/packages/compiler/src/native/prepare.ts index f7f399f80..a238029dc 100644 --- a/packages/compiler/src/native/prepare.ts +++ b/packages/compiler/src/native/prepare.ts @@ -239,6 +239,7 @@ function validFeatures(value: unknown): value is NativeLinkFeatures { "dgram", "dnsPromises", "dgramDyn", + "textCodecDyn", "watch", "foreignFfi", "nodeTest", diff --git a/packages/compiler/src/native/sanitizer.test.ts b/packages/compiler/src/native/sanitizer.test.ts index 4c031a8e7..33a4c19b9 100644 --- a/packages/compiler/src/native/sanitizer.test.ts +++ b/packages/compiler/src/native/sanitizer.test.ts @@ -114,6 +114,7 @@ test.skipIf(process.platform !== "linux" && process.platform !== "darwin")( dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, diff --git a/packages/compiler/surface-manifest.json b/packages/compiler/surface-manifest.json index 81fb23ae2..3387217b5 100644 --- a/packages/compiler/surface-manifest.json +++ b/packages/compiler/surface-manifest.json @@ -1,6 +1,6 @@ { "schemaVersion": 2, - "compilerVersion": "0.4.2", + "compilerVersion": "0.4.3", "coverage": [ "Entries are projected mechanically from the compiler's own decision tables: the diagnostics registry, the unsupported-syntax dispatch tables, the stdlib and node-builtin lowering tables, and the supported-builtin-module list. Nothing is hand-maintained; the manifest regenerates byte-identically from the source tree at this version.", "Absence from this manifest means 'not projected', never 'unsupported'. Surfaces lowered through dedicated code paths rather than tables are not yet projected: console, JSON, Promise/async and the timer surface, the net/http/tls/https/dgram/dns/assert/test/stream/readline module member surfaces, template literals, the regex slice, global functions (parseInt, parseFloat, isNaN, isFinite), and the process surface outside its ambient slice.", @@ -612,9 +612,7 @@ "id": "node-builtin.crypto.getCipherInfo", "kind": "node-builtin", "name": "crypto.getCipherInfo", - "status": "unsupported", - "code": "SC2020", - "note": "cipher metadata has no lowering — createCipheriv and createDecipheriv support aes-128/192/256 in cbc, ctr, gcm, ecb, cfb, cfb8, cfb1, and ofb modes, des-ede3 and des-ede, chacha20, and chacha20-poly1305" + "status": "static" }, { "id": "node-builtin.crypto.getDiffieHellman", diff --git a/packages/compiler/test/int32-element-array-emission.test.ts b/packages/compiler/test/int32-element-array-emission.test.ts index 0b68a5d3e..605f64e04 100644 --- a/packages/compiler/test/int32-element-array-emission.test.ts +++ b/packages/compiler/test/int32-element-array-emission.test.ts @@ -77,6 +77,19 @@ function literalCounter(n: number): number { for (let i = 0; i < n; i++) a[0] = a[0]! + 1; return a[0]!; } +function walk(n: number, steps: number): number { + const next: number[] = []; + for (let i = 0; i < n; i++) next.push(i + 1 < n ? i + 1 : 0); + let at = next[0]!; + let total = 0; + for (let s = 0; s < steps; s++) { + const t = next[at]!; + next[at] = at; + at = t; + total = (total + at) | 0; + } + return total; +} function joined(n: number): string { const a = [n | 0, 2]; return a.join(","); @@ -91,6 +104,7 @@ console.log( shortCounter(), literalCounter(n), joined(n), + walk(n | 0, 5), ); `; @@ -119,6 +133,16 @@ test("arrays storing only int32 elements get 4-byte slots", async () => { expect(counts).not.toContain("popArray"); }); +test("an array that stores a cursor read back from its own elements gets 4-byte slots", async () => { + const llvm = await emitLlvm(source); + const walk = functions(llvm, "walk"); + expect(walk).not.toBe(""); + // `at` is assigned `next[at]!` in the loop and stored back into `next`: + // it holds an int32 whenever the array's elements are int32s. + expect(walk).toMatch(/@scr_arr_new_small\(i32 7\)/); + expect(walk).toMatch(/getelementptr inbounds i32, ptr %\w+, i64 %\w+\n\s+%\w+ = load i32/); +}); + test("arrays with a store that has no exact integer range, or that escape, keep doubles", async () => { const llvm = await emitLlvm(source); for (const name of ["fraction", "negativeZero", "joined"]) { @@ -196,3 +220,42 @@ test("only the candidates the proof keeps are rewritten", async () => { for (const fn of selected.functions) if (fn.name !== "compose") expect(fn).toBe(mod.functions.find((f) => f.name === fn.name)); }); + +const sharedSource = ` +const data: number[] = []; +for (let i = 0; i < 64; i++) data.push((i * 7) & 255); +function fnv(values: number[]): number { + let h = 2166136261; + for (let i = 0; i < values.length; i++) h = Math.imul(h ^ values[i]!, 16777619) >>> 0; + return h; +} +function mean(values: number[]): number { + let s = 0; + for (let i = 0; i < values.length; i++) s += values[i]!; + return s / values.length; +} +function first(values: number[]): number { + return values[0]!; +} +const ints = [1, 2, 3]; +const halves = [0.5, 1.5]; +const pick: (values: number[]) => number = first; +console.log(fnv(data), mean(ints), mean(halves), pick([4])); +`; + +test("arrays passed to helpers and held in module globals get 4-byte slots", async () => { + const mod = await lowerIr(sharedSource); + const proposed = markInt32ArrayCandidates(mod).int32ArrayCandidates ?? []; + // A global and the parameter every call passes it to; a function used as + // a value keeps its parameter. + expect(proposed).toEqual(expect.arrayContaining(["\0%g.e.data", "fnv\0values.0"])); + expect(proposed.some((key) => key.startsWith("first\0"))).toBe(false); + const llvm = await emitLlvm(sharedSource); + // The helper reads int32 slots; `mean`, also called with fractions, + // reads doubles. + expect(functions(llvm, "fnv")).toMatch( + /getelementptr inbounds i32, ptr %\w+, i64 %\w+\n\s+%\w+ = load i32/, + ); + expect(functions(llvm, "mean")).not.toMatch(/getelementptr inbounds i32, ptr/); + expect(llvm).toMatch(/@scr_arr_new(_small)?\(i32 7/); +}); diff --git a/packages/compiler/test/js-class-layouts.test.ts b/packages/compiler/test/js-class-layouts.test.ts index c8491f3e8..110d007ee 100644 --- a/packages/compiler/test/js-class-layouts.test.ts +++ b/packages/compiler/test/js-class-layouts.test.ts @@ -101,6 +101,51 @@ delete loose.label; expect(layout(deleted, "Shape").tracks).toBe(true); }); +test("freezing values other than class instances keeps JavaScript class layouts fixed", async () => { + const module = await lower(`${SHAPES} +/** @param {{ value: any }} data */ +function settle(data) { + data.value = Object.freeze(data.value); + Object.seal(data); + return Object.preventExtensions(data.value); +} +settle({ value: { sides: 1 } }); +const square = new Square(); +Object.freeze(square); +`); + expect(layout(module, "Shape").tracks).toBe(false); + expect(layout(module, "Square").tracks).toBe(false); +}); + +test("accessors that share no field's name keep JavaScript class layouts fixed", async () => { + const module = await lower(` +class Path { + constructor(key) { + this.cache = []; + this.key = key; + } + get path() { + if (!this.cache.length) this.cache.push(this.key); + return this.cache; + } +} +class Named extends Path { + constructor(key) { + super(key); + this.label = "named"; + } + /** @param {string} value */ + set label2(value) { + this.label = value; + } +} +console.log(new Named("k").path.length); +`); + expect(layout(module, "Path")).toEqual({ fields: "cache:dyn key:dyn", tracks: false }); + expect(layout(module, "Named").tracks).toBe(false); + expect(layout(module, "Named").fields).toBe("cache:dyn key:dyn label:string"); +}); + test("JavaScript subclass fields keep their types when the base constructor only assigns fields", async () => { const module = await lower(` class Base { diff --git a/packages/compiler/test/js-ic-emission.test.ts b/packages/compiler/test/js-ic-emission.test.ts index 47d641bff..54ea13d26 100644 --- a/packages/compiler/test/js-ic-emission.test.ts +++ b/packages/compiler/test/js-ic-emission.test.ts @@ -64,6 +64,50 @@ console.log(read(o), at(dyn([5, 6]), 1), call(o), o.n); } }); +test("property sites keep their caches in programs that declare classes and subclasses", async () => { + const dir = await mkdtemp(join(tmpdir(), "scriptc-js-ic-class-")); + try { + const entry = join(dir, "main.js"); + const outPath = join(dir, "main.ir.json"); + // Declared classes give every property helper class branches and + // property-bag lookups, and a subclass adds a normalization step; plain + // objects still read through the caches. + await writeFile( + entry, + ` +class Point { + constructor(x) { this.x = x; } +} +class Tagged extends Point { + constructor(x, tag) { super(x); this.tag = tag; } +} +function dyn(v) { return v; } +function read(o) { return o.x; } +function write(o, v) { o.x = v; } +function at(a, i) { return a[i]; } +var o = dyn({ x: 1 }); +var p = dyn(new Point(4)); +var t = dyn(new Tagged(6, "t")); +write(o, 3); +write(p, 5); +write(t, 7); +console.log(read(o), read(p), read(t), at(dyn([5, 6]), 1), at(p, "x"), at(t, "tag")); +`, + ); + const result = await compile(entry, { outDir: dir, outPath, outputKind: "ir" }); + if (!result.ok) + throw new Error(result.diagnostics.map((d) => `${d.code}: ${d.message}`).join("\n")); + const mod = deserializeModule(await readFile(outPath, "utf8")); + expect(validateModule(mod)).toEqual([]); + const llvm = emitLlvmModule(mod); + expect(body(llvm, "read")).toContain("@scr_js_load_probe("); + expect(body(llvm, "write")).toContain("@scr_js_store_probe("); + expect(body(llvm, "at")).toContain("@scr_js_keyed_probe("); + } finally { + await rm(dir, { recursive: true, force: true }); + } +}); + test("typed programs have no cache sites and keep the ordinary runtime", async () => { const dir = await mkdtemp(join(tmpdir(), "scriptc-js-ic-typed-")); try { diff --git a/packages/compiler/test/native-codegen-integration.test.ts b/packages/compiler/test/native-codegen-integration.test.ts index d398fa1d3..cecebb49c 100644 --- a/packages/compiler/test/native-codegen-integration.test.ts +++ b/packages/compiler/test/native-codegen-integration.test.ts @@ -378,7 +378,7 @@ done: expect(await symbols(helperObject, ["-u"])).toEqual(await symbols(clangObject, ["-u"])); expect(await symbols(helperObject, ["-gU"])).toEqual(await symbols(clangObject, ["-gU"])); expect(await symbols(helperObject, ["-gU"])).toEqual(["0000000000000000 T _main"]); - expect(await symbols(helperObject, ["-u"])).toContain("_scr_runtime_abi_v36"); + expect(await symbols(helperObject, ["-u"])).toContain("_scr_runtime_abi_v37"); const clangExe = join(dir, "clang-program"); const linkDriver = join(dir, "clang-driver.c"); @@ -712,7 +712,7 @@ done: .split("\n"); expect(undefinedSymbols).toContain("_sf_scale"); expect(undefinedSymbols).toContain("_sf_callback_mix"); - expect(undefinedSymbols).toContain("_scr_runtime_abi_v36"); + expect(undefinedSymbols).toContain("_scr_runtime_abi_v37"); }); test("helper and clang object paths retain the same outbound FFI C ABI", async () => { diff --git a/packages/compiler/test/node-index.test.ts b/packages/compiler/test/node-index.test.ts new file mode 100644 index 000000000..23fbc5203 --- /dev/null +++ b/packages/compiler/test/node-index.test.ts @@ -0,0 +1,157 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { beforeAll, expect, test } from "vitest"; +import { compile, deserializeModule } from "../src/index.js"; +import type { IrExpr, IrModule, IrStmt, IrType } from "../src/ir/ir.js"; +import { eachNodeOfKinds, IrNodeIndex, type IrNodeKind } from "../src/ir/node-index.js"; +import { everyModuleNode, everyStmtList } from "../src/ir/traverse.js"; + +async function lowerSource(source: string): Promise { + const dir = await mkdtemp(join(tmpdir(), "scriptc-node-index-")); + try { + const entry = join(dir, "main.ts"); + await writeFile(entry, source); + const outPath = join(dir, "main.ir.json"); + const result = await compile(entry, { outPath, outDir: dir, outputKind: "ir" }); + if (!result.ok) throw new Error(result.diagnostics.map((d) => d.message).join("\n")); + return deserializeModule(await readFile(outPath, "utf8")); + } finally { + await rm(dir, { recursive: true, force: true }); + } +} + +let mod: IrModule; +let index: IrNodeIndex; + +beforeAll(async () => { + mod = await lowerSource(` +class Shape { + constructor(public readonly name: string, protected sides: number[]) {} + area(): number { return 0; } +} +class Square extends Shape { + constructor(private side: number) { super("square", [side, side, side, side]); } + override area(): number { return this.side * this.side; } +} +type Result = { ok: true; value: number } | { ok: false; error: string }; +function parse(text: string): Result { + const n = Number(text); + return Number.isNaN(n) ? { ok: false, error: "bad " + text } : { ok: true, value: n }; +} +function* count(limit: number): Generator { + for (let i = 0; i < limit; i++) yield i; +} +const totals = new Map(); +const seen = new Set(); +let calls = 0; +function record(key: string, value: number): void { + calls++; + totals.set(key, (totals.get(key) ?? 0) + value); + seen.add(key); +} +const adder = (k: number) => (x: number): number => x + k; +const inc = adder(1); +try { + for (const text of ["1", "x", "3"]) { + const r = parse(text); + if (r.ok) record("ok", inc(r.value)); + else throw new Error(r.error); + } +} catch (e) { + record("error", String(e).length); +} +let sum = 0; +for (const v of count(4)) sum += v; +const shapes: Shape[] = [new Square(2), new Shape("none", [])]; +console.log(sum, calls, [...totals], seen.size, shapes.map((s) => s.area()).join(",")); +`); + index = new IrNodeIndex(mod); +}); + +/** Every node everyStmtList visits in `body`, with whether it is a statement. */ +function walked(body: IrStmt[]): [IrExpr | IrStmt, boolean][] { + const out: [IrExpr | IrStmt, boolean][] = []; + everyStmtList(body, { + expr: (node) => (out.push([node, false]), true), + stmt: (node) => (out.push([node, true]), true), + }); + return out; +} + +function noted( + fn: IrModule["functions"][number], + scan: ReturnType | undefined, +): [IrExpr | IrStmt, boolean][] { + const out: [IrExpr | IrStmt, boolean][] = []; + eachNodeOfKinds(fn, scan, (node, statement) => { + out.push([node, statement]); + }); + return out; +} + +const NO_KINDS: ReadonlySet = new Set(); + +test("the index visits each body's nodes in everyStmtList's order", () => { + expect(mod.functions.length).toBeGreaterThan(5); + const all = index.scanExcept(NO_KINDS); + for (const fn of mod.functions) expect(noted(fn, all)).toEqual(walked(fn.body)); +}); + +const SOME_KINDS: readonly IrNodeKind[] = ["call", "varDecl", "closure", "return", "fieldGet"]; + +test("a scan visits only its kinds, in walk order, and the walk answers the same", () => { + const scan = index.scan(SOME_KINDS); + let visited = 0; + for (const fn of mod.functions) { + const expected = walked(fn.body).filter(([node]) => SOME_KINDS.includes(node.kind)); + expect(noted(fn, scan)).toEqual(expected); + // Without a scan, every node reaches `note`, as the walk visits them. + expect(noted(fn, undefined)).toEqual(walked(fn.body)); + visited += expected.length; + } + expect(visited).toBeGreaterThan(20); + expect(index.scan(SOME_KINDS)).toBe(scan); +}); + +test("a false answer stops the visit, with or without the index", () => { + const all = index.scanExcept(NO_KINDS); + for (const fn of mod.functions) { + for (const scan of [all, undefined]) { + let calls = 0; + eachNodeOfKinds(fn, scan, () => ++calls < 3); + expect(calls).toBe(Math.min(3, walked(fn.body).length)); + } + } +}); + +test("a function the index did not record is walked", () => { + const all = index.scanExcept(NO_KINDS); + for (const fn of mod.functions) { + const copy = { ...fn }; + expect(index.visit(copy, all, () => {})).toBe(false); + expect(noted(copy, all)).toEqual(walked(fn.body)); + } +}); + +test("the expression scan visits every expression", () => { + const expressions = index.expressions(); + for (const fn of mod.functions) + expect(noted(fn, expressions)).toEqual(walked(fn.body).filter(([, statement]) => !statement)); +}); + +test("the index's types are the type objects everyModuleNode visits, each once", () => { + const visited = new Set(); + everyModuleNode(mod, { + expr: () => true, + stmt: () => true, + type: (type) => (visited.add(type), true), + }); + expect(new Set(index.types)).toEqual(visited); + expect(index.types.length).toBe(visited.size); +}); + +test("a kind no node has matches nothing", () => { + const scan = index.scan(["regexLit"]); + for (const fn of mod.functions) expect(noted(fn, scan)).toEqual([]); +}); diff --git a/packages/compiler/test/package-shim-guard.test.ts b/packages/compiler/test/package-shim-guard.test.ts index a8620c1ce..9843739cd 100644 --- a/packages/compiler/test/package-shim-guard.test.ts +++ b/packages/compiler/test/package-shim-guard.test.ts @@ -51,6 +51,7 @@ describe("package shims compile without refusals", () => { expect(entries.map((e) => e.slice(shimRoot.length + 1))).toEqual([ "chalk/v5.ts", "fs-extra/index.ts", + "hono-node-server/v2.ts", "hot-shots/v8-unix-dgram.ts", "hot-shots/v8.ts", "undici/v5.ts", diff --git a/packages/compiler/test/source-output.test.ts b/packages/compiler/test/source-output.test.ts index 8b8e3883c..ce411f102 100644 --- a/packages/compiler/test/source-output.test.ts +++ b/packages/compiler/test/source-output.test.ts @@ -3,6 +3,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { afterEach, expect, test } from "vitest"; import { compile, deserializeModule, validateModule } from "../src/index.js"; +import { setLibDiagnosticsStore } from "../src/frontend/ts7/lib-diagnostics.js"; const dirs: string[] = []; afterEach(async () => { @@ -188,6 +189,38 @@ test("source outputs ignore invalid external compiler selection and create no ex } }); +test("source outputs neither read nor create declaration diagnostics a previous build stored", async () => { + const { dir, entry, outDir } = await fixture(); + const oldCache = process.env["SCRIPTC_CACHE_DIR"]; + const cache = join(dir, "must-not-exist"); + process.env["SCRIPTC_CACHE_DIR"] = cache; + // A store an earlier build of this process left configured. + const reads: string[] = []; + setLibDiagnosticsStore( + { read: (key) => (reads.push(key), null), write: () => {} }, + "earlier-checker", + ); + try { + for (const [kind, name] of [ + ["ir", "main.ir.json"], + ["llvm", "main.ll"], + ] as const) { + const result = await compile(entry, { + outDir, + outPath: join(outDir, name), + outputKind: kind, + }); + expect(result.ok).toBe(true); + } + expect(reads).toEqual([]); + await expect(readdir(cache)).rejects.toMatchObject({ code: "ENOENT" }); + } finally { + setLibDiagnosticsStore(null); + if (oldCache === undefined) delete process.env["SCRIPTC_CACHE_DIR"]; + else process.env["SCRIPTC_CACHE_DIR"] = oldCache; + } +}); + test.each(["wasm64-wasi", "totally-invalid"])( "source output rejects unsupported target %s without writing an artifact", async (target) => { diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7154-crypto-cipher-info.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7154-crypto-cipher-info.cjs.json new file mode 100644 index 000000000..65e5f162e --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7154-crypto-cipher-info.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7154-crypto-cipher-info.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7155-cipher-aes-ccm.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7155-cipher-aes-ccm.cjs.json new file mode 100644 index 000000000..a52818c74 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7155-cipher-aes-ccm.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7155-cipher-aes-ccm.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7156-cipher-key-wrap.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7156-cipher-key-wrap.cjs.json new file mode 100644 index 000000000..f2473c303 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7156-cipher-key-wrap.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7156-cipher-key-wrap.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7157-crypto-cipher-info-typed.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7157-crypto-cipher-info-typed.ts.json new file mode 100644 index 000000000..340d6e7b1 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7157-crypto-cipher-info-typed.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7157-crypto-cipher-info-typed.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7260-teardown-segments.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7260-teardown-segments.ts.json new file mode 100644 index 000000000..a7eac099f --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7260-teardown-segments.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7260-teardown-segments.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7264-array-dense-paired-loops.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7264-array-dense-paired-loops.ts.json new file mode 100644 index 000000000..8396d237c --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7264-array-dense-paired-loops.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7264-array-dense-paired-loops.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7281-http-server-connect-tunnel.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7281-http-server-connect-tunnel.ts.json new file mode 100644 index 000000000..2328d508b --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7281-http-server-connect-tunnel.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7281-http-server-connect-tunnel.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7282-http2-once-add-listener.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7282-http2-once-add-listener.ts.json new file mode 100644 index 000000000..96c879197 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7282-http2-once-add-listener.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7282-http2-once-add-listener.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7287-http-held-request-backpressure.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7287-http-held-request-backpressure.ts.json new file mode 100644 index 000000000..3a4358d78 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7287-http-held-request-backpressure.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7287-http-held-request-backpressure.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7380-cycle-deferred-folded-key/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7380-cycle-deferred-folded-key/main.ts.json new file mode 100644 index 000000000..18ef785c5 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7380-cycle-deferred-folded-key/main.ts.json @@ -0,0 +1,8 @@ +{ + "order": [ + "/tests/corpus/7380-cycle-deferred-folded-key/util.ts", + "/tests/corpus/7380-cycle-deferred-folded-key/core.ts", + "/tests/corpus/7380-cycle-deferred-folded-key/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts.json new file mode 100644 index 000000000..65d9b19e9 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts.json @@ -0,0 +1,8 @@ +{ + "order": [ + "/tests/corpus/7381-cycle-deferred-reached-tdz/beta.ts", + "/tests/corpus/7381-cycle-deferred-reached-tdz/alpha.ts", + "/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7382-cycle-static-this-tdz/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7382-cycle-static-this-tdz/main.ts.json new file mode 100644 index 000000000..986683684 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7382-cycle-static-this-tdz/main.ts.json @@ -0,0 +1,8 @@ +{ + "order": [ + "/tests/corpus/7382-cycle-static-this-tdz/beta.ts", + "/tests/corpus/7382-cycle-static-this-tdz/alpha.ts", + "/tests/corpus/7382-cycle-static-this-tdz/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts.json new file mode 100644 index 000000000..69b6d63bb --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts.json @@ -0,0 +1,8 @@ +{ + "order": [ + "/tests/corpus/7383-cycle-class-expression-name-tdz/beta.ts", + "/tests/corpus/7383-cycle-class-expression-name-tdz/alpha.ts", + "/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7390-record-key-read-presence.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7390-record-key-read-presence.ts.json new file mode 100644 index 000000000..74bd7e30d --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7390-record-key-read-presence.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7390-record-key-read-presence.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7404-frozen-records.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7404-frozen-records.ts.json new file mode 100644 index 000000000..d85f2aab5 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7404-frozen-records.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7404-frozen-records.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7405-frozen-records-sloppy.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7405-frozen-records-sloppy.cjs.json new file mode 100644 index 000000000..3d87d019b --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7405-frozen-records-sloppy.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7405-frozen-records-sloppy.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7406-frozen-containers.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7406-frozen-containers.ts.json new file mode 100644 index 000000000..34fb3ba0e --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7406-frozen-containers.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7406-frozen-containers.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7445-crypto-key-pair-options.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7445-crypto-key-pair-options.mjs.json new file mode 100644 index 000000000..5d57afbef --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7445-crypto-key-pair-options.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7445-crypto-key-pair-options.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7446-crypto-rsa-key-pair-errors.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7446-crypto-rsa-key-pair-errors.ts.json new file mode 100644 index 000000000..f055ddab9 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7446-crypto-rsa-key-pair-errors.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7446-crypto-rsa-key-pair-errors.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7447-union-record-arm-width.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7447-union-record-arm-width.mjs.json new file mode 100644 index 000000000..68bd750d9 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7447-union-record-arm-width.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7447-union-record-arm-width.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7448-union-record-arm-width.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7448-union-record-arm-width.ts.json new file mode 100644 index 000000000..498ee884a --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7448-union-record-arm-width.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7448-union-record-arm-width.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7449-crypto-raw-keys.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7449-crypto-raw-keys.mjs.json new file mode 100644 index 000000000..48bf2dfcc --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7449-crypto-raw-keys.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7449-crypto-raw-keys.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7460-transform-writev.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7460-transform-writev.mjs.json new file mode 100644 index 000000000..1015e1354 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7460-transform-writev.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7460-transform-writev.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7461-transform-writev.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7461-transform-writev.cjs.json new file mode 100644 index 000000000..4d3c86215 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7461-transform-writev.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7461-transform-writev.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7462-transform-writev.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7462-transform-writev.ts.json new file mode 100644 index 000000000..9b42f2d60 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7462-transform-writev.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7462-transform-writev.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7463-stream-option-values.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7463-stream-option-values.ts.json new file mode 100644 index 000000000..5aa543dea --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7463-stream-option-values.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7463-stream-option-values.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7464-stream-option-values.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7464-stream-option-values.mjs.json new file mode 100644 index 000000000..2a438d418 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7464-stream-option-values.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7464-stream-option-values.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7465-transform-callback-values.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7465-transform-callback-values.ts.json new file mode 100644 index 000000000..17633d208 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7465-transform-callback-values.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7465-transform-callback-values.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7466-stream-construct-assign.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7466-stream-construct-assign.mjs.json new file mode 100644 index 000000000..f5bb52517 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7466-stream-construct-assign.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7466-stream-construct-assign.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7500-pipeline-callback-undefined.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7500-pipeline-callback-undefined.ts.json new file mode 100644 index 000000000..fe65b63f0 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7500-pipeline-callback-undefined.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7500-pipeline-callback-undefined.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7501-pipeline-callback-undefined.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7501-pipeline-callback-undefined.mjs.json new file mode 100644 index 000000000..8d52980fe --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7501-pipeline-callback-undefined.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7501-pipeline-callback-undefined.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7502-readline-cursor-undefined.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7502-readline-cursor-undefined.ts.json new file mode 100644 index 000000000..6fd23042e --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7502-readline-cursor-undefined.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7502-readline-cursor-undefined.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7543-var-list-declarator-order.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7543-var-list-declarator-order.js.json new file mode 100644 index 000000000..d193e84ed --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7543-var-list-declarator-order.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7543-var-list-declarator-order.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7545-string-regex-checked-arguments.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7545-string-regex-checked-arguments.js.json new file mode 100644 index 000000000..03b3197c4 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7545-string-regex-checked-arguments.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7545-string-regex-checked-arguments.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7548-cjs-receiverless-builtin-calls/main.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7548-cjs-receiverless-builtin-calls/main.cjs.json new file mode 100644 index 000000000..f3b8ba419 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7548-cjs-receiverless-builtin-calls/main.cjs.json @@ -0,0 +1,7 @@ +{ + "order": [ + "/tests/corpus/7548-cjs-receiverless-builtin-calls/lib.cjs", + "/tests/corpus/7548-cjs-receiverless-builtin-calls/main.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7549-cjs-minified-esmodule-stamp/main.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7549-cjs-minified-esmodule-stamp/main.cjs.json new file mode 100644 index 000000000..08adae885 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7549-cjs-minified-esmodule-stamp/main.cjs.json @@ -0,0 +1,8 @@ +{ + "order": [ + "/tests/corpus/7549-cjs-minified-esmodule-stamp/stamped.cjs", + "/tests/corpus/7549-cjs-minified-esmodule-stamp/module-stamped.cjs", + "/tests/corpus/7549-cjs-minified-esmodule-stamp/main.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7553-var-for-in-of-zero-iterations.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7553-var-for-in-of-zero-iterations.js.json new file mode 100644 index 000000000..9f470ca37 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7553-var-for-in-of-zero-iterations.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7553-var-for-in-of-zero-iterations.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7580-int32-arrays-across-calls.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7580-int32-arrays-across-calls.ts.json new file mode 100644 index 000000000..33282756c --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7580-int32-arrays-across-calls.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7580-int32-arrays-across-calls.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7620-js-class-view-descriptors.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7620-js-class-view-descriptors.js.json new file mode 100644 index 000000000..2cb1ece22 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7620-js-class-view-descriptors.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7620-js-class-view-descriptors.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7621-js-class-field-methods.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7621-js-class-field-methods.js.json new file mode 100644 index 000000000..a7e5fdd14 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7621-js-class-field-methods.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7621-js-class-field-methods.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7640-child-spawn-before-fast-exit.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7640-child-spawn-before-fast-exit.ts.json new file mode 100644 index 000000000..0b82f6091 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7640-child-spawn-before-fast-exit.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7640-child-spawn-before-fast-exit.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7642-element-assignment-order.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7642-element-assignment-order.ts.json new file mode 100644 index 000000000..17369f496 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7642-element-assignment-order.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7642-element-assignment-order.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7733-property-bag-late-prototype.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7733-property-bag-late-prototype.js.json new file mode 100644 index 000000000..2b9778666 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7733-property-bag-late-prototype.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7733-property-bag-late-prototype.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7734-caught-instanceof-class-binding.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7734-caught-instanceof-class-binding.js.json new file mode 100644 index 000000000..9ce0ed4f0 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7734-caught-instanceof-class-binding.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7734-caught-instanceof-class-binding.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7738-private-logical-assignment-untyped.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7738-private-logical-assignment-untyped.js.json new file mode 100644 index 000000000..4865d3206 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7738-private-logical-assignment-untyped.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7738-private-logical-assignment-untyped.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7740-handle-method-values.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7740-handle-method-values.js.json new file mode 100644 index 000000000..ab7f75639 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7740-handle-method-values.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7740-handle-method-values.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7741-abort-signal-listener-value.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7741-abort-signal-listener-value.js.json new file mode 100644 index 000000000..c556e243e --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7741-abort-signal-listener-value.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7741-abort-signal-listener-value.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7742-response-clone-tee.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7742-response-clone-tee.ts.json new file mode 100644 index 000000000..a62bef2d0 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7742-response-clone-tee.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7742-response-clone-tee.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7743-transform-stream-identity.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7743-transform-stream-identity.ts.json new file mode 100644 index 000000000..0cfff542b --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7743-transform-stream-identity.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7743-transform-stream-identity.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7747-text-codec-untyped-slot.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7747-text-codec-untyped-slot.js.json new file mode 100644 index 000000000..23efb7db6 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7747-text-codec-untyped-slot.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7747-text-codec-untyped-slot.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7748-npm-any-bytes-receiver/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7748-npm-any-bytes-receiver/main.ts.json new file mode 100644 index 000000000..89db88211 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7748-npm-any-bytes-receiver/main.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7748-npm-any-bytes-receiver/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7749-text-codec-unknown-roundtrip.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7749-text-codec-unknown-roundtrip.ts.json new file mode 100644 index 000000000..dd9f068f3 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7749-text-codec-unknown-roundtrip.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7749-text-codec-unknown-roundtrip.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7790-js-class-integrity-layouts.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7790-js-class-integrity-layouts.js.json new file mode 100644 index 000000000..526b86044 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7790-js-class-integrity-layouts.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7790-js-class-integrity-layouts.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7791-js-class-accessor-layouts.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7791-js-class-accessor-layouts.js.json new file mode 100644 index 000000000..5350539f0 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7791-js-class-accessor-layouts.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7791-js-class-accessor-layouts.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7792-untyped-property-reads-mixed-receivers.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7792-untyped-property-reads-mixed-receivers.js.json new file mode 100644 index 000000000..1b28b3fbe --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7792-untyped-property-reads-mixed-receivers.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7792-untyped-property-reads-mixed-receivers.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7793-untyped-key-writes.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7793-untyped-key-writes.js.json new file mode 100644 index 000000000..b887e0fdb --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7793-untyped-key-writes.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7793-untyped-key-writes.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7794-untyped-array-iteration.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7794-untyped-array-iteration.js.json new file mode 100644 index 000000000..5ffe60f53 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7794-untyped-array-iteration.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7794-untyped-array-iteration.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7795-untyped-global-constructors.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7795-untyped-global-constructors.js.json new file mode 100644 index 000000000..48eb29e33 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7795-untyped-global-constructors.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7795-untyped-global-constructors.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7981-typed-array-paired-loops.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7981-typed-array-paired-loops.ts.json new file mode 100644 index 000000000..71fa483a8 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/7981-typed-array-paired-loops.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/7981-typed-array-paired-loops.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/class-brand-probe-keys.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/class-brand-probe-keys.ts.json new file mode 100644 index 000000000..fd60e3ff9 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/class-brand-probe-keys.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/class-brand-probe-keys.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/fetch-headers-copy-cache.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/fetch-headers-copy-cache.ts.json new file mode 100644 index 000000000..bd69651b1 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/fetch-headers-copy-cache.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/fetch-headers-copy-cache.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/string-last-index-of-ascii.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/string-last-index-of-ascii.ts.json new file mode 100644 index 000000000..1d3107cd5 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/string-last-index-of-ascii.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/string-last-index-of-ascii.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/stream-runtime-option-order.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/stream-runtime-option-order.ts.json new file mode 100644 index 000000000..40b64edac --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/stream-runtime-option-order.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/diagnostics/stream-runtime-option-order.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/7546-minified-esm-rules/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/7546-minified-esm-rules/main.ts.json new file mode 100644 index 000000000..d026fb402 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/7546-minified-esm-rules/main.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/fixtures/npm/cases/7546-minified-esm-rules/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/7547-minified-cjs-chunks/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/7547-minified-cjs-chunks/main.ts.json new file mode 100644 index 000000000..212551ccd --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/7547-minified-cjs-chunks/main.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/fixtures/npm/cases/7547-minified-cjs-chunks/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/hono-node-server/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/hono-node-server/main.ts.json new file mode 100644 index 000000000..58f450083 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/hono-node-server/main.ts.json @@ -0,0 +1,7 @@ +{ + "order": [ + "/packages/compiler/shims/hono-node-server/v2.ts", + "/tests/fixtures/npm/cases/hono-node-server/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/types-boundary-prototype/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/types-boundary-prototype/main.ts.json new file mode 100644 index 000000000..2abe1bd8f --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/types-boundary-prototype/main.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/fixtures/npm/cases/types-boundary-prototype/main.ts" + ], + "diags": [] +} diff --git a/packages/experimental/package.json b/packages/experimental/package.json index cfb4151a4..b3e8a6fb1 100644 --- a/packages/experimental/package.json +++ b/packages/experimental/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/experimental", - "version": "0.4.2", + "version": "0.4.3", "description": "Experimental scriptc APIs, with Node.js implementations: lifetime regions for ProgramLifetime objects, intern tables shared between threads, and per-thread slots on shared objects", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/llvm-darwin-arm64/package.json b/packages/llvm-darwin-arm64/package.json index 615fe9d81..ac99f1661 100644 --- a/packages/llvm-darwin-arm64/package.json +++ b/packages/llvm-darwin-arm64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/llvm-darwin-arm64", - "version": "0.4.2", + "version": "0.4.3", "description": "Pinned LLVM code-generation helper for scriptc on macOS arm64", "license": "Apache-2.0 WITH LLVM-exception", "homepage": "https://scriptc.dev", diff --git a/packages/llvm-darwin-x64/package.json b/packages/llvm-darwin-x64/package.json index fd6caf976..ca632dcd4 100644 --- a/packages/llvm-darwin-x64/package.json +++ b/packages/llvm-darwin-x64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/llvm-darwin-x64", - "version": "0.4.2", + "version": "0.4.3", "description": "Pinned LLVM code-generation helper for scriptc on macOS x64", "license": "Apache-2.0 WITH LLVM-exception", "homepage": "https://scriptc.dev", diff --git a/packages/llvm-linux-arm64-gnu/package.json b/packages/llvm-linux-arm64-gnu/package.json index 7d66257f4..566b38e40 100644 --- a/packages/llvm-linux-arm64-gnu/package.json +++ b/packages/llvm-linux-arm64-gnu/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/llvm-linux-arm64-gnu", - "version": "0.4.2", + "version": "0.4.3", "description": "Pinned LLVM code-generation helper for scriptc on Linux arm64", "license": "Apache-2.0 WITH LLVM-exception", "homepage": "https://scriptc.dev", diff --git a/packages/llvm-linux-arm64-musl/package.json b/packages/llvm-linux-arm64-musl/package.json index b3a01089d..a9288c2ca 100644 --- a/packages/llvm-linux-arm64-musl/package.json +++ b/packages/llvm-linux-arm64-musl/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/llvm-linux-arm64-musl", - "version": "0.4.2", + "version": "0.4.3", "description": "Statically linked LLVM code-generation helper for scriptc on Linux arm64 musl", "license": "Apache-2.0 WITH LLVM-exception", "homepage": "https://scriptc.dev", diff --git a/packages/llvm-linux-x64-gnu/package.json b/packages/llvm-linux-x64-gnu/package.json index 0b3264ec7..c55f51847 100644 --- a/packages/llvm-linux-x64-gnu/package.json +++ b/packages/llvm-linux-x64-gnu/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/llvm-linux-x64-gnu", - "version": "0.4.2", + "version": "0.4.3", "description": "Pinned LLVM code-generation helper for scriptc on Linux x64", "license": "Apache-2.0 WITH LLVM-exception", "homepage": "https://scriptc.dev", diff --git a/packages/llvm-linux-x64-musl/package.json b/packages/llvm-linux-x64-musl/package.json index 6c83d1cb9..b257085bf 100644 --- a/packages/llvm-linux-x64-musl/package.json +++ b/packages/llvm-linux-x64-musl/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/llvm-linux-x64-musl", - "version": "0.4.2", + "version": "0.4.3", "description": "Statically linked LLVM code-generation helper for scriptc on Linux x64 musl", "license": "Apache-2.0 WITH LLVM-exception", "homepage": "https://scriptc.dev", diff --git a/packages/llvm-win32-x64-msvc/package.json b/packages/llvm-win32-x64-msvc/package.json index 9fb682f0f..0f3f99331 100644 --- a/packages/llvm-win32-x64-msvc/package.json +++ b/packages/llvm-win32-x64-msvc/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/llvm-win32-x64-msvc", - "version": "0.4.2", + "version": "0.4.3", "description": "Pinned LLVM code-generation helper for scriptc on Windows x64", "license": "Apache-2.0 WITH LLVM-exception", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-android-arm64/package.json b/packages/runtime-android-arm64/package.json index 362fbc572..392ef1717 100644 --- a/packages/runtime-android-arm64/package.json +++ b/packages/runtime-android-arm64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-android-arm64", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc library runtime pack for Android arm64", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-darwin-arm64/package.json b/packages/runtime-darwin-arm64/package.json index a4176ed60..354526024 100644 --- a/packages/runtime-darwin-arm64/package.json +++ b/packages/runtime-darwin-arm64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-darwin-arm64", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for macOS arm64", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-darwin-arm64/scripts/build.mjs b/packages/runtime-darwin-arm64/scripts/build.mjs index 069cb66e9..e492c36b4 100644 --- a/packages/runtime-darwin-arm64/scripts/build.mjs +++ b/packages/runtime-darwin-arm64/scripts/build.mjs @@ -340,7 +340,7 @@ async function build() { package: packageManifest.name, version: packageManifest.version, target: RUNTIME_PACK_MATRIX.target, - runtime_abi: { version: 36, marker: "scr_runtime_abi_v36" }, + runtime_abi: { version: 37, marker: "scr_runtime_abi_v37" }, compiler: { command: compiler, identity: compilerVersion, diff --git a/packages/runtime-darwin-x64/package.json b/packages/runtime-darwin-x64/package.json index 91bc4182b..58aa358b2 100644 --- a/packages/runtime-darwin-x64/package.json +++ b/packages/runtime-darwin-x64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-darwin-x64", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for macOS x64", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-ios-arm64/package.json b/packages/runtime-ios-arm64/package.json index 2fdd68897..6193c5258 100644 --- a/packages/runtime-ios-arm64/package.json +++ b/packages/runtime-ios-arm64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-ios-arm64", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc library runtime pack for iOS arm64", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-ios-simulator-arm64/package.json b/packages/runtime-ios-simulator-arm64/package.json index cbdf96d1d..7d01c45d3 100644 --- a/packages/runtime-ios-simulator-arm64/package.json +++ b/packages/runtime-ios-simulator-arm64/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-ios-simulator-arm64", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc library runtime pack for iOS simulator arm64", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-linux-arm64-gnu/package.json b/packages/runtime-linux-arm64-gnu/package.json index e0d228925..419080c3a 100644 --- a/packages/runtime-linux-arm64-gnu/package.json +++ b/packages/runtime-linux-arm64-gnu/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-linux-arm64-gnu", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for Linux arm64 glibc", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-linux-arm64-musl/package.json b/packages/runtime-linux-arm64-musl/package.json index 70eee223f..2bae09884 100644 --- a/packages/runtime-linux-arm64-musl/package.json +++ b/packages/runtime-linux-arm64-musl/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-linux-arm64-musl", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for Linux arm64 musl", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-linux-x64-gnu/package.json b/packages/runtime-linux-x64-gnu/package.json index 5822fd21d..6749ddeea 100644 --- a/packages/runtime-linux-x64-gnu/package.json +++ b/packages/runtime-linux-x64-gnu/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-linux-x64-gnu", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for Linux x64 glibc", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-linux-x64-musl/package.json b/packages/runtime-linux-x64-musl/package.json index 1db34d4a7..9b4803406 100644 --- a/packages/runtime-linux-x64-musl/package.json +++ b/packages/runtime-linux-x64-musl/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-linux-x64-musl", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for Linux x64 musl", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-pack-common/runtime-pack-matrix.mjs b/packages/runtime-pack-common/runtime-pack-matrix.mjs index 285e767f4..0ab467236 100644 --- a/packages/runtime-pack-common/runtime-pack-matrix.mjs +++ b/packages/runtime-pack-common/runtime-pack-matrix.mjs @@ -49,6 +49,8 @@ export const BASE_RUNTIME_SOURCES = [ "scr_itable.c", "scr_slots.c", "scr_alloc.c", + // crypto.getCipherInfo's table; it links only into programs that call it. + "scr_cipher_info.c", ]; export const OPTIONAL_RUNTIME_SOURCES = [ @@ -146,6 +148,8 @@ export const OPTIONAL_RUNTIME_SOURCES = [ ["scr_dns_promises.c", all("dgram", "dnsPromises")], // Sockets as untyped values run over the dgram unit. ["scr_dgram_dyn.c", all("dgram", "dgramDyn")], + // TextEncoder and TextDecoder records boxed as untyped values. + ["scr_text_codec_dyn.c", "textCodecDyn"], ["scr_watch.c", "watch"], ["scr_ffi_queue.c", "foreignFfi"], ["scr_test.c", "nodeTest"], diff --git a/packages/runtime-pack-common/scripts/build.mjs b/packages/runtime-pack-common/scripts/build.mjs index f15233924..e22f2a37c 100644 --- a/packages/runtime-pack-common/scripts/build.mjs +++ b/packages/runtime-pack-common/scripts/build.mjs @@ -437,7 +437,7 @@ async function build() { package: packageManifest.name, version: packageManifest.version, target: matrix.target, - runtime_abi: { version: 36, marker: "scr_runtime_abi_v36" }, + runtime_abi: { version: 37, marker: "scr_runtime_abi_v37" }, compiler: { command: compiler, identity: compilerVersion, diff --git a/packages/runtime-wasm32-wasi/package.json b/packages/runtime-wasm32-wasi/package.json index 5bbd23755..63a95601d 100644 --- a/packages/runtime-wasm32-wasi/package.json +++ b/packages/runtime-wasm32-wasi/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-wasm32-wasi", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for WASI Preview 1", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime-win32-x64-msvc/package.json b/packages/runtime-win32-x64-msvc/package.json index df4a9a109..01bec7c31 100644 --- a/packages/runtime-win32-x64-msvc/package.json +++ b/packages/runtime-win32-x64-msvc/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime-win32-x64-msvc", - "version": "0.4.2", + "version": "0.4.3", "description": "Precompiled scriptc runtime pack for Windows x64 MSVC", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime/package.json b/packages/runtime/package.json index 9232266af..d367ad6bf 100644 --- a/packages/runtime/package.json +++ b/packages/runtime/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/runtime", - "version": "0.4.2", + "version": "0.4.3", "description": "scriptc native runtime sources and vendored dependencies", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/runtime/src/scr_array.c b/packages/runtime/src/scr_array.c index 91328a1e8..200c96ac0 100644 --- a/packages/runtime/src/scr_array.c +++ b/packages/runtime/src/scr_array.c @@ -1689,6 +1689,126 @@ void scr_arr_release(ScrArr *a) { double scr_arr_len(ScrArr *a) { return SCR_ARR_LEN_F64(a->len); } +/* ── frozen arrays ────────────────────────────────────────────────────── + * Node's TypeErrors for writes to a frozen array, shared by the emitted + * guards of published and frozen arrays and the checked-dynamic tier. */ +static void scr_frozen_throw(const char *before, double index, const char *after) { + char text[64]; + size_t n = scr_f64_to_str(index, text); + ScrJsonBuf buffer; + scr_jb_init(&buffer); + scr_jb_puts(&buffer, before); + scr_jb_write(&buffer, text, n); + scr_jb_puts(&buffer, after); + scr_throw_error(SCR_ERR_TYPE, scr_jb_finish(&buffer)); +} + +/* A relative array index argument (fill, copyWithin, splice): ToIntegerOrInfinity + * of `value`, counted from the end when negative, clamped to [0, len]. */ +static size_t scr_frozen_relative_index(double value, size_t len) { + if (value != value) return 0; + double n = trunc(value); + if (n < 0) return n + (double)len <= 0 ? 0 : (size_t)(n + (double)len); + return n >= (double)len ? len : (size_t)n; +} + +static void scr_frozen_read_only(double index) { + scr_frozen_throw("Cannot assign to read only property '", index, "' of object '[object Array]'"); +} + +static void scr_frozen_not_extensible(double index) { + scr_frozen_throw("Cannot add property ", index, ", object is not extensible"); +} + +static void scr_frozen_undeletable(double index) { + scr_frozen_throw("Cannot delete property '", index, "' of [object Array]"); +} + +/* Each case finds the operation's first write in the order the spec makes + * its writes. */ +bool scr_array_frozen_error(int op, size_t len, double x, double y, double z) { + switch (op) { + case 0: + if (x >= 0 && x < (double)len && x == (double)(size_t)x) scr_frozen_read_only(x); + else scr_frozen_not_extensible(x); + return true; + case 1: + if (y == 0) break; + scr_frozen_not_extensible((double)len); + return true; + case 2: + case 6: + if (len == 0) break; + if (op == 6 && len > 1) scr_frozen_read_only(0); + else scr_frozen_undeletable((double)(len - 1)); + return true; + case 3: break; + case 4: + if (y == 0) break; + scr_frozen_not_extensible(len ? (double)len + y - 1 : 0); + return true; + case 5: + if (!(x >= 0 && x < (double)len && x == (double)(size_t)x)) return false; + scr_frozen_undeletable(x); + return true; + case 7: + if (len < 2) return false; + scr_frozen_read_only(0); + return true; + case 8: { + size_t k = scr_frozen_relative_index(x, len), final = scr_frozen_relative_index(y, len); + if (k >= final) return false; + scr_frozen_read_only((double)k); + return true; + } + case 9: { + size_t to = scr_frozen_relative_index(x, len), from = scr_frozen_relative_index(y, len); + size_t final = scr_frozen_relative_index(z, len); + if (final <= from || len <= to) return false; + size_t count = final - from < len - to ? final - from : len - to; + /* An overlapping copy toward the end writes its last index first. */ + if (from < to && to < from + count) to += count - 1; + scr_frozen_read_only((double)to); + return true; + } + case 10: { + size_t start = scr_frozen_relative_index(x, len); + double count = y != y ? 0 : trunc(y); + size_t removed = count <= 0 ? 0 : count >= (double)(len - start) ? len - start : (size_t)count; + size_t items = (size_t)z; + if (items < removed) { + /* Elements after the removed ones move down, then the tail goes. */ + if (start < len - removed) scr_frozen_read_only((double)(start + items)); + else scr_frozen_undeletable((double)(len - 1)); + return true; + } + if (items > removed) { + /* Elements after the removed ones move up, past the end first. */ + if (len - removed > start) scr_frozen_not_extensible((double)(len - removed + items - 1)); + else if (start < len) scr_frozen_read_only((double)start); + else scr_frozen_not_extensible((double)start); + return true; + } + if (items > 0) { + scr_frozen_read_only((double)start); + return true; + } + break; + } + default: + return false; + } + static const char length[] = "Cannot assign to read only property 'length' of object '[object Array]'"; + scr_throw_error_msg(SCR_ERR_TYPE, length, sizeof length - 1); + return true; +} + +void scr_arr_freeze(ScrArr *a) { + if (a->rc != SIZE_MAX) a->spare |= SCR_ARR_FROZEN; +} + +bool scr_arr_is_frozen(const ScrArr *a) { return scr_arr_frozen(a); } + bool scr_arr_has(const ScrArr *a, double i) { size_t idx; if (!scr_arr_valid_index(i, &idx)) return scr_arr_prop_get(a, i, NULL); @@ -4990,6 +5110,8 @@ bool scr_arr_sort_dense(ScrArr *a, ScrClosure *f, uint32_t arity, bool owned, in if (a->rc == SIZE_MAX || elem == SCR_ELEM_CREF || !scr_arr_dense_values(a)) return false; size_t n = a->len; if (n < 2) return true; + /* A frozen array takes the snapshot path, whose write-back throws. */ + if (scr_arr_frozen(a)) return false; if (n > SIZE_MAX / 2 / sizeof(uint64_t)) scr_arr_oom(); ScrSortCompare c = scr_sort_compare_for(a, f, arity, order, key); /* Nothing else can reach a private copy: its own storage is sorted, and @@ -5012,15 +5134,12 @@ bool scr_arr_sort_dense(ScrArr *a, ScrClosure *f, uint32_t arity, bool owned, in free(src); return true; } - if (a->rc == SIZE_MAX) { - /* The comparator published the array: the first write fails as in Node - * (scr_throw_published_array's index write). */ + if (a->rc == SIZE_MAX || scr_arr_frozen(a)) { + /* The comparator published or froze the array: the first write fails + * as in Node. */ if (retained) for (size_t i = 0; i < n; i++) scr_elem_release_with(elem, ops, src[i]); - static const char assign[] = "Cannot assign to read only property '0' of object '[object Array]'"; - static const char add[] = "Cannot add property 0, object is not extensible"; - if (a->len > 0) scr_throw_error_msg(SCR_ERR_TYPE, assign, sizeof assign - 1); - else scr_throw_error_msg(SCR_ERR_TYPE, add, sizeof add - 1); + scr_array_frozen_error(0, a->len, 0, 0, 0); } else if (a->len >= n && a->cap >= n) { if (!retained) { memcpy(a->data, src, n * sizeof(*src)); diff --git a/packages/runtime/src/scr_array.h b/packages/runtime/src/scr_array.h index 4666750ea..d0c0ea1a8 100644 --- a/packages/runtime/src/scr_array.h +++ b/packages/runtime/src/scr_array.h @@ -128,7 +128,8 @@ typedef struct ScrArr { * header. Kept beside `elem` so the RC fast paths test it with one load. */ uint8_t cyc; /* The low SCR_ARR_OPS_SHIFT bits: storage flags (SCR_ARR_INLINE_SLOTS, - * SCR_ARR_HEAD_ROOM, SCR_ARR_UNSHIFTED, SCR_ARR_HAS_EXT, SCR_ARR_STATES). The rest: the + * SCR_ARR_HEAD_ROOM, SCR_ARR_UNSHIFTED, SCR_ARR_HAS_EXT, SCR_ARR_STATES) + * and SCR_ARR_FROZEN. The rest: the * registry id of the element type's ops (scr_arr_ops), 0 for every kind * but SCR_ELEM_REF and SCR_ELEM_CREF. */ uint16_t spare; @@ -204,6 +205,10 @@ _Static_assert(sizeof(ScrArr) == sizeof(size_t) + 16 + sizeof(void *), #define SCR_ARR_UNSHIFTED 4u #define SCR_ARR_HAS_EXT 8u #define SCR_ARR_STATES 16u +/* Object.freeze of the array (scr_arr_freeze). Emitted guards test it + * before the writes of array types a program may freeze; the runtime's own + * writes assume it is clear. */ +#define SCR_ARR_FROZEN 32u /* An array starts without state bytes. Without them, every index below * `len` is a present value in dense storage: len <= cap, there are no @@ -214,6 +219,19 @@ _Static_assert(sizeof(ScrArr) == sizeof(size_t) + 16 + sizeof(void *), * states (scr_arr_add_states), which it keeps. */ static inline bool scr_arr_has_states(const ScrArr *a) { return (a->spare & SCR_ARR_STATES) != 0; } +static inline bool scr_arr_frozen(const ScrArr *a) { return (a->spare & SCR_ARR_FROZEN) != 0; } +/* Sets SCR_ARR_FROZEN; immortal arrays are left as they are. */ +void scr_arr_freeze(ScrArr *a); +bool scr_arr_is_frozen(const ScrArr *a); +/* Throws Node's TypeError for the first write that the array operation + * `op` makes to a frozen array of length `len`, or answers false when the + * operation writes nothing. op: 0 index write (x = index), 1 push (y = + * argument count), 2 pop, 3 length write, 4 unshift (y = argument count), + * 5 delete (x = index), 6 shift, 7 reverse or sort, 8 fill (x = start, y = + * end), 9 copyWithin (x = target, y = start, z = end), 10 splice (x = + * start, y = delete count, z = item count). */ +bool scr_array_frozen_error(int op, size_t len, double x, double y, double z); + /* The start of the dense slots' allocation: `data` less any head room. */ static inline uint64_t *scr_arr_block(const ScrArr *a) { return (a->spare & SCR_ARR_HEAD_ROOM) ? a->data - (size_t)a->data[-1] : a->data; @@ -236,7 +254,7 @@ static inline struct ScrArrExt *scr_arr_ext_of(const ScrArr *a) { * Id 0 is no ops; ids are assigned on each descriptor's first array, in * any thread. A program with more element types than the field holds keeps * the ops of the rest in the array's ext (SCR_ARR_OPS_OVERFLOW). */ -#define SCR_ARR_OPS_SHIFT 5 +#define SCR_ARR_OPS_SHIFT 6 #define SCR_ARR_OPS_OVERFLOW (UINT16_MAX >> SCR_ARR_OPS_SHIFT) extern const ScrArrOps *scr_arr_ops_table[SCR_ARR_OPS_OVERFLOW]; /* By registry id: 1 when the descriptor has SCR_ARR_OPS_RELEASE_NOTHING. diff --git a/packages/runtime/src/scr_async.c b/packages/runtime/src/scr_async.c index 9e3365fbe..6afd174b7 100644 --- a/packages/runtime/src/scr_async.c +++ b/packages/runtime/src/scr_async.c @@ -72,15 +72,26 @@ void __lsan_unregister_root_region(const void *p, size_t size) __attribute__((we #if !defined(_WIN32) && !defined(__wasi__) -/* Keep stacks out of malloc's size classes. Reuse a bounded number during - * a burst of promise/generator work, then unmap the spares before the loop - * sleeps. This avoids both heap fragmentation and per-call mmap overhead. - * The sanitizer lane unmaps every finished stack immediately. */ +/* Keep stacks out of malloc's size classes. Reuse a bounded number of + * finished stacks, which avoids both heap fragmentation and per-call mmap + * overhead. A server's loop sleeps between nearly every pair of requests, + * so unmapping the spares at each sleep would map, fault in and unmap a + * stack for every request that awaits. Spares therefore outlive short + * sleeps: at most once per SCR_FIBER_TRIM_MS, a sleep unmaps the spares + * that no fiber took since the previous trim (the pool's low-water mark), + * and a sleep longer than twice that unmaps every spare, so an idle + * program gives the memory back as before. The sanitizer lane unmaps + * every finished stack immediately. */ #ifndef SCR_ASAN_FIBERS #define SCR_FIBER_SPARES 64 +#define SCR_FIBER_TRIM_MS 1000.0 static SCR_TL void *scr_fiber_spares[SCR_FIBER_SPARES]; static SCR_TL size_t scr_fiber_nspares; static SCR_TL bool scr_fiber_cleanup_registered; +/* The fewest spares the pool held since the last trim: the spares below + * this index went unused for the whole window. */ +static SCR_TL size_t scr_fiber_spares_low; +static SCR_TL double scr_fiber_trimmed_at; #endif static void scr_fiber_stacks_clear(void) { @@ -89,6 +100,29 @@ static void scr_fiber_stacks_clear(void) { (void)munmap(scr_fiber_spares[--scr_fiber_nspares], SCR_FIBER_STACK); scr_fiber_spares[scr_fiber_nspares] = NULL; } + scr_fiber_spares_low = 0; +#endif +} + +/* The loop is about to sleep for `wait` ms (see above). */ +static void scr_fiber_stacks_trim(double now, double wait) { +#ifndef SCR_ASAN_FIBERS + if (wait > 2 * SCR_FIBER_TRIM_MS) { + scr_fiber_stacks_clear(); + scr_fiber_trimmed_at = now; + return; + } + if (now - scr_fiber_trimmed_at < SCR_FIBER_TRIM_MS) return; + size_t unused = scr_fiber_spares_low; + for (size_t i = 0; i < unused; i++) (void)munmap(scr_fiber_spares[i], SCR_FIBER_STACK); + scr_fiber_nspares -= unused; + memmove(scr_fiber_spares, scr_fiber_spares + unused, scr_fiber_nspares * sizeof *scr_fiber_spares); + memset(scr_fiber_spares + scr_fiber_nspares, 0, unused * sizeof *scr_fiber_spares); + scr_fiber_spares_low = scr_fiber_nspares; + scr_fiber_trimmed_at = now; +#else + (void)now; + (void)wait; #endif } @@ -97,6 +131,7 @@ static void *scr_fiber_stack_new(void) { if (scr_fiber_nspares > 0) { void *stack = scr_fiber_spares[--scr_fiber_nspares]; scr_fiber_spares[scr_fiber_nspares] = NULL; + if (scr_fiber_nspares < scr_fiber_spares_low) scr_fiber_spares_low = scr_fiber_nspares; return stack; } if (!scr_fiber_cleanup_registered) { @@ -4599,7 +4634,7 @@ static bool scr_loop_run_pass(ScrPromise *top_level, bool first_checkpoint) { /* An idle moment: a full pass the death rule deferred runs now. */ if (due > now && scr_cyc_collect_idle()) now = scr_now_ms(); #if !defined(_WIN32) && !defined(__wasi__) - if (due > now) scr_fiber_stacks_clear(); + if (due > now) scr_fiber_stacks_trim(now, due - now); #endif #ifdef SCR_WORKERS if (due > now) scr_sa_loop_waiting(true); diff --git a/packages/runtime/src/scr_bytes.h b/packages/runtime/src/scr_bytes.h index 9b382dda5..2e4b44409 100644 --- a/packages/runtime/src/scr_bytes.h +++ b/packages/runtime/src/scr_bytes.h @@ -357,6 +357,18 @@ ScrStr *scr_text_decode(const ScrBytes *b); double scr_text_decoder_encoding(const ScrDyn *label); ScrStr *scr_text_decoder_name(double encoding); ScrDyn *scr_text_encode_into(const ScrDyn *source, const ScrDyn *destination); +/* TextEncoder and TextDecoder records as checked-dynamic handles + * (scr_text_codec_dyn.c, linked by the textCodecDyn runtime feature). The + * boxes borrow the record and retain it through `retain`; a decoder also + * names the byte offsets of its encoding (f64), fatal and ignoreBOM (i8) + * and state (dyn) fields. from_dyn answers the retained record or throws + * the path-annotated TypeError and answers NULL. */ +ScrDyn *scr_text_encoder_to_dyn(void *rec, void *(*retain)(void *), void (*release)(void *)); +ScrDyn *scr_text_decoder_to_dyn(void *rec, void *(*retain)(void *), void (*release)(void *), + size_t encoding, size_t fatal, size_t ignore_bom, size_t state); +void *scr_text_codec_from_dyn(const ScrDyn *d, bool decoder, const struct ScrDynPath *path, + const char *want); +bool scr_text_codec_is(const ScrDyn *d, bool decoder); bool scr_buffer_is_ascii(const ScrDyn *input); bool scr_buffer_is_utf8(const ScrDyn *input); ScrBytes *scr_buffer_transcode(const ScrDyn *source, const ScrDyn *from, const ScrDyn *to); diff --git a/packages/runtime/src/scr_child.c b/packages/runtime/src/scr_child.c index df7543500..a70b10894 100644 --- a/packages/runtime/src/scr_child.c +++ b/packages/runtime/src/scr_child.c @@ -2780,7 +2780,8 @@ static void scr_child_settle(ScrChild *c) { scr_children_reffed_n--; } if (c->state == SCR_CHILD_EXITED) { - scr_child_stream_drain_now(c->out_stream); + if (!c->spawn_done) scr_child_fire_spawn(c); + if (!scr_exc_pending()) scr_child_stream_drain_now(c->out_stream); if (!scr_exc_pending()) scr_child_stream_drain_now(c->err_stream); if (!scr_exc_pending()) scr_ipc_settle(c->ipc); } @@ -5607,9 +5608,12 @@ static void scr_child_settle(ScrChild *c) { /* Streams first, settled after: an exit listener registered from a * draining 'data'/'end' callback still fires below, like Node's. The * SPAWN-FAILURE path skips the drain — Node fires 'error' first and - * the streams' immediate EOF delivers 'end' on the next turn. */ + * the streams' immediate EOF delivers 'end' on the next turn. A child + * spawned during this pass can exit before the next pass fires 'spawn'; + * Node emits 'spawn' before any data or 'exit', so it fires here first. */ if (c->state == SCR_CHILD_EXITED) { - scr_child_stream_drain_now(c->out_stream); + if (!c->spawn_done) scr_child_fire_spawn(c); + if (!scr_exc_pending()) scr_child_stream_drain_now(c->out_stream); if (!scr_exc_pending()) scr_child_stream_drain_now(c->err_stream); if (!scr_exc_pending()) scr_ipc_settle(c->ipc); } diff --git a/packages/runtime/src/scr_cipher.c b/packages/runtime/src/scr_cipher.c index 00d229bad..a2db11434 100644 --- a/packages/runtime/src/scr_cipher.c +++ b/packages/runtime/src/scr_cipher.c @@ -37,17 +37,23 @@ typedef enum { SC_CFB1, SC_OFB, SC_CHACHA20, + /* CCM and the key wraps stay last: sc_transform tests mode >= SC_CCM. */ + SC_CCM, + SC_WRAP, /* RFC 3394 AES key wrap */ + SC_WRAP_PAD, /* RFC 5649 AES key wrap with padding */ } ScCipherMode; typedef struct { const char *name; ScCipherMode mode; unsigned key_len; - bool des; /* triple DES (two- or three-key) rather than AES */ + bool des; /* triple DES (two- or three-key) rather than AES */ + bool inverse; /* key wrap with the inverse cipher (the -inv names) */ } ScCipherSpec; /* Node's names for the implemented ciphers, including OpenSSL's aliases. - * Lookup is case-insensitive, like OpenSSL's. */ + * Lookup is case-insensitive, like OpenSSL's, and scans in order, so the + * CCM and key wrap names follow the others. */ static const ScCipherSpec sc_specs[] = { {"aes-128-cbc", SC_CBC, 16}, {"aes-192-cbc", SC_CBC, 24}, {"aes-256-cbc", SC_CBC, 32}, {"aes128", SC_CBC, 16}, {"aes192", SC_CBC, 24}, {"aes256", SC_CBC, 32}, @@ -68,6 +74,38 @@ static const ScCipherSpec sc_specs[] = { {"des-ede-ecb", SC_ECB, 16, true}, {"des-ede-cfb", SC_CFB, 16, true}, {"des-ede-ofb", SC_OFB, 16, true}, {"chacha20", SC_CHACHA20, 32}, + {"aes-128-ccm", SC_CCM, 16}, {"aes-192-ccm", SC_CCM, 24}, {"aes-256-ccm", SC_CCM, 32}, + {"id-aes128-ccm", SC_CCM, 16}, {"id-aes192-ccm", SC_CCM, 24}, {"id-aes256-ccm", SC_CCM, 32}, + {"aes-128-wrap", SC_WRAP, 16, false, false}, + {"aes128-wrap", SC_WRAP, 16, false, false}, + {"id-aes128-wrap", SC_WRAP, 16, false, false}, + {"aes-128-wrap-inv", SC_WRAP, 16, false, true}, + {"aes128-wrap-inv", SC_WRAP, 16, false, true}, + {"aes-128-wrap-pad", SC_WRAP_PAD, 16, false, false}, + {"aes128-wrap-pad", SC_WRAP_PAD, 16, false, false}, + {"id-aes128-wrap-pad", SC_WRAP_PAD, 16, false, false}, + {"aes-128-wrap-pad-inv", SC_WRAP_PAD, 16, false, true}, + {"aes128-wrap-pad-inv", SC_WRAP_PAD, 16, false, true}, + {"aes-192-wrap", SC_WRAP, 24, false, false}, + {"aes192-wrap", SC_WRAP, 24, false, false}, + {"id-aes192-wrap", SC_WRAP, 24, false, false}, + {"aes-192-wrap-inv", SC_WRAP, 24, false, true}, + {"aes192-wrap-inv", SC_WRAP, 24, false, true}, + {"aes-192-wrap-pad", SC_WRAP_PAD, 24, false, false}, + {"aes192-wrap-pad", SC_WRAP_PAD, 24, false, false}, + {"id-aes192-wrap-pad", SC_WRAP_PAD, 24, false, false}, + {"aes-192-wrap-pad-inv", SC_WRAP_PAD, 24, false, true}, + {"aes192-wrap-pad-inv", SC_WRAP_PAD, 24, false, true}, + {"aes-256-wrap", SC_WRAP, 32, false, false}, + {"aes256-wrap", SC_WRAP, 32, false, false}, + {"id-aes256-wrap", SC_WRAP, 32, false, false}, + {"aes-256-wrap-inv", SC_WRAP, 32, false, true}, + {"aes256-wrap-inv", SC_WRAP, 32, false, true}, + {"aes-256-wrap-pad", SC_WRAP_PAD, 32, false, false}, + {"aes256-wrap-pad", SC_WRAP_PAD, 32, false, false}, + {"id-aes256-wrap-pad", SC_WRAP_PAD, 32, false, false}, + {"aes-256-wrap-pad-inv", SC_WRAP_PAD, 32, false, true}, + {"aes256-wrap-pad-inv", SC_WRAP_PAD, 32, false, true}, }; /* Ciphers OpenSSL 3 keeps in its legacy provider, which Node 24 does not @@ -139,7 +177,8 @@ typedef struct ScCipher { int tag_state; unsigned char tag[16]; unsigned char block[16]; /* CBC: chaining value; CTR: counter block; CFB, OFB: - shift register; chacha20: the IV */ + shift register; chacha20, CCM: the IV; key wrap: + the integrity check value */ unsigned char buf[16]; /* CBC, ECB: buffered input; CTR: keystream block */ size_t bufsz; /* CBC, ECB: buffered bytes; CTR, CFB, OFB: keystream offset */ @@ -152,6 +191,19 @@ typedef struct ScCipher { uint64_t aad_len; /* chacha20-poly1305 AAD bytes */ uint64_t text_len; /* chacha20-poly1305 data bytes */ bool aad_open; /* AAD was added since the last data or padding */ + /* CCM: OpenSSL's ccm128 state (the counter block, whose first byte holds + * the L and M parameters and the Adata flag, and the CBC-MAC), its + * provider's flags, and Node's one-update rule. A decipher's expected + * tag is `tag`. */ + size_t ccm_iv_len; + unsigned char ccm_nonce[16]; + unsigned char ccm_cmac[16]; + bool ccm_iv_set, ccm_len_set, ccm_tag_set; + bool ccm_updated; /* a decipher's one update() ran; key wrap: the one update() */ + bool ccm_auth_failed; /* that update failed to authenticate */ + bool ccm_verified; /* a decipher's tag matched the one it computed */ + double ccm_max_message; + bool wrap_decrypt_block; /* key wrap: the AES key schedule decrypts */ ScrStr *decoder; /* the StringDecoder encoding once string output starts */ double pending; /* the decoder's carried bytes */ /* The Transform of a cipher used as a stream (+1) and its release @@ -513,6 +565,10 @@ static size_t sc_iv_length(ScCipherMode mode, bool des) { return 12; case SC_CHACHA20: return 16; + case SC_WRAP: + return 8; + case SC_WRAP_PAD: + return 4; default: return des ? 8 : 16; } @@ -530,9 +586,9 @@ static void sc_unsupported_cipher(const ScrStr *name) { scr_jb_puts(&b, "The cipher '"); scr_jb_put_str(&b, name); scr_jb_puts(&b, "' has no native implementation; the supported ciphers are aes-128, aes-192 and " - "aes-256 in cbc, ctr, gcm, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede3 in cbc, ecb, " - "cfb, cfb8, cfb1 and ofb modes, des-ede in cbc, ecb, cfb and ofb modes, chacha20 and " - "chacha20-poly1305"); + "aes-256 in cbc, ctr, gcm, ccm, ecb, cfb, cfb8, cfb1 and ofb modes and their wrap, " + "wrap-pad, wrap-inv and wrap-pad-inv key wraps, des-ede3 in cbc, ecb, cfb, cfb8, cfb1 " + "and ofb modes, des-ede in cbc, ecb, cfb and ofb modes, chacha20 and chacha20-poly1305"); ScrStr *message = scr_jb_finish(&b); scr_throw_error_msg_code(SCR_ERR_ERROR, message->data, message->len, "SC2020"); scr_str_release(message); @@ -618,7 +674,10 @@ static ScrDyn *sc_cipher_new(bool decrypt, ScrDyn *cipher, ScrDyn *key, ScrDyn * goto done; } size_t expected_iv = sc_iv_length(spec->mode, spec->des); - if (spec->mode == SC_GCM ? iv_len == 0 : iv_len != expected_iv || (expected_iv > 0 && !iv_bytes)) { + bool iv_ok = spec->mode == SC_GCM ? iv_len > 0 + : spec->mode == SC_CCM ? iv_len >= 7 && iv_len <= 13 + : iv_len == expected_iv && (expected_iv == 0 || iv_bytes); + if (!iv_ok) { sc_throw(SCR_ERR_TYPE, "ERR_CRYPTO_INVALID_IV", "Invalid initialization vector"); goto done; } @@ -646,6 +705,24 @@ static ScrDyn *sc_cipher_new(bool decrypt, ScrDyn *cipher, ScrDyn *key, ScrDyn * sc_invalid_tag((unsigned long)tag_option); goto done; } + } else if (spec->mode == SC_CCM) { + /* InitAuthenticated: CCM needs the tag length, which OpenSSL takes when + * it is even and 4 to 16 bytes. */ + if (tag_option < 0) { + ScrJsonBuf b; + scr_jb_init(&b); + scr_jb_puts(&b, "authTagLength required for "); + scr_jb_put_str(&b, cipher->v.str); + ScrStr *message = scr_jb_finish(&b); + scr_throw_error_msg_code(SCR_ERR_TYPE, message->data, message->len, "ERR_CRYPTO_INVALID_AUTH_TAG"); + scr_str_release(message); + goto done; + } + if (tag_option < 4 || tag_option > 16 || (unsigned long)tag_option % 2 != 0) { + sc_invalid_tag((unsigned long)tag_option); + goto done; + } + tag_len = (int)tag_option; } if (key_bytes->len != spec->key_len) { sc_throw(SCR_ERR_RANGE, "ERR_CRYPTO_INVALID_KEYLEN", "Invalid key length"); @@ -709,6 +786,29 @@ static ScrDyn *sc_cipher_new(bool decrypt, ScrDyn *cipher, ScrDyn *key, ScrDyn * rc = mbedtls_aes_setkey_enc(&c->aes, key_bytes->data, bits); memcpy(c->block, iv_bytes->data, 16); break; + case SC_WRAP: + case SC_WRAP_PAD: + /* SP 800-38F: wrapping is the forward transformation (encryption) and + * unwrapping the inverse, and the -inv ciphers swap the two. The IV is + * the wrap's integrity check value. */ + rc = decrypt != spec->inverse ? mbedtls_aes_setkey_dec(&c->aes, key_bytes->data, bits) + : mbedtls_aes_setkey_enc(&c->aes, key_bytes->data, bits); + c->wrap_decrypt_block = decrypt != spec->inverse; + memcpy(c->block, iv_bytes->data, iv_len); + break; + case SC_CCM: { + /* CRYPTO_ccm128_init with M = the tag length and L = 15 - the IV + * length; Node caps the message at what a 12- or 13-byte IV's length + * field holds, and at INT_MAX otherwise. */ + rc = mbedtls_aes_setkey_enc(&c->aes, key_bytes->data, bits); + unsigned L = 15 - (unsigned)iv_len; + c->ccm_nonce[0] = (unsigned char)(((L - 1) & 7) | ((((unsigned)tag_len - 2) / 2) & 7) << 3); + memcpy(c->block, iv_bytes->data, iv_len); + c->ccm_iv_len = iv_len; + c->ccm_iv_set = true; + c->ccm_max_message = iv_len == 13 ? 65535 : iv_len == 12 ? 16777215 : 2147483647; + break; + } case SC_GCM: rc = mbedtls_gcm_setkey(&c->gcm, MBEDTLS_CIPHER_ID_AES, key_bytes->data, bits); if (rc == 0) @@ -970,7 +1070,12 @@ static bool sc_chacha_finish(ScCipher *c, unsigned char tag[16]) { /* The cipher output of an update over len bytes (+1 Buffer), or NULL * after a backend failure (for example AES-GCM's 2^36 - 32 byte message * limit) has retired the context and thrown Node's update error. */ +static ScrBytes *sc_ccm_update(ScCipher *c, const unsigned char *in, size_t len); +static ScrBytes *sc_wrap_update(ScCipher *c, const unsigned char *in, size_t len); + static ScrBytes *sc_transform(ScCipher *c, const unsigned char *in, size_t len) { + /* CCM and the key wraps, the last modes, take their own update. */ + if (c->mode >= SC_CCM) return c->mode == SC_CCM ? sc_ccm_update(c, in, len) : sc_wrap_update(c, in, len); size_t room = len + 16; ScrBytes *out = scr_bytes_new(SCR_BYTES_U8, (double)room); out->is_buffer = true; @@ -1006,6 +1111,10 @@ static ScrBytes *sc_transform(ScCipher *c, const unsigned char *in, size_t len) ok = len == 0 || sc_chacha20_update(c, in, len, out->data); produced = len; break; + case SC_CCM: + case SC_WRAP: + case SC_WRAP_PAD: + break; } if (!ok) { mbedtls_platform_zeroize(out->data, room); @@ -1019,6 +1128,379 @@ static ScrBytes *sc_transform(ScCipher *c, const unsigned char *in, size_t len) return out; } +/* ── CCM ────────────────────────────────────────────────────────────── + * OpenSSL's ccm128 (crypto/modes/ccm128.c) over the AES block, step for + * step, with its provider's flags (ciphercommon_ccm.c) and Node's rules + * (one update, the message-length cap, a decipher's failure reported by + * final()), so every call sequence answers as Node's does. */ + +static void sc_ccm_block(ScCipher *c, const unsigned char in[16], unsigned char out[16]) { + mbedtls_aes_crypt_ecb(&c->aes, MBEDTLS_AES_ENCRYPT, in, out); +} + +/* CRYPTO_ccm128_setiv: the nonce and the message length. */ +static void sc_ccm_setiv(ScCipher *c, uint64_t mlen) { + unsigned L = c->ccm_nonce[0] & 7; + if (L >= 3) { + c->ccm_nonce[8] = (unsigned char)(mlen >> 56); + c->ccm_nonce[9] = (unsigned char)(mlen >> 48); + c->ccm_nonce[10] = (unsigned char)(mlen >> 40); + c->ccm_nonce[11] = (unsigned char)(mlen >> 32); + } else { + memset(c->ccm_nonce + 8, 0, 8); + } + c->ccm_nonce[12] = (unsigned char)(mlen >> 24); + c->ccm_nonce[13] = (unsigned char)(mlen >> 16); + c->ccm_nonce[14] = (unsigned char)(mlen >> 8); + c->ccm_nonce[15] = (unsigned char)mlen; + c->ccm_nonce[0] &= (unsigned char)~0x40; + memcpy(c->ccm_nonce + 1, c->block, 14 - L); +} + +/* CRYPTO_ccm128_aad. */ +static void sc_ccm_aad(ScCipher *c, const unsigned char *aad, size_t alen) { + if (alen == 0) return; + c->ccm_nonce[0] |= 0x40; + sc_ccm_block(c, c->ccm_nonce, c->ccm_cmac); + size_t i; + if (alen < 0x10000 - 0x100) { + c->ccm_cmac[0] ^= (unsigned char)(alen >> 8); + c->ccm_cmac[1] ^= (unsigned char)alen; + i = 2; + } else if ((uint64_t)alen >= (uint64_t)1 << 32) { + c->ccm_cmac[0] ^= 0xFF; + c->ccm_cmac[1] ^= 0xFF; + for (int k = 0; k < 8; k++) c->ccm_cmac[2 + k] ^= (unsigned char)((uint64_t)alen >> (56 - 8 * k)); + i = 10; + } else { + c->ccm_cmac[0] ^= 0xFF; + c->ccm_cmac[1] ^= 0xFE; + for (int k = 0; k < 4; k++) c->ccm_cmac[2 + k] ^= (unsigned char)((uint64_t)alen >> (24 - 8 * k)); + i = 6; + } + do { + for (; i < 16 && alen; ++i, ++aad, --alen) c->ccm_cmac[i] ^= *aad; + sc_ccm_block(c, c->ccm_cmac, c->ccm_cmac); + i = 0; + } while (alen); +} + +/* ctr64_inc: the counter block's low eight bytes, big-endian. */ +static void sc_ccm_ctr_inc(unsigned char *counter) { + for (int n = 15; n >= 8; n--) + if (++counter[n]) return; +} + +/* CRYPTO_ccm128_encrypt and CRYPTO_ccm128_decrypt: false (-1) for a + * length that differs from the one the counter block holds. */ +static bool sc_ccm_crypt(ScCipher *c, const unsigned char *in, unsigned char *out, size_t len, bool decrypt) { + unsigned char flags0 = c->ccm_nonce[0]; + unsigned char scratch[16]; + if (!(flags0 & 0x40)) sc_ccm_block(c, c->ccm_nonce, c->ccm_cmac); + unsigned L = flags0 & 7; + c->ccm_nonce[0] = (unsigned char)L; + uint64_t n = 0; + for (unsigned i = 15 - L; i < 15; ++i) { + n |= c->ccm_nonce[i]; + c->ccm_nonce[i] = 0; + n <<= 8; + } + n |= c->ccm_nonce[15]; + c->ccm_nonce[15] = 1; + if (n != (uint64_t)len) return false; + /* Each full block runs the counter block's encryption before the + * CBC-MAC's, in both directions, and XORs in 64-bit words, so the two + * AES operations of a block overlap. `out` may be `in`. */ + while (len >= 16) { + uint64_t data[2], mac[2], pad[2]; + sc_ccm_block(c, c->ccm_nonce, scratch); + sc_ccm_ctr_inc(c->ccm_nonce); + memcpy(data, in, 16); + memcpy(mac, c->ccm_cmac, 16); + memcpy(pad, scratch, 16); + pad[0] ^= data[0]; + pad[1] ^= data[1]; + /* The MAC covers the plaintext: the input when encrypting, the output + * when decrypting. */ + mac[0] ^= decrypt ? pad[0] : data[0]; + mac[1] ^= decrypt ? pad[1] : data[1]; + memcpy(out, pad, 16); + memcpy(c->ccm_cmac, mac, 16); + sc_ccm_block(c, c->ccm_cmac, c->ccm_cmac); + in += 16; + out += 16; + len -= 16; + } + if (len) { + if (decrypt) { + sc_ccm_block(c, c->ccm_nonce, scratch); + for (size_t i = 0; i < len; ++i) c->ccm_cmac[i] ^= (out[i] = (unsigned char)(scratch[i] ^ in[i])); + sc_ccm_block(c, c->ccm_cmac, c->ccm_cmac); + } else { + for (size_t i = 0; i < len; ++i) c->ccm_cmac[i] ^= in[i]; + sc_ccm_block(c, c->ccm_cmac, c->ccm_cmac); + sc_ccm_block(c, c->ccm_nonce, scratch); + for (size_t i = 0; i < len; ++i) out[i] = (unsigned char)(scratch[i] ^ in[i]); + } + } + for (unsigned i = 15 - L; i < 16; ++i) c->ccm_nonce[i] = 0; + sc_ccm_block(c, c->ccm_nonce, scratch); + for (int i = 0; i < 16; i++) c->ccm_cmac[i] ^= scratch[i]; + c->ccm_nonce[0] = flags0; + mbedtls_platform_zeroize(scratch, sizeof scratch); + return true; +} + +/* CRYPTO_ccm128_tag: the MAC, when the counter block still holds the + * tag length M (a failed call clears it). */ +static bool sc_ccm_tag(const ScCipher *c, unsigned char *tag) { + unsigned M = ((c->ccm_nonce[0] >> 3) & 7) * 2 + 2; + if (M != (unsigned)c->tag_len) return false; + memcpy(tag, c->ccm_cmac, M); + return true; +} + +/* The provider's ccm_cipher_internal for data (`out` given): the length + * is set by the first data unless setAAD set it; encryption marks the tag + * computed; decryption needs the tag and checks it, and on success clears + * the flags so a second call fails. */ +static bool sc_ccm_data(ScCipher *c, const unsigned char *in, unsigned char *out, size_t len) { + if (!c->ccm_iv_set) return false; + if (!c->ccm_len_set) { + sc_ccm_setiv(c, len); + c->ccm_len_set = true; + } + if (!c->decrypt) { + if (!sc_ccm_crypt(c, in, out, len, false)) return false; + c->ccm_tag_set = true; + return true; + } + if (!c->ccm_tag_set) return false; + unsigned char computed[16]; + bool ok = sc_ccm_crypt(c, in, out, len, true) && sc_ccm_tag(c, computed) && + sc_tag_equal(computed, c->tag, (size_t)c->tag_len); + mbedtls_platform_zeroize(computed, sizeof computed); + if (!ok) { + if (len) mbedtls_platform_zeroize(out, len); + return false; + } + c->ccm_iv_set = c->ccm_tag_set = c->ccm_len_set = false; + c->ccm_verified = true; + return true; +} + +static bool sc_ccm_length_ok(const ScCipher *c, double len) { + if (len <= c->ccm_max_message) return true; + sc_throw(SCR_ERR_RANGE, "ERR_CRYPTO_INVALID_MESSAGELEN", "Invalid message length"); + return false; +} + +/* update(): a decipher takes one, and its authentication failure leaves + * empty output for final() to report. A decipher authenticates even an + * empty message, as OpenSSL does: its tag still covers the nonce, the AAD + * and the length. A cipher's empty update is skipped, and final() then + * runs the empty message. */ +static ScrBytes *sc_ccm_update(ScCipher *c, const unsigned char *in, size_t len) { + if (!sc_ccm_length_ok(c, (double)len)) return NULL; + if (c->decrypt && c->ccm_updated) { + sc_throw(SCR_ERR_ERROR, NULL, "Trying to add data in unsupported state"); + return NULL; + } + ScrBytes *out = scr_bytes_new(SCR_BYTES_U8, (double)(len ? len : 1)); + out->is_buffer = true; + bool ok = (len == 0 && !c->decrypt) || sc_ccm_data(c, len ? in : out->data, out->data, len); + if (c->decrypt) { + c->ccm_updated = true; + if (!ok) c->ccm_auth_failed = true; + out->len = ok ? len : 0; + return out; + } + if (!ok) { + mbedtls_platform_zeroize(out->data, len); + scr_bytes_release(out); + sc_throw(SCR_ERR_ERROR, NULL, "Trying to add data in unsupported state"); + return NULL; + } + out->len = len; + return out; +} + +/* final(): a decipher succeeds only when its update verified the tag; a + * cipher runs the provider's final (an empty message when no data was + * encrypted) and reads the tag, which retires the provider's state. */ +static bool sc_ccm_final(ScCipher *c) { + if (c->decrypt) return c->tag_state == SC_TAG_SET_BY_USER && c->ccm_verified; + unsigned char dummy = 0, sink = 0; + if (c->ccm_iv_set && !c->ccm_tag_set && !sc_ccm_data(c, &dummy, &sink, 0)) return false; + if (!c->ccm_tag_set || !sc_ccm_tag(c, c->tag)) return false; + c->tag_state = SC_TAG_COMPUTED; + c->ccm_tag_set = c->ccm_iv_set = c->ccm_len_set = false; + return true; +} + +/* setAAD(buffer, { plaintextLength }): the length goes first, as Node + * passes it to OpenSSL, then the AAD. False when the provider refuses. */ +static bool sc_ccm_set_aad(ScCipher *c, const ScrBytes *aad, double plaintext_length) { + if (!c->ccm_iv_set) return false; + sc_ccm_setiv(c, (uint64_t)plaintext_length); + c->ccm_len_set = true; + sc_ccm_aad(c, aad->data, aad->len); + return true; +} + +/* ── key wrap ───────────────────────────────────────────────────────── + * OpenSSL's CRYPTO_128_wrap, CRYPTO_128_unwrap and their RFC 5649 + * padded forms (crypto/modes/wrap128.c) over the key's AES transform, and + * Node's one-update rule: a failed update() leaves the cipher usable, an + * empty one counts, and final() without one fails. */ + +static void sc_wrap_block(ScCipher *c, const unsigned char in[16], unsigned char out[16]) { + mbedtls_aes_crypt_ecb(&c->aes, c->wrap_decrypt_block ? MBEDTLS_AES_DECRYPT : MBEDTLS_AES_ENCRYPT, in, out); +} + +static void sc_wrap_xor_t(unsigned char *a, size_t t) { + a[7] ^= (unsigned char)(t & 0xff); + if (t > 0xff) { + a[6] ^= (unsigned char)((t >> 8) & 0xff); + a[5] ^= (unsigned char)((t >> 16) & 0xff); + a[4] ^= (unsigned char)((t >> 24) & 0xff); + } +} + +/* CRYPTO_128_wrap: 0 unless the input is at least 16 bytes in 8-byte + * blocks. `out` may be `in`. */ +static size_t sc_wrap(ScCipher *c, const unsigned char *iv, unsigned char *out, const unsigned char *in, size_t inlen) { + if ((inlen & 7) || inlen < 16) return 0; + unsigned char b[16]; + size_t t = 1; + memmove(out + 8, in, inlen); + memcpy(b, iv, 8); + for (int j = 0; j < 6; j++) { + unsigned char *r = out + 8; + for (size_t i = 0; i < inlen; i += 8, t++, r += 8) { + memcpy(b + 8, r, 8); + sc_wrap_block(c, b, b); + sc_wrap_xor_t(b, t); + memcpy(r, b + 8, 8); + } + } + memcpy(out, b, 8); + mbedtls_platform_zeroize(b, sizeof b); + return inlen + 8; +} + +/* crypto_128_unwrap_raw: the plaintext into `out` and the recovered + * integrity check value into `aiv`. */ +static size_t sc_unwrap_raw(ScCipher *c, unsigned char aiv[8], unsigned char *out, const unsigned char *in, size_t inlen) { + inlen -= 8; + if ((inlen & 7) || inlen < 16) return 0; + unsigned char b[16]; + size_t t = 6 * (inlen >> 3); + memcpy(b, in, 8); + memmove(out, in + 8, inlen); + for (int j = 0; j < 6; j++) { + unsigned char *r = out + inlen - 8; + for (size_t i = 0; i < inlen; i += 8, t--, r -= 8) { + sc_wrap_xor_t(b, t); + memcpy(b + 8, r, 8); + sc_wrap_block(c, b, b); + memcpy(r, b + 8, 8); + } + } + memcpy(aiv, b, 8); + mbedtls_platform_zeroize(b, sizeof b); + return inlen; +} + +static size_t sc_unwrap(ScCipher *c, const unsigned char *iv, unsigned char *out, const unsigned char *in, size_t inlen) { + unsigned char got[8]; + size_t ret = sc_unwrap_raw(c, got, out, in, inlen); + if (!ret) return 0; + if (!sc_tag_equal(got, iv, 8)) { + mbedtls_platform_zeroize(out, ret); + return 0; + } + return ret; +} + +/* CRYPTO_128_wrap_pad: the 4-byte ICV and the 32-bit length, then the + * data padded with zeros to 8 bytes; one block is a single encryption. */ +static size_t sc_wrap_pad(ScCipher *c, const unsigned char *icv, unsigned char *out, const unsigned char *in, size_t inlen) { + size_t padded = (inlen + 7) / 8 * 8; + if (inlen == 0 || inlen >= ((size_t)1 << 31)) return 0; + unsigned char aiv[8]; + memcpy(aiv, icv, 4); + aiv[4] = (unsigned char)((inlen >> 24) & 0xFF); + aiv[5] = (unsigned char)((inlen >> 16) & 0xFF); + aiv[6] = (unsigned char)((inlen >> 8) & 0xFF); + aiv[7] = (unsigned char)(inlen & 0xFF); + if (padded == 8) { + memmove(out + 8, in, inlen); + memcpy(out, aiv, 8); + memset(out + 8 + inlen, 0, padded - inlen); + sc_wrap_block(c, out, out); + return 16; + } + memmove(out, in, inlen); + memset(out + inlen, 0, padded - inlen); + return sc_wrap(c, aiv, out, out, padded); +} + +/* CRYPTO_128_unwrap_pad: the ICV, the length within the last 8 bytes and + * zero padding must all check. */ +static size_t sc_unwrap_pad(ScCipher *c, const unsigned char *icv, unsigned char *out, const unsigned char *in, size_t inlen) { + if ((inlen & 7) || inlen < 16) return 0; + size_t n = inlen / 8 - 1, padded; + unsigned char aiv[8]; + if (inlen == 16) { + unsigned char buff[16]; + sc_wrap_block(c, in, buff); + memcpy(aiv, buff, 8); + memcpy(out, buff + 8, 8); + padded = 8; + mbedtls_platform_zeroize(buff, sizeof buff); + } else { + padded = inlen - 8; + if (sc_unwrap_raw(c, aiv, out, in, inlen) != padded) { + mbedtls_platform_zeroize(out, inlen); + return 0; + } + } + size_t plain = (size_t)aiv[4] << 24 | (size_t)aiv[5] << 16 | (size_t)aiv[6] << 8 | aiv[7]; + static const unsigned char zeros[8] = {0}; + if (!sc_tag_equal(aiv, icv, 4) || 8 * (n - 1) >= plain || plain > 8 * n || + !sc_tag_equal(out + plain, zeros, padded - plain)) { + mbedtls_platform_zeroize(out, padded); + return 0; + } + return plain; +} + +static ScrBytes *sc_wrap_update(ScCipher *c, const unsigned char *in, size_t len) { + if (c->ccm_updated) { + sc_throw(SCR_ERR_ERROR, NULL, "Trying to add data in unsupported state"); + return NULL; + } + ScrBytes *out = scr_bytes_new(SCR_BYTES_U8, (double)(len + 16)); + out->is_buffer = true; + size_t produced = 0; + if (len > 0) { + bool pad = c->mode == SC_WRAP_PAD; + produced = !c->decrypt ? (pad ? sc_wrap_pad(c, c->block, out->data, in, len) : sc_wrap(c, c->block, out->data, in, len)) + : (pad ? sc_unwrap_pad(c, c->block, out->data, in, len) + : sc_unwrap(c, c->block, out->data, in, len)); + if (produced == 0) { + mbedtls_platform_zeroize(out->data, len + 16); + scr_bytes_release(out); + sc_throw(SCR_ERR_ERROR, NULL, "Trying to add data in unsupported state"); + return NULL; + } + } + c->ccm_updated = true; + out->len = produced; + return out; +} + /* ── output encodings ─────────────────────────────────────────────────── */ /* Node's getDecoder: the StringDecoder for this encoding, created on first @@ -1131,7 +1613,7 @@ static ScrDyn *sc_update(ScCipher *c, ScrDyn *data, ScrDyn *input_encoding, ScrD } static bool sc_aead(const ScCipher *c) { - return c->mode == SC_GCM || c->mode == SC_CHACHA; + return c->mode == SC_GCM || c->mode == SC_CHACHA || c->mode == SC_CCM; } /* final()'s output bytes (+1 Buffer), or NULL with the error thrown. */ @@ -1164,6 +1646,13 @@ static ScrBytes *sc_final_bytes(ScCipher *c) { case SC_OFB: case SC_CHACHA20: break; + case SC_CCM: + ok = sc_ccm_final(c); + break; + case SC_WRAP: + case SC_WRAP_PAD: + backend = c->ccm_updated; + break; case SC_GCM: { unsigned char full[16]; size_t ignored; @@ -1231,6 +1720,20 @@ static ScrDyn *sc_set_aad(ScCipher *c, ScrDyn *self, ScrDyn *buffer, ScrDyn *opt ScrBytes *aad = sc_buffer_or_view(buffer, "aadbuf", encoding); scr_dyn_release(encoding); if (!aad) return NULL; + if (c->mode == SC_CCM && c->active) { + bool ok = true; + if (plaintext_length < 0) { + sc_throw(SCR_ERR_TYPE, "ERR_MISSING_ARGS", "options.plaintextLength required for CCM mode with AAD"); + ok = false; + } else if (!sc_ccm_length_ok(c, plaintext_length)) { + ok = false; + } else if (!sc_ccm_set_aad(c, aad, plaintext_length)) { + sc_invalid_state("setAAD"); + ok = false; + } + scr_bytes_release(aad); + return ok ? scr_dyn_retain(self) : NULL; + } /* After data, GCM refuses any AAD and chacha20-poly1305 refuses * non-empty AAD. */ bool ok = c->active && sc_aead(c) && !(c->data_started && (c->mode == SC_GCM || aad->len > 0)); @@ -1314,6 +1817,7 @@ static ScrDyn *sc_set_auth_tag(ScCipher *c, ScrDyn *self, ScrDyn *buffer, ScrDyn c->tag_len = (int)len; memcpy(c->tag, tag->data, len); c->tag_state = SC_TAG_SET_BY_USER; + if (c->mode == SC_CCM) c->ccm_tag_set = true; scr_bytes_release(tag); return scr_dyn_retain(self); } diff --git a/packages/runtime/src/scr_cipher_info.c b/packages/runtime/src/scr_cipher_info.c new file mode 100644 index 000000000..c785724cb --- /dev/null +++ b/packages/runtime/src/scr_cipher_info.c @@ -0,0 +1,200 @@ +/* scriptc runtime — crypto.getCipherInfo(nameOrNid[, options]) over the + * table Node's OpenSSL ciphers generate (scr_cipher_info.h): cipher.js's + * argument checks, then GetCipherInfo's keyLength and ivLength rules and + * its info object, in Node's key order. Names match case-insensitively; + * nids are OpenSSL's. Arguments are borrowed. */ +#include "scr_runtime.h" + +#include +#include +#include +#include + +#include "scr_cipher_info.h" + +#define CI_COUNT(table) (sizeof(table) / sizeof((table)[0])) + +/* validateInt32(value, name): false with Node's error pending. */ +static bool ci_validate_int32(const char *name, const ScrDyn *value) { + if (value->kind != SCR_DYN_NUM) { + scr_dyn_arg_type_fail(name, "of type number", value); + return false; + } + double v = value->v.num; + bool integer = isfinite(v) && trunc(v) == v; + if (integer && v >= INT32_MIN && v <= INT32_MAX) return true; + char received[48]; + scr_num_received(v, received); + char text[192]; + int n = integer ? snprintf(text, sizeof text, + "The value of \"%s\" is out of range. It must be >= -2147483648 && <= 2147483647. " + "Received %s", + name, received) + : snprintf(text, sizeof text, + "The value of \"%s\" is out of range. It must be an integer. Received %s", name, + received); + scr_throw_error_msg_code(SCR_ERR_RANGE, text, n < 0 ? 0 : (size_t)n, "ERR_OUT_OF_RANGE"); + return false; +} + +/* A plain view of a statically typed argument (+1), or NULL after a + * throw. */ +static ScrDyn *ci_plain(ScrDyn *value) { + if (value->kind == SCR_DYN_TYPED_REF) return scr_dyn_typed_ref_materialize(value); + return scr_dyn_retain(value); +} + +static const ScCipherInfo *ci_by_name(const ScrStr *name) { + char lower[64]; + if (name->len == 0 || name->len >= sizeof lower) return NULL; + for (size_t i = 0; i < name->len; i++) { + unsigned char ch = (unsigned char)name->data[i]; + lower[i] = (char)(ch >= 'A' && ch <= 'Z' ? ch + 32 : ch); + } + lower[name->len] = '\0'; + size_t lo = 0, hi = CI_COUNT(sc_cipher_info_names); + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + int cmp = strcmp(sc_cipher_info_names[mid].name, lower); + if (cmp == 0) return &sc_cipher_infos[sc_cipher_info_names[mid].info]; + if (cmp < 0) lo = mid + 1; + else hi = mid; + } + return NULL; +} + +static const ScCipherInfo *ci_by_nid(double nid) { + size_t lo = 0, hi = CI_COUNT(sc_cipher_info_nids); + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + double at = sc_cipher_info_nids[mid].nid; + if (at == nid) return &sc_cipher_infos[sc_cipher_info_nids[mid].info]; + if (at < nid) lo = mid + 1; + else hi = mid; + } + return NULL; +} + +/* GetCipherInfo's option rules: a cipher that takes neither option (the + * legacy provider's) answers undefined for either; keyLength must equal + * the cipher's; ivLength must equal its IV length, or lie in 7 to 13 for + * CCM and 1 to 15 for OCB, and GCM takes any. */ +static bool ci_accepts(const ScCipherInfo *info, bool has_key, int key, bool has_iv, int iv) { + if ((has_key || has_iv) && (info->key_rule == SC_INFO_NONE || info->iv_rule == SC_INFO_NONE)) return false; + if (has_key && key != info->key_length) return false; + if (!has_iv) return true; + switch (info->iv_rule) { + case SC_INFO_CCM: return iv >= 7 && iv <= 13; + case SC_INFO_OCB: return iv >= 1 && iv <= 15; + case SC_INFO_ANY: return true; + default: return iv == (info->iv_length < 0 ? 0 : info->iv_length); + } +} + +static void ci_set_str(ScrDyn *obj, const char *key, const char *text) { + ScrStr *s = scr_str_new(text, strlen(text)); + ScrDyn *v = scr_dyn_new_str(s); + scr_str_release(s); + scr_dyn_obj_set(obj, key, strlen(key), v); +} + +static void ci_set_num(ScrDyn *obj, const char *key, double value) { + scr_dyn_obj_set(obj, key, strlen(key), scr_dyn_new_num(value)); +} + +/* The info object: mode, name, nid, blockSize (not for stream ciphers), + * ivLength (not when it is 0) and keyLength. */ +static ScrDyn *ci_object(const ScCipherInfo *info, int iv) { + ScrDyn *obj = scr_dyn_new_obj(); + if (info->mode) ci_set_str(obj, "mode", info->mode); + ci_set_str(obj, "name", info->name); + if (info->nid >= 0) ci_set_num(obj, "nid", info->nid); + if (info->block_size >= 0) ci_set_num(obj, "blockSize", info->block_size); + if (iv > 0) ci_set_num(obj, "ivLength", iv); + ci_set_num(obj, "keyLength", info->key_length); + return obj; +} + +/* Whether options is a class instance whose class defines keyLength or + * ivLength, which the runtime cannot read through the class's accessors. */ +static bool ci_class_option(const ScrDyn *options) { + if (options->kind != SCR_DYN_TYPED_REF) return false; + bool found = false; + static const char *const keys[] = {"keyLength", "ivLength"}; + for (size_t i = 0; i < 2 && !found; i++) { + ScrStr *key = scr_str_new(keys[i], strlen(keys[i])); + found = scr_dyn_typed_ref_class_has(options, key); + scr_str_release(key); + } + return found; +} + +/* An option as `const { keyLength, ivLength } = options` reads it: own and + * inherited properties, through getters. +1, or NULL with the getter's + * error pending. */ +static ScrDyn *ci_option(const ScrDyn *options, const char *key) { + if (options->kind == SCR_DYN_OBJ || options->kind == SCR_DYN_PROXY) + return scr_dyn_obj_read(options, key, strlen(key)); + return scr_dyn_retain((ScrDyn *)scr_dyn_undefined()); +} + +static ScrDyn *ci_get(const ScrDyn *name_or_nid, const ScrDyn *options, bool class_option) { + if (name_or_nid->kind != SCR_DYN_STR && name_or_nid->kind != SCR_DYN_NUM) { + scr_dyn_arg_type_fail("nameOrNid", "one of type string or number", name_or_nid); + return NULL; + } + if (name_or_nid->kind == SCR_DYN_NUM && !ci_validate_int32("nameOrNid", name_or_nid)) return NULL; + bool has_key = false, has_iv = false; + int key = 0, iv = 0; + if (options->kind != SCR_DYN_UNDEF) { + ScrStr *type = scr_dyn_typeof(options); + bool object = type->len == 6 && memcmp(type->data, "object", 6) == 0; + scr_str_release(type); + if (!object || options->kind == SCR_DYN_NULL || options->kind == SCR_DYN_ARR) { + scr_dyn_arg_type_fail("options", "of type object", options); + return NULL; + } + if (class_option) { + static const char message[] = "getCipherInfo options whose class defines keyLength or ivLength have no " + "lowering; pass an object with those properties"; + scr_throw_error_msg_code(SCR_ERR_ERROR, message, sizeof message - 1, "SC2020"); + return NULL; + } + /* Both are read before either is validated, as in Node. */ + ScrDyn *key_value = ci_option(options, "keyLength"); + if (!key_value) return NULL; + ScrDyn *iv_value = ci_option(options, "ivLength"); + if (!iv_value) { + scr_dyn_release(key_value); + return NULL; + } + bool ok = true; + if (key_value->kind != SCR_DYN_UNDEF) { + ok = ci_validate_int32("options.keyLength", key_value); + has_key = true; + key = ok ? (int)key_value->v.num : 0; + } + if (ok && iv_value->kind != SCR_DYN_UNDEF) { + ok = ci_validate_int32("options.ivLength", iv_value); + has_iv = true; + iv = ok ? (int)iv_value->v.num : 0; + } + scr_dyn_release(key_value); + scr_dyn_release(iv_value); + if (!ok) return NULL; + } + const ScCipherInfo *info = + name_or_nid->kind == SCR_DYN_STR ? ci_by_name(name_or_nid->v.str) : ci_by_nid(name_or_nid->v.num); + if (!info || !ci_accepts(info, has_key, key, has_iv, iv)) return scr_dyn_retain((ScrDyn *)scr_dyn_undefined()); + return ci_object(info, has_iv ? iv : info->iv_length); +} + +ScrDyn *scr_crypto_get_cipher_info(ScrDyn *name_or_nid, ScrDyn *options) { + ScrDyn *name = ci_plain(name_or_nid); + if (!name) return NULL; + ScrDyn *opts = ci_plain(options); + ScrDyn *result = opts ? ci_get(name, opts, ci_class_option(options)) : NULL; + if (opts) scr_dyn_release(opts); + scr_dyn_release(name); + return result; +} diff --git a/packages/runtime/src/scr_cipher_info.h b/packages/runtime/src/scr_cipher_info.h new file mode 100644 index 000000000..edb18caea --- /dev/null +++ b/packages/runtime/src/scr_cipher_info.h @@ -0,0 +1,615 @@ +/* Generated by scripts/gen-cipher-info.mjs from Node v24.22.0's + * crypto.getCipherInfo: every cipher OpenSSL names, with the fields Node + * reports (-1 or NULL for a field it leaves out) and the keyLength and + * ivLength options each accepts. Do not edit. */ +#ifndef SCR_CIPHER_INFO_H +#define SCR_CIPHER_INFO_H + +enum { SC_INFO_NONE, SC_INFO_EXACT, SC_INFO_CCM, SC_INFO_OCB, SC_INFO_ANY }; + +typedef struct { + const char *mode; + const char *name; + int nid; + int block_size; + int iv_length; + int key_length; + unsigned char key_rule; + unsigned char iv_rule; +} ScCipherInfo; + +static const ScCipherInfo sc_cipher_infos[] = { + {"ecb", "camellia-128-ecb", 754, 16, -1, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "camellia-192-ecb", 755, 16, -1, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "camellia-192-ofb", 767, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-192-cfb", 758, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "camellia-192-ctr", 967, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "camellia-128-ofb", 766, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-128-cfb", 757, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "camellia-256-ecb", 756, 16, -1, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "camellia-256-ofb", 768, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-256-cfb", 759, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "camellia-256-ctr", 971, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "camellia-128-ctr", 963, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "sm4-ecb", 1133, 16, -1, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "sm4-cbc", 1134, 16, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "sm4-ofb", 1135, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "sm4-cfb", 1137, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "sm4-ctr", 1139, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "camellia-128-cbc", 751, 16, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "camellia-192-cbc", 752, 16, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "camellia-256-cbc", 753, 16, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "aria-128-ecb", 1065, 16, -1, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "aria-192-ctr", 1074, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "aria-256-ecb", 1075, 16, -1, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aria-256-cbc", 1076, 16, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-256-cfb", 1077, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "aria-256-ofb", 1078, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "aria-256-ctr", 1079, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aria-128-cbc", 1066, 16, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-128-cfb", 1067, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"gcm", "aria-128-gcm", 1123, 1, 12, 16, SC_INFO_EXACT, SC_INFO_ANY}, + {"gcm", "aria-192-gcm", 1124, 1, 12, 24, SC_INFO_EXACT, SC_INFO_ANY}, + {"gcm", "aria-256-gcm", 1125, 1, 12, 32, SC_INFO_EXACT, SC_INFO_ANY}, + {"ccm", "aria-128-ccm", 1120, 1, 12, 16, SC_INFO_EXACT, SC_INFO_CCM}, + {"ccm", "aria-192-ccm", 1121, 1, 12, 24, SC_INFO_EXACT, SC_INFO_CCM}, + {"ccm", "aria-256-ccm", 1122, 1, 12, 32, SC_INFO_EXACT, SC_INFO_CCM}, + {"ofb", "aria-128-ofb", 1068, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "aria-128-ctr", 1069, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "aria-192-ecb", 1070, 16, -1, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aria-192-cbc", 1071, 16, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-192-cfb", 1072, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "aria-192-ofb", 1073, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "id-smime-alg-cms3deswrap", 246, 8, -1, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "des-ede3-cbc", 44, 8, 8, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"xts", "aes-128-xts", 913, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"xts", "aes-256-xts", 914, 1, 16, 64, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "des-ede", 32, 8, -1, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "aes-128-ecb", 418, 16, -1, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aes-128-cbc", 419, 16, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "aes-192-ecb", 422, 16, -1, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aes-192-cbc", 423, 16, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "aes-192-ofb", 424, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-192-cfb", 425, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "id-aes192-wrap", 789, 8, 8, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"gcm", "id-aes192-gcm", 898, 1, 12, 24, SC_INFO_EXACT, SC_INFO_ANY}, + {"ccm", "id-aes192-ccm", 899, 1, 12, 24, SC_INFO_EXACT, SC_INFO_CCM}, + {"wrap", "id-aes192-wrap-pad", 900, 8, 4, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "aes-128-ofb", 420, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-128-cfb", 421, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "aes-256-ecb", 426, 16, -1, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aes-256-cbc", 427, 16, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "aes-256-ofb", 428, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-256-cfb", 429, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "id-aes256-wrap", 790, 8, 8, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"gcm", "id-aes256-gcm", 901, 1, 12, 32, SC_INFO_EXACT, SC_INFO_ANY}, + {"ccm", "id-aes256-ccm", 902, 1, 12, 32, SC_INFO_EXACT, SC_INFO_CCM}, + {"wrap", "id-aes256-wrap-pad", 903, 8, 4, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "id-aes128-wrap", 788, 8, 8, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"gcm", "id-aes128-gcm", 895, 1, 12, 16, SC_INFO_EXACT, SC_INFO_ANY}, + {"ccm", "id-aes128-ccm", 896, 1, 12, 16, SC_INFO_EXACT, SC_INFO_CCM}, + {"wrap", "id-aes128-wrap-pad", 897, 8, 4, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aes-128-cbc-cts", -1, 16, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-128-cfb1", 650, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-128-cfb8", 653, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "aes-128-ctr", 904, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"gcm-siv", "aes-128-gcm-siv", -1, 1, 12, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ocb", "aes-128-ocb", 958, 16, 12, 16, SC_INFO_EXACT, SC_INFO_OCB}, + {"siv", "aes-128-siv", -1, 1, -1, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "aes-128-wrap-inv", -1, 8, 8, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "aes-128-wrap-pad-inv", -1, 8, 4, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aes-192-cbc-cts", -1, 16, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-192-cfb1", 651, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-192-cfb8", 654, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "aes-192-ctr", 905, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"gcm-siv", "aes-192-gcm-siv", -1, 1, 12, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ocb", "aes-192-ocb", 959, 16, 12, 24, SC_INFO_EXACT, SC_INFO_OCB}, + {"siv", "aes-192-siv", -1, 1, -1, 48, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "aes-192-wrap-inv", -1, 8, 8, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "aes-192-wrap-pad-inv", -1, 8, 4, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "aes-256-cbc-cts", -1, 16, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-256-cfb1", 652, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aes-256-cfb8", 655, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ctr", "aes-256-ctr", 906, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"gcm-siv", "aes-256-gcm-siv", -1, 1, 12, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ocb", "aes-256-ocb", 960, 16, 12, 32, SC_INFO_EXACT, SC_INFO_OCB}, + {"siv", "aes-256-siv", -1, 1, -1, 64, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "aes-256-wrap-inv", -1, 8, 8, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"wrap", "aes-256-wrap-pad-inv", -1, 8, 4, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-128-cfb1", 1080, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-128-cfb8", 1083, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-192-cfb1", 1081, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-192-cfb8", 1084, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-256-cfb1", 1082, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "aria-256-cfb8", 1085, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "bf-cbc", 91, 8, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "bf-cfb", 93, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ecb", "bf-ecb", 92, 8, -1, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ofb", "bf-ofb", 94, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "camellia-128-cbc-cts", -1, 16, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-128-cfb1", 760, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-128-cfb8", 763, 1, 16, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "camellia-192-cbc-cts", -1, 16, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-192-cfb1", 761, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-192-cfb8", 764, 1, 16, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "camellia-256-cbc-cts", -1, 16, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-256-cfb1", 762, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "camellia-256-cfb8", 765, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "cast5-cbc", 108, 8, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "cast5-cfb", 110, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ecb", "cast5-ecb", 109, 8, -1, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ofb", "cast5-ofb", 111, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"stream", "chacha20", 1019, -1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"stream", "chacha20-poly1305", 1018, -1, 12, 32, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cbc", "des-cbc", 31, 8, 8, 8, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "des-cfb", 30, 1, 8, 8, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "des-cfb1", 656, 1, 8, 8, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "des-cfb8", 657, 1, 8, 8, SC_INFO_NONE, SC_INFO_NONE}, + {"ecb", "des-ecb", 29, 8, -1, 8, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "des-ede-cbc", 43, 8, 8, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "des-ede-cfb", 60, 1, 8, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "des-ede-ofb", 62, 1, 8, 16, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ecb", "des-ede3", 33, 8, -1, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "des-ede3-cfb", 61, 1, 8, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "des-ede3-cfb1", 658, 1, 8, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"cfb", "des-ede3-cfb8", 659, 1, 8, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "des-ede3-ofb", 63, 1, 8, 24, SC_INFO_EXACT, SC_INFO_EXACT}, + {"ofb", "des-ofb", 45, 1, 8, 8, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "desx-cbc", 80, 8, 8, 24, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "idea-cbc", 34, 8, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "idea-cfb", 35, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ecb", "idea-ecb", 36, 8, -1, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ofb", "idea-ofb", 46, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "rc2-cbc", 37, 8, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "rc2-40-cbc", 98, 8, 8, 5, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "rc2-64-cbc", 166, 8, 8, 8, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "rc2-cfb", 39, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ecb", "rc2-ecb", 38, 8, -1, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ofb", "rc2-ofb", 40, 1, 8, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"stream", "rc4", 5, -1, -1, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"stream", "rc4-40", 97, -1, -1, 5, SC_INFO_NONE, SC_INFO_NONE}, + {"stream", "rc4-hmac-md5", 915, -1, -1, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cbc", "seed-cbc", 777, 16, 16, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"cfb", "seed-cfb", 779, 1, 16, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ecb", "seed-ecb", 776, 16, -1, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ofb", "seed-ofb", 778, 1, 16, 16, SC_INFO_NONE, SC_INFO_NONE}, + {"ccm", "sm4-ccm", -1, 1, 12, 16, SC_INFO_EXACT, SC_INFO_CCM}, + {"gcm", "sm4-gcm", -1, 1, 12, 16, SC_INFO_EXACT, SC_INFO_ANY}, + {"xts", "sm4-xts", -1, 1, 16, 32, SC_INFO_EXACT, SC_INFO_EXACT}, +}; + +/* Lowercase names, sorted, with their record. */ +static const struct { + const char *name; + unsigned short info; +} sc_cipher_info_names[] = { + {"0.3.4401.5.3.1.9.1", 0}, + {"0.3.4401.5.3.1.9.21", 1}, + {"0.3.4401.5.3.1.9.23", 2}, + {"0.3.4401.5.3.1.9.24", 3}, + {"0.3.4401.5.3.1.9.29", 4}, + {"0.3.4401.5.3.1.9.3", 5}, + {"0.3.4401.5.3.1.9.4", 6}, + {"0.3.4401.5.3.1.9.41", 7}, + {"0.3.4401.5.3.1.9.43", 8}, + {"0.3.4401.5.3.1.9.44", 9}, + {"0.3.4401.5.3.1.9.49", 10}, + {"0.3.4401.5.3.1.9.9", 11}, + {"1.2.156.10197.1.104.1", 12}, + {"1.2.156.10197.1.104.2", 13}, + {"1.2.156.10197.1.104.3", 14}, + {"1.2.156.10197.1.104.4", 15}, + {"1.2.156.10197.1.104.7", 16}, + {"1.2.392.200011.61.1.1.1.2", 17}, + {"1.2.392.200011.61.1.1.1.3", 18}, + {"1.2.392.200011.61.1.1.1.4", 19}, + {"1.2.410.200046.1.1.1", 20}, + {"1.2.410.200046.1.1.10", 21}, + {"1.2.410.200046.1.1.11", 22}, + {"1.2.410.200046.1.1.12", 23}, + {"1.2.410.200046.1.1.13", 24}, + {"1.2.410.200046.1.1.14", 25}, + {"1.2.410.200046.1.1.15", 26}, + {"1.2.410.200046.1.1.2", 27}, + {"1.2.410.200046.1.1.3", 28}, + {"1.2.410.200046.1.1.34", 29}, + {"1.2.410.200046.1.1.35", 30}, + {"1.2.410.200046.1.1.36", 31}, + {"1.2.410.200046.1.1.37", 32}, + {"1.2.410.200046.1.1.38", 33}, + {"1.2.410.200046.1.1.39", 34}, + {"1.2.410.200046.1.1.4", 35}, + {"1.2.410.200046.1.1.5", 36}, + {"1.2.410.200046.1.1.6", 37}, + {"1.2.410.200046.1.1.7", 38}, + {"1.2.410.200046.1.1.8", 39}, + {"1.2.410.200046.1.1.9", 40}, + {"1.2.840.113549.1.9.16.3.6", 41}, + {"1.2.840.113549.3.7", 42}, + {"1.3.111.2.1619.0.1.1", 43}, + {"1.3.111.2.1619.0.1.2", 44}, + {"1.3.14.3.2.17", 45}, + {"2.16.840.1.101.3.4.1.1", 46}, + {"2.16.840.1.101.3.4.1.2", 47}, + {"2.16.840.1.101.3.4.1.21", 48}, + {"2.16.840.1.101.3.4.1.22", 49}, + {"2.16.840.1.101.3.4.1.23", 50}, + {"2.16.840.1.101.3.4.1.24", 51}, + {"2.16.840.1.101.3.4.1.25", 52}, + {"2.16.840.1.101.3.4.1.26", 53}, + {"2.16.840.1.101.3.4.1.27", 54}, + {"2.16.840.1.101.3.4.1.28", 55}, + {"2.16.840.1.101.3.4.1.3", 56}, + {"2.16.840.1.101.3.4.1.4", 57}, + {"2.16.840.1.101.3.4.1.41", 58}, + {"2.16.840.1.101.3.4.1.42", 59}, + {"2.16.840.1.101.3.4.1.43", 60}, + {"2.16.840.1.101.3.4.1.44", 61}, + {"2.16.840.1.101.3.4.1.45", 62}, + {"2.16.840.1.101.3.4.1.46", 63}, + {"2.16.840.1.101.3.4.1.47", 64}, + {"2.16.840.1.101.3.4.1.48", 65}, + {"2.16.840.1.101.3.4.1.5", 66}, + {"2.16.840.1.101.3.4.1.6", 67}, + {"2.16.840.1.101.3.4.1.7", 68}, + {"2.16.840.1.101.3.4.1.8", 69}, + {"aes-128-cbc", 47}, + {"aes-128-cbc-cts", 70}, + {"aes-128-ccm", 68}, + {"aes-128-cfb", 57}, + {"aes-128-cfb1", 71}, + {"aes-128-cfb8", 72}, + {"aes-128-ctr", 73}, + {"aes-128-ecb", 46}, + {"aes-128-gcm", 67}, + {"aes-128-gcm-siv", 74}, + {"aes-128-ocb", 75}, + {"aes-128-ofb", 56}, + {"aes-128-siv", 76}, + {"aes-128-wrap", 66}, + {"aes-128-wrap-inv", 77}, + {"aes-128-wrap-pad", 69}, + {"aes-128-wrap-pad-inv", 78}, + {"aes-128-xts", 43}, + {"aes-192-cbc", 49}, + {"aes-192-cbc-cts", 79}, + {"aes-192-ccm", 54}, + {"aes-192-cfb", 51}, + {"aes-192-cfb1", 80}, + {"aes-192-cfb8", 81}, + {"aes-192-ctr", 82}, + {"aes-192-ecb", 48}, + {"aes-192-gcm", 53}, + {"aes-192-gcm-siv", 83}, + {"aes-192-ocb", 84}, + {"aes-192-ofb", 50}, + {"aes-192-siv", 85}, + {"aes-192-wrap", 52}, + {"aes-192-wrap-inv", 86}, + {"aes-192-wrap-pad", 55}, + {"aes-192-wrap-pad-inv", 87}, + {"aes-256-cbc", 59}, + {"aes-256-cbc-cts", 88}, + {"aes-256-ccm", 64}, + {"aes-256-cfb", 61}, + {"aes-256-cfb1", 89}, + {"aes-256-cfb8", 90}, + {"aes-256-ctr", 91}, + {"aes-256-ecb", 58}, + {"aes-256-gcm", 63}, + {"aes-256-gcm-siv", 92}, + {"aes-256-ocb", 93}, + {"aes-256-ofb", 60}, + {"aes-256-siv", 94}, + {"aes-256-wrap", 62}, + {"aes-256-wrap-inv", 95}, + {"aes-256-wrap-pad", 65}, + {"aes-256-wrap-pad-inv", 96}, + {"aes-256-xts", 44}, + {"aes128", 47}, + {"aes128-wrap", 66}, + {"aes128-wrap-inv", 77}, + {"aes128-wrap-pad", 69}, + {"aes128-wrap-pad-inv", 78}, + {"aes192", 49}, + {"aes192-wrap", 52}, + {"aes192-wrap-inv", 86}, + {"aes192-wrap-pad", 55}, + {"aes192-wrap-pad-inv", 87}, + {"aes256", 59}, + {"aes256-wrap", 62}, + {"aes256-wrap-inv", 95}, + {"aes256-wrap-pad", 65}, + {"aes256-wrap-pad-inv", 96}, + {"aria-128-cbc", 27}, + {"aria-128-ccm", 32}, + {"aria-128-cfb", 28}, + {"aria-128-cfb1", 97}, + {"aria-128-cfb8", 98}, + {"aria-128-ctr", 36}, + {"aria-128-ecb", 20}, + {"aria-128-gcm", 29}, + {"aria-128-ofb", 35}, + {"aria-192-cbc", 38}, + {"aria-192-ccm", 33}, + {"aria-192-cfb", 39}, + {"aria-192-cfb1", 99}, + {"aria-192-cfb8", 100}, + {"aria-192-ctr", 21}, + {"aria-192-ecb", 37}, + {"aria-192-gcm", 30}, + {"aria-192-ofb", 40}, + {"aria-256-cbc", 23}, + {"aria-256-ccm", 34}, + {"aria-256-cfb", 24}, + {"aria-256-cfb1", 101}, + {"aria-256-cfb8", 102}, + {"aria-256-ctr", 26}, + {"aria-256-ecb", 22}, + {"aria-256-gcm", 31}, + {"aria-256-ofb", 25}, + {"aria128", 27}, + {"aria192", 38}, + {"aria256", 23}, + {"bf", 103}, + {"bf-cbc", 103}, + {"bf-cfb", 104}, + {"bf-ecb", 105}, + {"bf-ofb", 106}, + {"blowfish", 103}, + {"camellia-128-cbc", 17}, + {"camellia-128-cbc-cts", 107}, + {"camellia-128-cfb", 6}, + {"camellia-128-cfb1", 108}, + {"camellia-128-cfb8", 109}, + {"camellia-128-ctr", 11}, + {"camellia-128-ecb", 0}, + {"camellia-128-ofb", 5}, + {"camellia-192-cbc", 18}, + {"camellia-192-cbc-cts", 110}, + {"camellia-192-cfb", 3}, + {"camellia-192-cfb1", 111}, + {"camellia-192-cfb8", 112}, + {"camellia-192-ctr", 4}, + {"camellia-192-ecb", 1}, + {"camellia-192-ofb", 2}, + {"camellia-256-cbc", 19}, + {"camellia-256-cbc-cts", 113}, + {"camellia-256-cfb", 9}, + {"camellia-256-cfb1", 114}, + {"camellia-256-cfb8", 115}, + {"camellia-256-ctr", 10}, + {"camellia-256-ecb", 7}, + {"camellia-256-ofb", 8}, + {"camellia128", 17}, + {"camellia192", 18}, + {"camellia256", 19}, + {"cast", 116}, + {"cast-cbc", 116}, + {"cast5-cbc", 116}, + {"cast5-cfb", 117}, + {"cast5-ecb", 118}, + {"cast5-ofb", 119}, + {"chacha20", 120}, + {"chacha20-poly1305", 121}, + {"des", 122}, + {"des-cbc", 122}, + {"des-cfb", 123}, + {"des-cfb1", 124}, + {"des-cfb8", 125}, + {"des-ecb", 126}, + {"des-ede", 45}, + {"des-ede-cbc", 127}, + {"des-ede-cfb", 128}, + {"des-ede-ecb", 45}, + {"des-ede-ofb", 129}, + {"des-ede3", 130}, + {"des-ede3-cbc", 42}, + {"des-ede3-cfb", 131}, + {"des-ede3-cfb1", 132}, + {"des-ede3-cfb8", 133}, + {"des-ede3-ecb", 130}, + {"des-ede3-ofb", 134}, + {"des-ofb", 135}, + {"des3", 42}, + {"des3-wrap", 41}, + {"desx", 136}, + {"desx-cbc", 136}, + {"id-aes128-ccm", 68}, + {"id-aes128-gcm", 67}, + {"id-aes128-wrap", 66}, + {"id-aes128-wrap-pad", 69}, + {"id-aes192-ccm", 54}, + {"id-aes192-gcm", 53}, + {"id-aes192-wrap", 52}, + {"id-aes192-wrap-pad", 55}, + {"id-aes256-ccm", 64}, + {"id-aes256-gcm", 63}, + {"id-aes256-wrap", 62}, + {"id-aes256-wrap-pad", 65}, + {"id-smime-alg-cms3deswrap", 41}, + {"idea-cbc", 137}, + {"idea-cfb", 138}, + {"idea-ecb", 139}, + {"idea-ofb", 140}, + {"rc2", 141}, + {"rc2-128", 141}, + {"rc2-40", 142}, + {"rc2-40-cbc", 142}, + {"rc2-64", 143}, + {"rc2-64-cbc", 143}, + {"rc2-cbc", 141}, + {"rc2-cfb", 144}, + {"rc2-ecb", 145}, + {"rc2-ofb", 146}, + {"rc4", 147}, + {"rc4-40", 148}, + {"rc4-hmac-md5", 149}, + {"seed", 150}, + {"seed-cbc", 150}, + {"seed-cfb", 151}, + {"seed-ecb", 152}, + {"seed-ofb", 153}, + {"sm4", 13}, + {"sm4-cbc", 13}, + {"sm4-ccm", 154}, + {"sm4-cfb", 15}, + {"sm4-cfb128", 15}, + {"sm4-ctr", 16}, + {"sm4-ecb", 12}, + {"sm4-gcm", 155}, + {"sm4-ofb", 14}, + {"sm4-ofb128", 14}, + {"sm4-xts", 156}, +}; + +/* Nids, ascending, with their record. */ +static const struct { + unsigned short nid; + unsigned short info; +} sc_cipher_info_nids[] = { + {5, 147}, + {29, 126}, + {30, 123}, + {31, 122}, + {32, 45}, + {33, 130}, + {34, 137}, + {35, 138}, + {36, 139}, + {37, 141}, + {38, 145}, + {39, 144}, + {40, 146}, + {43, 127}, + {44, 42}, + {45, 135}, + {46, 140}, + {60, 128}, + {61, 131}, + {62, 129}, + {63, 134}, + {80, 136}, + {91, 103}, + {92, 105}, + {93, 104}, + {94, 106}, + {97, 148}, + {98, 142}, + {108, 116}, + {109, 118}, + {110, 117}, + {111, 119}, + {166, 143}, + {246, 41}, + {418, 46}, + {419, 47}, + {420, 56}, + {421, 57}, + {422, 48}, + {423, 49}, + {424, 50}, + {425, 51}, + {426, 58}, + {427, 59}, + {428, 60}, + {429, 61}, + {650, 71}, + {651, 80}, + {652, 89}, + {653, 72}, + {654, 81}, + {655, 90}, + {656, 124}, + {657, 125}, + {658, 132}, + {659, 133}, + {751, 17}, + {752, 18}, + {753, 19}, + {754, 0}, + {755, 1}, + {756, 7}, + {757, 6}, + {758, 3}, + {759, 9}, + {760, 108}, + {761, 111}, + {762, 114}, + {763, 109}, + {764, 112}, + {765, 115}, + {766, 5}, + {767, 2}, + {768, 8}, + {776, 152}, + {777, 150}, + {778, 153}, + {779, 151}, + {788, 66}, + {789, 52}, + {790, 62}, + {895, 67}, + {896, 68}, + {897, 69}, + {898, 53}, + {899, 54}, + {900, 55}, + {901, 63}, + {902, 64}, + {903, 65}, + {904, 73}, + {905, 82}, + {906, 91}, + {913, 43}, + {914, 44}, + {915, 149}, + {958, 75}, + {959, 84}, + {960, 93}, + {963, 11}, + {967, 4}, + {971, 10}, + {1018, 121}, + {1019, 120}, + {1065, 20}, + {1066, 27}, + {1067, 28}, + {1068, 35}, + {1069, 36}, + {1070, 37}, + {1071, 38}, + {1072, 39}, + {1073, 40}, + {1074, 21}, + {1075, 22}, + {1076, 23}, + {1077, 24}, + {1078, 25}, + {1079, 26}, + {1080, 97}, + {1081, 99}, + {1082, 101}, + {1083, 98}, + {1084, 100}, + {1085, 102}, + {1120, 32}, + {1121, 33}, + {1122, 34}, + {1123, 29}, + {1124, 30}, + {1125, 31}, + {1133, 12}, + {1134, 13}, + {1135, 14}, + {1137, 15}, + {1139, 16}, + {1198, 76}, + {1199, 85}, + {1200, 94}, + {1248, 155}, + {1249, 154}, + {1290, 156}, +}; + +#endif diff --git a/packages/runtime/src/scr_console.c b/packages/runtime/src/scr_console.c index 6def8d866..086340014 100644 --- a/packages/runtime/src/scr_console.c +++ b/packages/runtime/src/scr_console.c @@ -148,7 +148,7 @@ static void scr_enable_utf8_console_output(void) { } #endif -void scr_runtime_abi_v36(void) {} +void scr_runtime_abi_v37(void) {} void scr_init(void) { #ifdef SCR_WORKERS diff --git a/packages/runtime/src/scr_core.h b/packages/runtime/src/scr_core.h index f9cc8cc5f..3707b8d50 100644 --- a/packages/runtime/src/scr_core.h +++ b/packages/runtime/src/scr_core.h @@ -81,7 +81,7 @@ void scr_init(void); /* Program objects emitted by the bundled LLVM helper reference this symbol. * Its versioned spelling makes a mismatched manual runtime link fail before * the program can start. */ -void scr_runtime_abi_v36(void); +void scr_runtime_abi_v37(void); void scr_runtime_workers_v8(void); /* ── the trap funnel (scr_console.c; scr_library.c under -DSCR_LIB) ────── diff --git a/packages/runtime/src/scr_crypto.h b/packages/runtime/src/scr_crypto.h index 0e6c6e07f..f61d27406 100644 --- a/packages/runtime/src/scr_crypto.h +++ b/packages/runtime/src/scr_crypto.h @@ -108,6 +108,9 @@ ScrDyn *scr_crypto_ecdh_convert_key(ScrDyn *key, ScrDyn *curve, ScrDyn *in_enc, * Node's error pending. */ ScrDyn *scr_crypto_dh_new(ScrDyn *size_or_key, ScrDyn *key_encoding, ScrDyn *generator, ScrDyn *generator_encoding); ScrDyn *scr_crypto_dh_group_new(ScrDyn *name); +/* crypto.getCipherInfo(nameOrNid, options) (scr_cipher_info.c): the info + * object or undefined (+1), or NULL with Node's argument error pending. */ +ScrDyn *scr_crypto_get_cipher_info(ScrDyn *name_or_nid, ScrDyn *options); /* crypto.checkPrimeSync, checkPrime, generatePrimeSync and generatePrime * (scr_dh.c): checked-dynamic arguments, borrowed; the sync forms answer * +1 or NULL with the error pending. The callbacks receive undefined as diff --git a/packages/runtime/src/scr_cycle.c b/packages/runtime/src/scr_cycle.c index 6dcd546b0..dbd636306 100644 --- a/packages/runtime/src/scr_cycle.c +++ b/packages/runtime/src/scr_cycle.c @@ -170,7 +170,8 @@ void scr_weak_dispose(void *object) { #include /* Every heap object's first member is `size_t rc`; the collector works on - * its count (SCR_RC_COUNT). */ + * its count (SCR_RC_COUNT), and tests it without a frozen record's bit + * (SCR_RC_REFS). */ #define SCR_RC(obj) SCR_RC_COUNT(obj) static void scr_cyc_oom(void) { @@ -293,24 +294,22 @@ static SCR_TL ScrDestroyEntry *scr_destroy_pending; static SCR_TL size_t scr_destroy_count, scr_destroy_capacity; #endif -void scr_rc_destroy(void *obj, void (*destroy)(void *)) { - if (scr_destroy_depth == SCR_DESTROY_DEPTH) { - if (scr_destroy_count == scr_destroy_capacity) { - size_t capacity = scr_destroy_capacity ? scr_destroy_capacity * 2 : 64; - if (capacity < scr_destroy_capacity || capacity > SIZE_MAX / sizeof(ScrDestroyEntry)) - scr_cyc_oom(); - ScrDestroyEntry *entries = realloc(scr_destroy_pending, capacity * sizeof(*entries)); - if (!entries) scr_cyc_oom(); - scr_destroy_pending = entries; - scr_destroy_capacity = capacity; - } - scr_destroy_pending[scr_destroy_count++] = (ScrDestroyEntry){ obj, destroy }; - return; +/* Queues a disposal past the depth budget; the outermost disposal runs it. */ +static void scr_rc_destroy_defer(void *obj, void (*destroy)(void *)) { + if (scr_destroy_count == scr_destroy_capacity) { + size_t capacity = scr_destroy_capacity ? scr_destroy_capacity * 2 : 64; + if (capacity < scr_destroy_capacity || capacity > SIZE_MAX / sizeof(ScrDestroyEntry)) + scr_cyc_oom(); + ScrDestroyEntry *entries = realloc(scr_destroy_pending, capacity * sizeof(*entries)); + if (!entries) scr_cyc_oom(); + scr_destroy_pending = entries; + scr_destroy_capacity = capacity; } - scr_destroy_depth++; - destroy(obj); - scr_destroy_depth--; - if (scr_destroy_depth) return; + scr_destroy_pending[scr_destroy_count++] = (ScrDestroyEntry){ obj, destroy }; +} + +/* Runs the queued disposals once the outermost one has returned. */ +static void scr_rc_destroy_drain(void) { while (scr_destroy_count) { ScrDestroyEntry entry = scr_destroy_pending[--scr_destroy_count]; scr_destroy_depth = 1; @@ -326,6 +325,30 @@ void scr_rc_destroy(void *obj, void (*destroy)(void *)) { } } +void scr_rc_destroy(void *obj, void (*destroy)(void *)) { + if (scr_destroy_depth == SCR_DESTROY_DEPTH) { + scr_rc_destroy_defer(obj, destroy); + return; + } + scr_destroy_depth++; + destroy(obj); + scr_destroy_depth--; + if (scr_destroy_depth == 0) scr_rc_destroy_drain(); +} + +void *scr_rc_destroy_seg(void *obj, void *(*destroy_seg)(void *, void *), + void (*destroy)(void *), void *tail) { + if (scr_destroy_depth == SCR_DESTROY_DEPTH) { + scr_rc_destroy_defer(obj, destroy); + return tail; + } + scr_destroy_depth++; + tail = destroy_seg(obj, tail); + scr_destroy_depth--; + if (scr_destroy_depth == 0) scr_rc_destroy_drain(); + return tail; +} + /* Live cycle-headered objects and the running count of OLD objects freed. * The inline allocation/free paths (scr_runtime.h, llvm/alloc.ts) maintain * both counters so full passes can measure old-generation reclamation. */ @@ -1308,7 +1331,8 @@ static void scr_mg_visit(void *child, void *ctx) { scr_mg_skip_edge(child); return; } - size_t rc = --SCR_RC(child); + --SCR_RC(child); + size_t rc = SCR_RC_REFS(child); if (scr_cyc_color(h) == SCR_CYC_GRAY) { if (rc == 0) { scr_gray_referenced--; @@ -1336,7 +1360,7 @@ static inline __attribute__((always_inline)) void scr_mark_gray(void *obj, uintp )) h->buf = scr_gray_comp; if (scr_white.n < SCR_CYC_GRAY_LIST_MAX) scr_cyc_push_gray(&scr_white, obj); - if (SCR_RC(obj) > 0) { + if (SCR_RC_REFS(obj) > 0) { scr_gray_referenced++; scr_comp_referenced++; } @@ -1408,7 +1432,7 @@ static void scr_scan_visit(void *child, void *ctx) { static void scr_scan(void *obj, uintptr_t depth) { ScrCycHdr *h = scr_cyc_hdr(obj); if (scr_cyc_color(h) != SCR_CYC_GRAY) return; - if (SCR_RC(obj) > 0) { + if (SCR_RC_REFS(obj) > 0) { scr_restore_from(obj); return; } @@ -1476,7 +1500,7 @@ static void scr_xg_destroy(void *obj) { } static SCR_CYC_COMPACT void scr_xg_release(void *obj) { - if (SCR_RC(obj) > 1) { + if (SCR_RC_REFS(obj) > 1) { SCR_RC(obj) -= 1; scr_cyc_on_release(obj); /* lost a reference: a possible cycle root */ return; @@ -1768,14 +1792,14 @@ static __attribute__((noinline)) size_t scr_yin_collect(void) { if (scr_cyc_gen(h) != SCR_CYC_YOUNG || !scr_cyc_is_buffered(h) || scr_cyc_buf_index(h) != k || scr_cyc_color(h) != SCR_CYC_BLACK) scr_trap("scriptc: corrupt young nursery entry\n"); - if (SCR_RC(v[k]) == 0 || SCR_RC(v[k]) < scr_yin_counts[k]) + if (SCR_RC_REFS(v[k]) == 0 || SCR_RC_REFS(v[k]) < scr_yin_counts[k]) scr_trap("scriptc: young object with uncounted references\n"); } #endif for (size_t k = 0; k < n; k++) { void *obj = v[k]; ScrCycHdr *h = scr_cyc_hdr(obj); - if (scr_cyc_color(h) == SCR_CYC_GRAY || SCR_RC(obj) <= scr_yin_counts[k]) continue; + if (scr_cyc_color(h) == SCR_CYC_GRAY || SCR_RC_REFS(obj) <= scr_yin_counts[k]) continue; scr_cyc_set_color(h, SCR_CYC_GRAY); scr_yin_push(&scr_yin_stack, obj); while (scr_yin_stack.n) { @@ -1799,7 +1823,7 @@ static __attribute__((noinline)) size_t scr_yin_collect(void) { scr_cyc_set_gen(h, SCR_CYC_YIN_PROMOTED); scr_cyc_clear_buffered(h); /* An outside root becomes a candidate (see "candidates" above). */ - if (SCR_RC(obj) > scr_yin_counts[k]) scr_cyc_on_release(obj); + if (SCR_RC_REFS(obj) > scr_yin_counts[k]) scr_cyc_on_release(obj); scr_yin_met(h->id, false); } else { scr_yin_met(h->id, true); @@ -2147,7 +2171,7 @@ static size_t scr_cyc_pass(unsigned gen_limit) { for (size_t j = start; j < end; j++) { void *obj = scr_white.v[j]; if (scr_cyc_color(scr_cyc_hdr(obj)) != SCR_CYC_GRAY) continue; - if (SCR_RC(obj) > 0) scr_restore_from(obj); + if (SCR_RC_REFS(obj) > 0) scr_restore_from(obj); else scr_white.v[kept++] = obj; } from = end; @@ -2176,7 +2200,7 @@ static size_t scr_cyc_pass(unsigned gen_limit) { * whitened only to be restored again. */ for (size_t i = 0; i < scr_cands.n; i++) { void *obj = scr_cands.v[i]; - if (scr_cyc_color(scr_cyc_hdr(obj)) == SCR_CYC_GRAY && SCR_RC(obj) > 0) scr_restore_from(obj); + if (scr_cyc_color(scr_cyc_hdr(obj)) == SCR_CYC_GRAY && SCR_RC_REFS(obj) > 0) scr_restore_from(obj); } for (size_t i = 0; i < scr_cands.n; i++) { void *obj = scr_cands.v[i]; diff --git a/packages/runtime/src/scr_dyn.h b/packages/runtime/src/scr_dyn.h index c3a6a8f38..73dad7bd3 100644 --- a/packages/runtime/src/scr_dyn.h +++ b/packages/runtime/src/scr_dyn.h @@ -200,6 +200,8 @@ typedef enum { X(SHARED_ARRAY_BUFFER, 37) \ X(SIGN, 67) /* node:crypto Sign (scr_pki.c) */ \ X(STDIO, 27) /* stable process stdin/stdout/stderr values */ \ + X(TEXT_DECODER, 76) /* compiled TextDecoder record (scr_text_codec_dyn.c) */ \ + X(TEXT_ENCODER, 77) /* compiled TextEncoder record (scr_text_codec_dyn.c) */ \ X(URL, 28) /* native WHATWG URL */ \ X(VERIFY, 68) /* node:crypto Verify (scr_pki.c) */ \ X(WASM_GLOBAL, 61) /* WebAssembly.Global */ \ @@ -212,6 +214,9 @@ typedef enum { X(WEB_CONTROLLER, 11) /* native ReadableStreamDefaultController */ \ X(WEB_READER, 10) /* native ReadableStreamDefaultReader */ \ X(WEB_STREAM, 9) /* native WHATWG ReadableStream */ \ + X(WEB_TRANSFORM, 73) /* native identity TransformStream (scr_fetch.c) */ \ + X(WEB_WRITABLE, 74) /* a TransformStream's WritableStream (scr_fetch.c) */ \ + X(WEB_WRITER, 75) /* WritableStreamDefaultWriter (scr_fetch.c) */ \ X(WORKER, 35) typedef enum { @@ -395,6 +400,25 @@ typedef struct ScrDynTypedClassOps { ScrStr *class_name; /* immortal constructor name; NULL for an anonymous class */ } ScrDynTypedClassOps; +/* A compiled JavaScript class whose instances record their own properties + * in a property bag and whose views inherit from the class's prototype + * data, emitted per class by the compiler. The view of such an instance is + * a fresh object holding exactly the bag's string-keyed properties, where + * each declared field among them takes its value from the native slot, and + * whose prototype is the class's. A method call can therefore resolve + * without building the view (scr_dyn_typed_ref_bag_method). + * bag: the instance's bag, borrowed; NULL or undefined when it has none; + * prototype: +1 prototype a fresh view of the instance takes; + * names: the declared fields whose values come from native slots; + * field: +1 view value of declared field `index`. */ +typedef struct ScrDynTypedBagClassOps { + ScrDyn *(*bag)(void *instance); + ScrDyn *(*prototype)(void *instance); + size_t count; + ScrStr *const *names; /* immortal field-name strings */ + ScrDyn *(*field)(void *instance, size_t index); +} ScrDynTypedBagClassOps; + struct ScrDyn { size_t rc; /* SIZE_MAX = immortal (unused for dyn; kept per convention) */ ScrDynKind kind; @@ -479,7 +503,9 @@ struct ScrDyn { * keep their ScrDynTypedRecordOps here instead, flagged by `record` * (scr_dyn_typed_ref_bind_record), and class instances their * ScrDynTypedClassOps, flagged by `klass` - * (scr_dyn_typed_ref_bind_class). NULL for other referents. */ + * (scr_dyn_typed_ref_bind_class), or their ScrDynTypedBagClassOps, + * flagged by `bag_class` (scr_dyn_typed_ref_bind_bag_class). NULL for + * other referents. */ const ScrDynTypedArrayOps *array; /* Both caches belong to this capsule. ReadableStream canonicalizes * capsules for repeated source references, so the refreshed dyn view @@ -488,11 +514,12 @@ struct ScrDyn { ScrDyn *materialized; ScrDynTypedCast *casts; /* One word keeps this, the largest payload, at its previous size. */ - uint32_t type_key_len : 28; + uint32_t type_key_len : 27; uint32_t traced : 1; /* native referent has a cycle header */ uint32_t observed : 1; /* native release reports this identity's disposal */ uint32_t record : 1; /* `array` holds ScrDynTypedRecordOps */ uint32_t klass : 1; /* `array` holds ScrDynTypedClassOps */ + uint32_t bag_class : 1; /* `array` holds ScrDynTypedBagClassOps */ } typed_ref; /* SCR_DYN_PROMISE: the retained promise. The boundary contract: it * settles with a dyn payload (SCR_EXC_REF ScrDyn fulfillment or a @@ -518,6 +545,10 @@ _Static_assert(offsetof(ScrDyn, null_proto) == sizeof(size_t) + sizeof(ScrDynKin "dyn-to-record conversions read null_proto at byte 13 (9 on wasm32)"); _Static_assert(offsetof(ScrDyn, v.typed_ref.type_key) == 16 + 3 * sizeof(void *), "typed_ref.type_key offset"); +/* Emitted brand probes read type_key_len as the low 27 bits of the 32-bit + * word after `casts` (native 88, wasm32 52). */ +_Static_assert(offsetof(ScrDyn, v.typed_ref.casts) == 16 + 8 * sizeof(void *), + "typed_ref.casts offset"); /* The region links ride in payload space the capsule variant already * reserves, so dyn nodes keep their size (and allocation size class). */ _Static_assert(sizeof(((ScrDyn *)0)->v.arr) <= sizeof(((ScrDyn *)0)->v.typed_ref) && @@ -629,7 +660,9 @@ static inline const char *scr_dyn_class_name(const ScrDyn *d) { /* The element hooks of a live array capsule, or NULL (including for record * and class capsules, which keep their hooks in the same slot). */ static inline const ScrDynTypedArrayOps *scr_dyn_typed_ref_array_ops(const ScrDyn *d) { - return d->v.typed_ref.record || d->v.typed_ref.klass ? NULL : d->v.typed_ref.array; + return d->v.typed_ref.record || d->v.typed_ref.klass || d->v.typed_ref.bag_class + ? NULL + : d->v.typed_ref.array; } /* BORROWED member lookup on a SCR_DYN_OBJ; NULL when the key is absent. */ @@ -935,6 +968,8 @@ ScrDyn *scr_dyn_new_obj_null_proto(void); ScrDyn *scr_dyn_obj_create(ScrDyn *prototype); ScrDyn *scr_dyn_obj_create_with_properties(ScrDyn *prototype, ScrDyn *descriptors); ScrDyn *scr_dyn_get_prototype_for_identity(ScrDyn *object); +bool scr_dyn_bag_create_field(ScrDyn *bag, const ScrStr *key); +bool scr_dyn_is_class_capsule(const ScrDyn *d); ScrDyn *scr_dyn_get_prototype(ScrDyn *object); ScrDyn *scr_dyn_get_own_property_descriptors(ScrDyn *object); ScrDyn *scr_dyn_set_prototype(ScrDyn *object, ScrDyn *prototype); @@ -1026,6 +1061,13 @@ bool scr_record_source_has(const void *rec); * that order. Undeclared keys are not copied, as the literal copies none. */ void scr_record_source_spread(void *dst, const void *src, const char *const *names, const size_t *lens, size_t count); +/* Object.freeze of a record a program may freeze: sets SCR_RC_FROZEN in its + * rc word (immortal records are left as they are). */ +void scr_record_freeze(void *record); +/* Node's TypeError for a write to a frozen record: operation 0 assigns the + * existing property `key`, 1 adds it, 2 deletes it. `owner` is the object's + * text in the message ("#"). */ +void scr_record_throw_frozen(ScrStr *key, ScrStr *owner, double operation); /* The commit of a live record view converts the view into a new record * `from`, whose conversion keeps the view's undeclared properties, then * moves its fields into `to`: this moves the kept properties to `to` too, @@ -1036,6 +1078,17 @@ ScrDyn *scr_record_source_apply(ScrDyn *snapshot, const void *rec); /* Attach the prototype member names of a class instance capsule * (idempotent), so `in` sees its methods and accessors. */ void scr_dyn_typed_ref_bind_class(ScrDyn *d, const ScrDynTypedClassOps *ops); +/* Attach the bag and prototype hooks of a capsule of a bag-tracked class + * instance (idempotent). */ +void scr_dyn_typed_ref_bind_bag_class(ScrDyn *d, const ScrDynTypedBagClassOps *ops); +/* The method `key` that a call on a bag-tracked class instance capsule + * reads from its view, +1, when an ordinary data property answers it: an + * own one (a declared field's value from its native slot, or another bag + * property's value), or else one on the view's prototype chain. NULL when + * only the view can answer (the caller builds it), and also NULL, with the + * exception pending, when the prototype hook threw. Never runs user + * code. */ +ScrDyn *scr_dyn_typed_ref_bag_method(const ScrDyn *d, const char *key, size_t key_len); /* The constructor name of a class instance capsule's own class, for * util.inspect; NULL when the capsule is not a bound class instance, its * class is anonymous, or its prototype was replaced. */ @@ -1478,6 +1531,19 @@ ScrDyn *scr_dyn_visit_view(const ScrDyn *capsule); * (scr_http_dyn_install / scr_net_dyn_install — emitted main() calls * them exactly when the unit is linked, the scr_net_install story), so * this always-linked core never references gated units. */ +/* A method a handle class answers through invoke, read as a value + * (`typeof req.json`, `raw[key]()`): the read answers one function per + * (tag, method) that calls invoke on its `this`. length is the method's + * Function.prototype.length; rejects marks a method whose call on another + * receiver answers a rejected promise instead of throwing (the async + * WebIDL operations: Body's json(), ReadableStream's cancel()). */ +typedef struct ScrDynHandleMethod { + const char *name; + uint32_t length; + bool rejects; + const char *invalid_this; /* NULL = the class's (ScrDynHandleOps) */ +} ScrDynHandleMethod; + typedef struct ScrDynHandleOps { const char *cls; /* Node's constructor name — error texts and dynCheck's "got ..." */ void *(*retain)(void *h); @@ -1543,6 +1609,16 @@ typedef struct ScrDynHandleOps { * stream unit builds the handle's Transform from these operations on its * first stream use. NULL = the class is not a stream. */ const struct ScrDynHandleTransform *transform; + /* The class's methods as values, terminated by a NULL name; NULL = a + * property read answers no methods (member calls still reach invoke). + * See ScrDynHandleMethod. */ + const ScrDynHandleMethod *methods; + /* The TypeError (code ERR_INVALID_THIS) a method value called on another + * receiver raises; NULL = WebIDL's "Illegal invocation". */ + const char *invalid_this; + /* A promise rejected with the pending exception (+1), for the rejects + * methods; the owning unit supplies it, since promises are a gated unit. */ + ScrDyn *(*rejected)(void); } ScrDynHandleOps; /* The operations a Transform handle gives the stream unit (scr_stream.c): diff --git a/packages/runtime/src/scr_dyn_builtin_stream.c b/packages/runtime/src/scr_dyn_builtin_stream.c index 926f85eea..2875cfa06 100644 --- a/packages/runtime/src/scr_dyn_builtin_stream.c +++ b/packages/runtime/src/scr_dyn_builtin_stream.c @@ -34,7 +34,7 @@ * receiver without a reference; the hook closures outlive the record. * * Not supported here (each throws the "not supported yet" Error where it - * runs): the options scr_stream.c's dyn option walk refuses, `_writev` and `_construct`, symbol event names, listeners() / + * runs): the options scr_stream.c's dyn option walk refuses, symbol event names, listeners() / * rawListeners(), `wrap()`, and pipes into values that are neither runtime * streams nor constructor-function streams. */ #include "scr_runtime.h" @@ -898,20 +898,15 @@ static ScrDyn *bcs_construct(int kind, ScrDyn *self, ScrDyn *const *args, size_t } bool failed = false; bool writes = kind != BCS_READABLE; - bool transformish = kind == BCS_TRANSFORM || kind == BCS_PASSTHROUGH; ScrDyn *opts = bcs_arg(args, argc, 0); /* Node's constructors assign options.writev and options.construct to * the instance (this._writev = options.writev), then read _writev and * _construct, so an option shadows the prototype's method. */ if (opts->kind == SCR_DYN_OBJ) { - if (writes && !transformish && !bcs_copy_option(self, opts, "writev", "_writev")) return NULL; + if (writes && !bcs_copy_option(self, opts, "writev", "_writev")) return NULL; if (!bcs_copy_option(self, opts, "construct", "_construct")) return NULL; } bool has_writev = writes && bcs_is_function(self, "_writev", &failed); - if (has_writev && transformish) { - bcs_unsupported("_writev on a constructor-function Transform is not supported yet"); - return NULL; - } bool has_construct = !failed && bcs_is_function(self, "_construct", &failed); if (failed) return NULL; ScrStream *s = kind == BCS_READABLE ? scr_stream_new_readable_dyn(opts) diff --git a/packages/runtime/src/scr_dyn_invoke.c b/packages/runtime/src/scr_dyn_invoke.c index 4039fda94..c8093f64c 100644 --- a/packages/runtime/src/scr_dyn_invoke.c +++ b/packages/runtime/src/scr_dyn_invoke.c @@ -740,6 +740,8 @@ static ScrDyn *dyn_live_array_method(ScrDyn *recv, int mid, ScrDyn *const *args, if (!ops) return NULL; ScrArr *arr = recv->v.typed_ref.ptr; size_t len = arr->len; + /* A frozen array's view refuses the write. */ + if (scr_arr_frozen(arr) && ((mid == DYN_M_push) || (mid == DYN_M_pop))) return NULL; if ((mid == DYN_M_push) && argc == 1 && ops->set) { if (!ops->set(arr, (double)len, args[0])) return NULL; *handled = true; @@ -879,6 +881,20 @@ static ScrDyn *scr_dyn_invoke_impl( scr_dyn_release(date_method); return result; } + /* A method of a bag-tracked class instance that its view holds as an + * ordinary data property, own or inherited: the call the view's + * dispatch below makes, without building the view. */ + size_t method_len = strlen(method); + ScrDyn *inherited = scr_dyn_typed_ref_bag_method(recv, method, method_len); + if (scr_exc_pending()) return NULL; + if (inherited && scr_dyn_is_callable(inherited)) { + scr_dyn_this_push_dyn(recv); + ScrDyn *result = scr_dyn_call(inherited, args, argc, what); + scr_dyn_this_pop(); + scr_dyn_release(inherited); + return result; + } + scr_dyn_release(inherited); ScrDyn *materialized = scr_dyn_typed_ref_materialize(recv); if (scr_exc_pending()) { scr_dyn_release(materialized); @@ -1098,16 +1114,20 @@ static ScrDyn *scr_dyn_invoke_impl( return NULL; } } - if (recv->v.arr.frozen && ((mid == DYN_M_push) || - (mid == DYN_M_pop) || (mid == DYN_M_shift) || - (mid == DYN_M_unshift) || (mid == DYN_M_splice))) { - static const char message[] = "Cannot assign to read only property 'length'"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); + /* A frozen array throws at the operation's first write, as in Node. */ + int frozen_op = !recv->v.arr.frozen ? -1 + : (mid == DYN_M_push) ? 1 + : (mid == DYN_M_pop) ? 2 + : (mid == DYN_M_shift) ? 6 + : (mid == DYN_M_unshift) ? 4 + : (mid == DYN_M_splice) ? 10 + : ((mid == DYN_M_sort) || (mid == DYN_M_reverse)) ? 7 + : -1; + double items = (mid == DYN_M_splice) ? (argc > 2 ? (double)(argc - 2) : 0) : (double)argc; + if (frozen_op >= 0 && + scr_array_frozen_error(frozen_op, len, (double)splice_start, frozen_op == 10 ? (double)splice_removed : items, items)) return NULL; - } - if ((grows && recv->non_extensible) || (shrinks && recv->v.arr.sealed && !(mid == DYN_M_shift)) || - (recv->v.arr.frozen && (((mid == DYN_M_sort) && len > 1) || - ((mid == DYN_M_reverse) && len > 1)))) { + if ((grows && recv->non_extensible) || (shrinks && recv->v.arr.sealed && !(mid == DYN_M_shift))) { static const char message[] = "Cannot modify a sealed or frozen array"; scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); return NULL; @@ -1376,11 +1396,7 @@ static ScrDyn *scr_dyn_invoke_impl( if (!dyn_arr_length_unchanged(recv, len)) return NULL; size_t start = dyn_rel_index(startD, len); size_t end = dyn_rel_index(endD, len); - if (recv->v.arr.frozen && start < end) { - static const char message[] = "Cannot assign to read only array index"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); - return NULL; - } + if (recv->v.arr.frozen && scr_array_frozen_error(8, len, (double)start, (double)end, 0)) return NULL; ScrDyn *fill = argc ? args[0] : scr_dyn_undefined(); if (start < end) scr_dyn_region_store(recv, fill); for (size_t i = start; i < end; i++) { @@ -1409,11 +1425,8 @@ static ScrDyn *scr_dyn_invoke_impl( size_t end = dyn_rel_index(endD, len); size_t count = end > start ? end - start : 0; if (count > len - target) count = len - target; - if (recv->v.arr.frozen && count > 0) { - static const char message[] = "Cannot assign to read only array index"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); + if (recv->v.arr.frozen && scr_array_frozen_error(9, len, (double)target, (double)start, (double)end)) return NULL; - } if (start < target && target < start + count) { for (size_t i = count; i > 0; i--) { ScrDyn *old = recv->v.arr.items[target + i - 1]; @@ -1958,7 +1971,7 @@ static bool dyn_global_construct(const ScrDyn *callee, const ScrDyn *args, ScrDy static ScrDyn *dyn_global_ctor(const char *name) { for (size_t i = 0; i < DYN_GLOBAL_CTORS; i++) { - if (strcmp(dyn_global_ctor_names[i], name) != 0) continue; + if (dyn_global_ctor_names[i][0] != name[0] || strcmp(dyn_global_ctor_names[i], name) != 0) continue; if (!dyn_global_ctors_registered) { dyn_global_ctors_registered = true; scr_atexit(dyn_global_ctors_cleanup); diff --git a/packages/runtime/src/scr_event_target.c b/packages/runtime/src/scr_event_target.c index 7c8a465a6..9ca91879a 100644 --- a/packages/runtime/src/scr_event_target.c +++ b/packages/runtime/src/scr_event_target.c @@ -1043,6 +1043,9 @@ static ScrDyn *scr_et_method_value(const char *key, size_t len, bool target) { const char *name = scr_et_method_names[i]; if (strlen(name) != len || memcmp(name, key, len) != 0) continue; if (!scr_et_method_values[i]) { + /* An AbortSignal answers these without the unit ever installing: + * exit releases the cached function either way. */ + scr_et_register_teardown(); ScrClosure *clo = scr_closure_new(NULL, 1); clo->caps[0] = scr_box_new(SCR_BOX_F64); scr_box_set_f64(clo->caps[0], (double)i); @@ -1054,6 +1057,10 @@ static ScrDyn *scr_et_method_value(const char *key, size_t len, bool target) { return NULL; } +ScrDyn *scr_event_target_method_value(const char *key, size_t len) { + return scr_et_method_value(key, len, true); +} + static ScrDyn *scr_event_target_value(const ScrEvent *ev) { return ev->target ? scr_dyn_retain(ev->target) : scr_dyn_new_null(); } diff --git a/packages/runtime/src/scr_event_target.h b/packages/runtime/src/scr_event_target.h index e787cdbdb..e1ff5d0da 100644 --- a/packages/runtime/src/scr_event_target.h +++ b/packages/runtime/src/scr_event_target.h @@ -56,6 +56,11 @@ void scr_et_core_drop(ScrEventTargetCore *core); ScrDyn *scr_et_add_listener(ScrEventTargetCore *core, ScrDyn *const *args, size_t argc); ScrDyn *scr_et_remove_listener(ScrEventTargetCore *core, ScrDyn *const *args, size_t argc); ScrDyn *scr_et_dispatch_event(ScrEventTargetCore *core, ScrDyn *self, ScrDyn *const *args, size_t argc); +/* EventTarget.prototype's method `key` (addEventListener, + * removeEventListener, dispatchEvent) as a function value (+1), the one + * every EventTarget answers, AbortSignal included: it runs on any target + * its `this` is. NULL for another name. */ +ScrDyn *scr_event_target_method_value(const char *key, size_t len); /* Dispatches an event handle (borrowed) without the argument checks. */ void scr_et_dispatch(ScrEventTargetCore *core, ScrDyn *self, ScrDyn *event); /* `on` attributes (Node's defineEventHandler): get answers +1 (null diff --git a/packages/runtime/src/scr_fetch.c b/packages/runtime/src/scr_fetch.c index 7cc2beef4..d28707d5d 100644 --- a/packages/runtime/src/scr_fetch.c +++ b/packages/runtime/src/scr_fetch.c @@ -147,6 +147,9 @@ struct SfStream { bool pull_again; bool discarded; bool request_discarding; + /* A zero high-water mark (a TransformStream's readable side): pull only + * while a read is pending. */ + bool hwm_zero; ScrDyn *error; SfReader *reader; /* owned while locked */ SfCollector *collector; /* owned while a body reader is active */ @@ -167,12 +170,23 @@ struct SfStream { SfTransfer *response_owner; /* weak; transfer owns the stream */ bool callbacks_tracked; SfStream *callbacks_next; + /* Told when the stream errors (a tee watching its source's closed + * promise); ctx is borrowed and the watcher clears it before it goes. */ + void (*error_watch)(void *ctx, ScrDyn *reason); + void *error_watch_ctx; }; struct SfHeaders { size_t rc; ScrArr *pairs; + /* Fetch's "sort and combine" list of `pairs` as flat name/value pairs, + * built on the first iteration step and dropped by every mutation. */ + ScrArr *sorted; bool immutable; + /* `pairs` is in the form sf_add_converted_headers builds: lowercase valid + * names, trimmed valid UTF-8 values, one combined pair per name except + * set-cookie. Headers built from wire lines are not (duplicates stay). */ + bool canonical; }; struct SfResponse { @@ -184,6 +198,23 @@ struct SfResponse { int status; bool redirected; bool null_body; + /* @hono/node-server's lightweight-response state (scr_fetch_response_light): + * whether a constructed Response is still one the package would write + * from its constructor arguments, and in which header form. 0: not (it + * was materialized by a body/metadata read, or never qualified); 1: no + * init headers; 2: a Headers (or pair list) form; 3: a plain record, + * whose raw entries light_record keeps in order. */ + int8_t light; + /* Form 2 reached from form 1 by a headers read: the package's Headers + * start with its default text content type when the body is not null. */ + bool light_default_ct; + /* The constructor added the text content type (a string body without + * one), which the package's lightweight headers never hold. */ + bool auto_ct; + /* The body's byte length (string or Uint8Array), -1 for a null body, + * -2 for a stream. */ + double light_length; + ScrArr *light_record; }; #ifndef SCR_FETCH_SIGNAL_ONLY @@ -713,7 +744,8 @@ static ScrDyn *sf_signal_get(void *ptr, const char *key, size_t len) { : scr_dyn_retain(scr_dyn_undefined()); } if (sf_name(key, len, "onabort")) return scr_et_handler_get(&s->core, "abort"); - return NULL; + /* Inherited from EventTarget: the function every target shares. */ + return scr_event_target_method_value(key, len); } static bool sf_signal_set(void *ptr, const char *key, size_t len, @@ -880,6 +912,7 @@ static void sf_stream_release(SfStream *s) { sf_response_discard_if_unobserved(s); return; } + scr_weak_dispose_inline(s); sf_stream_drop_chunks(s); if (s->reader) { s->reader->stream = NULL; @@ -1273,6 +1306,7 @@ static void sf_stream_error(SfStream *s, ScrDyn *reason) { if (s->reader) { sf_reject_promise_reason(s->reader->closed, reason); } + if (s->error_watch) s->error_watch(s->error_watch_ctx, reason); if (s->request_owner) { sf_transfer_stream_error(s->request_owner, reason); } @@ -1600,6 +1634,14 @@ static void sf_stream_schedule_pull_complete(SfStream *s) { scr_queue_microtask(cb); } +/* A consumer waiting for the stream's next chunk: a pending read, a body + * collector, or a fetch upload (fetch keeps reading the request body, + * including after an aborted transfer stops writing it). */ +static bool sf_stream_has_consumer(const SfStream *s) { + return s->collector || (s->reader && s->reader->pending_head) || s->request_owner || + s->request_discarding; +} + static void sf_stream_pull(SfStream *s) { if (!s->started || (!s->pull_cb && !sf_stream_has_from_source(s)) || @@ -1607,6 +1649,10 @@ static void sf_stream_pull(SfStream *s) { s->error) { return; } + /* ReadableStreamDefaultControllerShouldCallPull: with no room in the + * queue, only a consumer asks for a chunk: a pending read, a body + * collector, or a fetch upload, whose reader is always waiting. */ + if (s->hwm_zero && !sf_stream_has_consumer(s)) return; if (s->pulling) { s->pull_again = true; return; @@ -1834,9 +1880,241 @@ static ScrPromise *sf_stream_collect(SfStream *s, int mode) { c->mode = mode; s->collector = c; sf_stream_drain(s); + if (s->hwm_zero && !s->head) sf_stream_pull(s); return p; } +/* ── ReadableStreamDefaultTee ────────────────────────────────────── */ + +/* + * The streams spec's ReadableStreamDefaultTee over a native stream: the + * tee locks the source with a reader and feeds two native branches whose + * pull and cancel algorithms are native functions. A pull on either branch + * reads one chunk from the source and enqueues it into every branch that + * was not canceled; canceling both branches cancels the source with both + * reasons. Response.clone and Request.clone use it (undici's cloneBody). + * + * Ownership: the tee owns the source, its reader and both branches; each + * branch's callbacks and every pending read own the tee. Closing, erroring + * or canceling a branch drops its callbacks (sf_stream_drop_source_callbacks), + * and fetch teardown drops the callbacks of branches nobody finished. + */ +typedef struct SfTee { + size_t rc; + SfStream *source; + SfReader *reader; + SfStream *branch[2]; + bool reading; + bool read_again; + bool canceled[2]; + ScrDyn *reason[2]; + ScrPromise *cancel_promise; +} SfTee; + +static SfTee *sf_tee_retain(SfTee *t) { + t->rc++; + return t; +} + +static void sf_tee_release(SfTee *t) { + if (!t || --t->rc > 0) return; + if (t->source->error_watch_ctx == t) { + t->source->error_watch = NULL; + t->source->error_watch_ctx = NULL; + } + sf_stream_release(t->branch[0]); + sf_stream_release(t->branch[1]); + sf_reader_release(t->reader); + sf_stream_release(t->source); + scr_dyn_release(t->reason[0]); + scr_dyn_release(t->reason[1]); + scr_promise_release(t->cancel_promise); + free(t); +} + +static void *sf_tee_retain_v(void *p) { return sf_tee_retain((SfTee *)p); } +static void sf_tee_release_v(void *p) { sf_tee_release((SfTee *)p); } + +static ScrBox *sf_tee_box(SfTee *t) { + ScrBox *box = scr_box_new_obj(&sf_tee_retain_v, &sf_tee_release_v, NULL); + scr_box_set_ref(box, sf_tee_retain(t)); + return box; +} + +static void sf_tee_pull(SfTee *t); + +/* The read request's chunk steps run in a microtask (spec: "queue a + * microtask"), after the read that produced the chunk settles. */ +static void sf_tee_chunk_task(ScrClosure *cb) { + SfTee *t = (SfTee *)scr_box_get_ref(cb->caps[0]); + ScrDyn *chunk = (ScrDyn *)scr_box_get_ref(cb->caps[1]); + t->read_again = false; + for (int i = 0; i < 2; i++) { + if (t->canceled[i]) continue; + SfStream *b = t->branch[i]; + if (b->close_requested || b->closed || b->error) continue; + sf_stream_enqueue_value(b, chunk); + if (scr_exc_pending()) scr_exc_clear(); + } + t->reading = false; + if (t->read_again) sf_tee_pull(t); + scr_dyn_release(chunk); + sf_tee_release(t); +} + +static void sf_tee_sink_chunk(void *ctx, ScrDyn *value) { + SfTee *t = ctx; + ScrClosure *cb = scr_closure_new((void *)&sf_tee_chunk_task, 2); + cb->caps[0] = sf_tee_box(t); + cb->caps[1] = scr_box_new_obj(&scr_dyn_retain_v, &scr_dyn_release_v, NULL); + scr_box_set_ref(cb->caps[1], scr_dyn_retain(value)); + scr_queue_microtask(cb); + sf_tee_release(t); /* the read request's reference */ +} + +static void sf_tee_sink_close(void *ctx) { + SfTee *t = ctx; + t->reading = false; + for (int i = 0; i < 2; i++) { + if (!t->canceled[i]) sf_stream_close(t->branch[i]); + } + if (!t->canceled[0] || !t->canceled[1]) { + scr_promise_fulfill_void(t->cancel_promise); + } + sf_tee_release(t); +} + +/* The source reader's closed promise rejected (spec: upon its rejection + * with r, error both branches, and resolve cancelPromise unless both + * branches were canceled), as a promise reaction: in a microtask. */ +static void sf_tee_source_errored_task(ScrClosure *cb) { + SfTee *t = (SfTee *)scr_box_get_ref(cb->caps[0]); + ScrDyn *reason = (ScrDyn *)scr_box_get_ref(cb->caps[1]); + sf_stream_error(t->branch[0], reason); + sf_stream_error(t->branch[1], reason); + if (!t->canceled[0] || !t->canceled[1]) scr_promise_fulfill_void(t->cancel_promise); + scr_dyn_release(reason); + sf_tee_release(t); +} + +/* The source errored, with or without a read of it pending. */ +static void sf_tee_source_errored(void *ctx, ScrDyn *reason) { + SfTee *t = ctx; + t->source->error_watch = NULL; + t->source->error_watch_ctx = NULL; + ScrClosure *cb = scr_closure_new((void *)&sf_tee_source_errored_task, 2); + cb->caps[0] = sf_tee_box(t); + cb->caps[1] = scr_box_new_obj(&scr_dyn_retain_v, &scr_dyn_release_v, NULL); + scr_box_set_ref(cb->caps[1], scr_dyn_retain(reason)); + scr_queue_microtask(cb); +} + +/* A read request's error steps only end the read: the source's error + * reaches the branches through sf_tee_source_errored. */ +static void sf_tee_sink_error(void *ctx, ScrDyn *reason) { + (void)reason; + SfTee *t = ctx; + t->reading = false; + sf_tee_release(t); +} + +static const ScrFetchReadSink sf_tee_sink = { + &sf_tee_sink_chunk, &sf_tee_sink_close, &sf_tee_sink_error}; + +static void sf_tee_pull(SfTee *t) { + if (t->reading) { + t->read_again = true; + return; + } + t->reading = true; + sf_tee_retain(t); /* owned by the read request */ + if (!sf_reader_read_request(t->reader, NULL, &sf_tee_sink, t)) { + scr_exc_clear(); + t->reading = false; + sf_tee_release(t); + } +} + +static ScrDyn *sf_tee_pull_thunk(ScrClosure *cb, ScrDyn *const *args, + size_t argc) { + (void)args; + (void)argc; + SfTee *t = (SfTee *)scr_box_get_ref(cb->caps[0]); + sf_tee_pull(t); + sf_tee_release(t); + return scr_dyn_retain(scr_dyn_undefined()); +} + +static ScrDyn *sf_tee_cancel(SfTee *t, int i, ScrDyn *reason) { + t->canceled[i] = true; + scr_dyn_release(t->reason[i]); + t->reason[i] = scr_dyn_retain(reason); + if (t->canceled[0] && t->canceled[1]) { + ScrDyn *composite = scr_dyn_new_arr(); + scr_dyn_arr_push(composite, scr_dyn_retain(t->reason[0])); + scr_dyn_arr_push(composite, scr_dyn_retain(t->reason[1])); + ScrPromise *result = sf_stream_cancel(t->source, composite, true); + scr_dyn_release(composite); + ScrDyn *boxed = scr_dyn_new_promise(result); + scr_promise_release(result); + sf_then_adopt(t->cancel_promise, boxed); + scr_dyn_release(boxed); + } + return scr_dyn_new_promise(t->cancel_promise); +} + +static ScrDyn *sf_tee_cancel_thunk_0(ScrClosure *cb, ScrDyn *const *args, + size_t argc) { + SfTee *t = (SfTee *)scr_box_get_ref(cb->caps[0]); + ScrDyn *out = sf_tee_cancel(t, 0, argc ? args[0] : scr_dyn_undefined()); + sf_tee_release(t); + return out; +} + +static ScrDyn *sf_tee_cancel_thunk_1(ScrClosure *cb, ScrDyn *const *args, + size_t argc) { + SfTee *t = (SfTee *)scr_box_get_ref(cb->caps[0]); + ScrDyn *out = sf_tee_cancel(t, 1, argc ? args[0] : scr_dyn_undefined()); + sf_tee_release(t); + return out; +} + +static ScrDyn *sf_tee_func(SfTee *t, ScrDynThunk thunk, const char *name) { + ScrClosure *cb = scr_closure_new((void *)thunk, 1); + cb->caps[0] = sf_tee_box(t); + return scr_dyn_new_func(cb, thunk, 1, "(controller)", name); +} + +/* Tees an unlocked, undisturbed stream: +1 branches in out[0] and out[1]. + * The source stays locked by the tee. */ +static void sf_stream_tee(SfStream *s, SfStream *out[2]) { + SfTee *t = calloc(1, sizeof *t); + if (!t) sf_oom(); + t->rc = 1; + t->source = sf_stream_retain(s); + t->reader = sf_stream_get_reader(s); + s->error_watch = &sf_tee_source_errored; + s->error_watch_ctx = t; + t->cancel_promise = scr_promise_new(); + scr_promise_mark_handled(t->cancel_promise); + for (int i = 0; i < 2; i++) { + SfStream *b = sf_stream_new_native(); + b->pull_cb = sf_tee_func(t, &sf_tee_pull_thunk, "pull"); + b->cancel_cb = sf_tee_func( + t, i == 0 ? &sf_tee_cancel_thunk_0 : &sf_tee_cancel_thunk_1, "cancel"); + b->source_this = scr_dyn_retain(scr_dyn_undefined()); + sf_stream_track_callbacks(b); + t->branch[i] = sf_stream_retain(b); + out[i] = b; + } + /* An errored source's closed promise has already rejected. */ + if (s->error) sf_tee_source_errored(t, s->error); + /* The branches start at once; a branch with room then pulls. */ + sf_stream_schedule_initial_pull(out[0]); + sf_stream_schedule_initial_pull(out[1]); + sf_tee_release(t); +} + ScrDyn *scr_fetch_stream_new(ScrDyn *source) { ScrDyn *source_view = source; ScrDyn *materialized = NULL; @@ -2146,6 +2424,24 @@ static ScrDyn *sf_controller_get(void *ptr, const char *key, size_t len) { static bool sf_header_name_ok(const char *s, size_t len); +/* A borrowed element of a runtime-built header pair array. These arrays are + * only ever appended to or rewritten whole, so they are dense. */ +static inline ScrStr *sf_pair_at(ScrArr *pairs, size_t i) { + if (i < pairs->filled) return (ScrStr *)(uintptr_t)scr_arr_dense_get(pairs, i); + return (ScrStr *)scr_arr_peek_ref(pairs, (double)i); +} + +static ScrStr *sf_header_name_lower(const char *data, size_t len); + +/* `name` in lowercase, +1: the same string when it has no uppercase byte. */ +static ScrStr *sf_header_name_lower_str(ScrStr *name) { + for (size_t i = 0; i < name->len; i++) { + char c = name->data[i]; + if (c >= 'A' && c <= 'Z') return sf_header_name_lower(name->data, name->len); + } + return scr_str_retain(name); +} + static ScrStr *sf_header_name_lower(const char *data, size_t len) { char *lower = malloc(len ? len : 1); if (!lower) sf_oom(); @@ -2214,13 +2510,8 @@ static SfHeaders *sf_headers_new_response(ScrArr *raw_pairs) { size_t n = (size_t)scr_arr_len(raw_pairs); ScrArr *pairs = scr_arr_new(SCR_ELEM_STR, n); for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *raw_name = scr_arr_get_ref(raw_pairs, (double)i); - scr_arr_push_ref( - pairs, sf_header_name_lower(raw_name->data, raw_name->len)); - scr_str_release(raw_name); - ScrStr *raw = scr_arr_get_ref(raw_pairs, (double)(i + 1)); - scr_arr_push_ref(pairs, sf_latin1_to_utf8(raw)); - scr_str_release(raw); + scr_arr_push_ref(pairs, sf_header_name_lower_str(sf_pair_at(raw_pairs, i))); + scr_arr_push_ref(pairs, sf_latin1_to_utf8(sf_pair_at(raw_pairs, i + 1))); } scr_arr_release(raw_pairs); return sf_headers_new(pairs, true); @@ -2234,9 +2525,17 @@ static SfHeaders *sf_headers_retain(SfHeaders *h) { static void sf_headers_release(SfHeaders *h) { if (!h || --h->rc > 0) return; scr_arr_release(h->pairs); + if (h->sorted) scr_arr_release(h->sorted); free(h); } +/* Every change to h->pairs goes through here first. */ +static void sf_headers_changed(SfHeaders *h) { + if (!h->sorted) return; + scr_arr_release(h->sorted); + h->sorted = NULL; +} + static void *sf_headers_retain_v(void *p) { return sf_headers_retain((SfHeaders *)p); } @@ -2276,7 +2575,7 @@ static ScrStr *sf_headers_name(const ScrDyn *value, const char *method) { scr_str_release(raw); return NULL; } - ScrStr *out = sf_header_name_lower(raw->data, raw->len); + ScrStr *out = sf_header_name_lower_str(raw); scr_str_release(raw); return out; } @@ -2287,40 +2586,36 @@ static bool sf_headers_key_eq(const ScrStr *left, const ScrStr *right) { } static ScrStr *sf_headers_get_value(SfHeaders *h, const ScrStr *name) { - size_t n = (size_t)scr_arr_len(h->pairs); + ScrArr *pairs = h->pairs; + size_t n = (size_t)scr_arr_len(pairs); size_t count = 0; size_t total = 0; + ScrStr *first = NULL; for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = scr_arr_get_ref(h->pairs, (double)i); - ScrStr *value = scr_arr_get_ref(h->pairs, (double)(i + 1)); - if (sf_headers_key_eq(key, name)) { - if (count > 0) total += 2; - total += value->len; - count++; - } - scr_str_release(key); - scr_str_release(value); + if (!sf_headers_key_eq(sf_pair_at(pairs, i), name)) continue; + ScrStr *value = sf_pair_at(pairs, i + 1); + if (count > 0) total += 2; + else first = value; + total += value->len; + count++; } if (count == 0) return NULL; + if (count == 1) return scr_str_retain(first); char *joined = malloc(total > 0 ? total : 1); if (!joined) sf_oom(); size_t at = 0; size_t emitted = 0; for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = scr_arr_get_ref(h->pairs, (double)i); - ScrStr *value = scr_arr_get_ref(h->pairs, (double)(i + 1)); - if (sf_headers_key_eq(key, name)) { - if (emitted > 0) { - joined[at++] = ','; - joined[at++] = ' '; - } - memcpy(joined + at, value->data, value->len); - at += value->len; - emitted++; + if (!sf_headers_key_eq(sf_pair_at(pairs, i), name)) continue; + ScrStr *value = sf_pair_at(pairs, i + 1); + if (emitted > 0) { + joined[at++] = ','; + joined[at++] = ' '; } - scr_str_release(key); - scr_str_release(value); + memcpy(joined + at, value->data, value->len); + at += value->len; + emitted++; } ScrStr *out = scr_str_new(joined, total); free(joined); @@ -2330,17 +2625,17 @@ static ScrStr *sf_headers_get_value(SfHeaders *h, const ScrStr *name) { static void sf_headers_delete_name(SfHeaders *h, const ScrStr *name) { ScrArr *old = h->pairs; size_t n = (size_t)scr_arr_len(old); + bool present = false; + for (size_t i = 0; i + 1 < n && !present; i += 2) + present = sf_headers_key_eq(sf_pair_at(old, i), name); + if (!present) return; + sf_headers_changed(h); ScrArr *next = scr_arr_new(SCR_ELEM_STR, n); for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = scr_arr_get_ref(old, (double)i); - ScrStr *value = scr_arr_get_ref(old, (double)(i + 1)); - if (sf_headers_key_eq(key, name)) { - scr_str_release(key); - scr_str_release(value); - continue; - } - scr_arr_push_ref(next, key); - scr_arr_push_ref(next, value); + ScrStr *key = sf_pair_at(old, i); + if (sf_headers_key_eq(key, name)) continue; + scr_arr_push_ref(next, scr_str_retain(key)); + scr_arr_push_ref(next, scr_str_retain(sf_pair_at(old, i + 1))); } h->pairs = next; scr_arr_release(old); @@ -2381,15 +2676,13 @@ static bool sf_headers_convert_pair( * Fetch-specific semicolon separator for Cookie. */ static void sf_headers_append_pair( SfHeaders *h, ScrStr *name, ScrStr *value) { + sf_headers_changed(h); if (!(name->len == 10 && memcmp(name->data, "set-cookie", 10) == 0)) { size_t n = (size_t)scr_arr_len(h->pairs); for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = scr_arr_get_ref(h->pairs, (double)i); - bool match = sf_headers_key_eq(key, name); - scr_str_release(key); - if (!match) continue; + if (!sf_headers_key_eq(sf_pair_at(h->pairs, i), name)) continue; - ScrStr *previous = scr_arr_get_ref(h->pairs, (double)(i + 1)); + ScrStr *previous = sf_pair_at(h->pairs, i + 1); const char *separator = name->len == 6 && memcmp(name->data, "cookie", 6) == 0 ? "; " @@ -2402,7 +2695,10 @@ static void sf_headers_append_pair( memcpy(joined_data + previous->len + 2, value->data, value->len); ScrStr *joined = scr_str_new(joined_data, joined_len); free(joined_data); - scr_str_release(previous); + /* Appending an empty value leaves the separator's space at the end of + * the combined value, which a copy trims again: the pairs are no + * longer in their canonical, trimmed form. */ + if (value->len == 0) h->canonical = false; scr_str_release(name); scr_str_release(value); scr_arr_set_ref(h->pairs, (double)(i + 1), joined); @@ -2429,7 +2725,7 @@ static ScrStr **sf_headers_sorted_names(SfHeaders *h, size_t *count_out) { if (capacity && !names) sf_oom(); size_t count = 0; for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = scr_arr_get_ref(h->pairs, (double)i); + ScrStr *key = sf_pair_at(h->pairs, i); bool seen = false; for (size_t j = 0; j < count; j++) { if (sf_headers_key_eq(names[j], key)) { @@ -2437,8 +2733,7 @@ static ScrStr **sf_headers_sorted_names(SfHeaders *h, size_t *count_out) { break; } } - if (seen) scr_str_release(key); - else names[count++] = key; + if (!seen) names[count++] = key; } if (count > 1) { qsort(names, count, sizeof *names, &sf_headers_name_cmp); @@ -2451,44 +2746,44 @@ static bool sf_headers_is_set_cookie(const ScrStr *name) { return name->len == 10 && memcmp(name->data, "set-cookie", 10) == 0; } -/* The `index`th entry of Fetch's "sort and combine" list: names in - * byte order, each with its combined value, except that every set-cookie - * value is its own entry (in insertion order), as Node's Headers - * iterator yields them. False past the end. Outputs are +1. */ +/* Fetch's "sort and combine" list as flat name/value pairs: names in byte + * order, each with its combined value, except that every set-cookie value + * is its own entry (in insertion order), as Node's Headers iterator yields + * them. Cached until the next mutation; borrowed. */ +static ScrArr *sf_headers_sorted(SfHeaders *h) { + if (h->sorted) return h->sorted; + size_t count = 0; + ScrStr **names = sf_headers_sorted_names(h, &count); /* borrowed names */ + size_t n = (size_t)scr_arr_len(h->pairs); + ScrArr *sorted = scr_arr_new(SCR_ELEM_STR, n); + for (size_t i = 0; i < count; i++) { + if (!sf_headers_is_set_cookie(names[i])) { + scr_arr_push_ref(sorted, scr_str_retain(names[i])); + scr_arr_push_ref(sorted, sf_headers_get_value(h, names[i])); + continue; + } + for (size_t j = 0; j + 1 < n; j += 2) { + if (!sf_headers_is_set_cookie(sf_pair_at(h->pairs, j))) continue; + scr_arr_push_ref(sorted, scr_str_retain(names[i])); + scr_arr_push_ref(sorted, scr_str_retain(sf_pair_at(h->pairs, j + 1))); + } + } + free(names); + h->sorted = sorted; + return sorted; +} + +/* The `index`th entry of the sort-and-combine list (sf_headers_sorted). + * False past the end. Outputs are +1. */ static bool sf_headers_sorted_entry(SfHeaders *h, size_t index, ScrStr **name_out, ScrStr **value_out) { - size_t count = 0; - ScrStr **names = sf_headers_sorted_names(h, &count); - bool found = false; - size_t at = 0; - for (size_t i = 0; i < count && !found; i++) { - if (!sf_headers_is_set_cookie(names[i])) { - if (at == index) { - *name_out = scr_str_retain(names[i]); - *value_out = sf_headers_get_value(h, names[i]); - found = true; - } - at++; - continue; - } - size_t n = (size_t)scr_arr_len(h->pairs); - for (size_t j = 0; j + 1 < n && !found; j += 2) { - ScrStr *key = scr_arr_get_ref(h->pairs, (double)j); - bool match = sf_headers_is_set_cookie(key); - scr_str_release(key); - if (!match) continue; - if (at == index) { - *name_out = scr_str_retain(names[i]); - *value_out = scr_arr_get_ref(h->pairs, (double)(j + 1)); - found = true; - } - at++; - } - } - for (size_t j = 0; j < count; j++) scr_str_release(names[j]); - free(names); - return found; + ScrArr *sorted = sf_headers_sorted(h); + size_t entries = (size_t)scr_arr_len(sorted) / 2; + if (index >= entries) return false; + *name_out = scr_str_retain(sf_pair_at(sorted, 2 * index)); + *value_out = scr_str_retain(sf_pair_at(sorted, 2 * index + 1)); + return true; } /* One step of a live Headers iterator (entries, keys or values): the @@ -2552,14 +2847,8 @@ static ScrDyn *sf_headers_invoke(void *ptr, ScrDyn *self, ScrDyn *out = scr_dyn_new_arr(); size_t n = (size_t)scr_arr_len(h->pairs); for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = scr_arr_get_ref(h->pairs, (double)i); - ScrStr *value = scr_arr_get_ref(h->pairs, (double)(i + 1)); - if (key->len == 10 && - memcmp(key->data, "set-cookie", 10) == 0) { - scr_dyn_arr_push(out, scr_dyn_new_str(value)); - } - scr_str_release(key); - scr_str_release(value); + if (sf_headers_is_set_cookie(sf_pair_at(h->pairs, i))) + scr_dyn_arr_push(out, scr_dyn_new_str(sf_pair_at(h->pairs, i + 1))); } return out; } @@ -2646,6 +2935,940 @@ static ScrDyn *sf_headers_get(void *ptr, const char *key, size_t len) { return NULL; } +/* ── TransformStream / WritableStream ────────────────────────────── */ + +/* + * The identity TransformStream (`new TransformStream()`): a native + * readable side (an SfStream with a zero high-water mark, fed by the + * transform) and a writable side implemented here with the streams spec's + * WritableStream state machine (default high-water mark 1, chunk size 1). + * The writable side has no other sink, so the TransformStream object, its + * WritableStream and the sink share one SfTransform; the handles differ + * only by tag. Promise reactions the spec chains on already-settled + * promises run after the same number of microtask turns as in Node's + * implementation (lib/internal/webstreams), so interleavings with user + * code match. + * + * Ownership: the transform owns its readable side and its writer; the + * readable side's pull/cancel callbacks own the transform until that side + * closes or errors (fetch teardown drops the callbacks of transforms + * nobody finished). A writer handle retains the transform while bound, + * like a reader handle retains its stream. + */ +enum { SFW_WRITABLE, SFW_ERRORING, SFW_ERRORED, SFW_CLOSED }; + +typedef struct SfTransform SfTransform; + +typedef struct SfWriteReq { + ScrDyn *chunk; + ScrPromise *promise; + struct SfWriteReq *next; +} SfWriteReq; + +typedef struct SfWriter { + size_t rc; + size_t handle_refs; + SfTransform *stream; /* weak; handle refs retain it while bound */ + ScrPromise *ready; + ScrPromise *closed; +} SfWriter; + +struct SfTransform { + size_t rc; + SfStream *readable; + /* WritableStream */ + bool started; /* the start algorithm's promise settled (one turn) */ + int state; + ScrDyn *stored_error; + SfWriter *writer; /* owned while locked */ + SfWriteReq *head; /* queued writes (chunk + request promise) */ + SfWriteReq *tail; + SfWriteReq *in_flight; + size_t queue_size; /* queueTotalSize: queued chunks + the in-flight one */ + bool close_queued; /* the close sentinel is in the queue */ + ScrPromise *close_request; + ScrPromise *in_flight_close; + bool has_pending_abort; + ScrPromise *pending_abort; + /* The sink abort found a finish promise from a cancel or close, which + * settles the abort instead (Node returns that promise). */ + bool abort_joins_finish; + /* The abort request's outcome between SFT_ABORT_DONE and + * SFT_ABORT_SETTLE: a rejection reason (+1), or NULL to fulfill. */ + ScrDyn *abort_settle_error; + ScrDyn *pending_abort_reason; + bool pending_abort_was_erroring; + bool backpressure; + /* TransformStream */ + bool t_backpressure; + ScrPromise *bp_change; + bool write_waiting; /* the in-flight write waits for bp_change */ + ScrDyn *write_error; /* the in-flight write's failure, if any */ + ScrDyn *cancel_reason; /* the readable side's cancel reason, until it reacts */ + ScrPromise *finish; /* finishPromise of close/abort/cancel */ +}; + +static SfTransform *sf_transform_retain(SfTransform *t) { + t->rc++; + return t; +} + +static void sf_write_req_free(SfWriteReq *w) { + if (!w) return; + scr_dyn_release(w->chunk); + scr_promise_release(w->promise); + free(w); +} + +static void sf_writer_release(SfWriter *w) { + if (!w || --w->rc > 0) return; + scr_promise_release(w->ready); + scr_promise_release(w->closed); + free(w); +} + +static void sf_transform_release(SfTransform *t) { + if (!t || --t->rc > 0) return; + sf_stream_release(t->readable); + scr_dyn_release(t->stored_error); + if (t->writer) { + t->writer->stream = NULL; + sf_writer_release(t->writer); + } + while (t->head) { + SfWriteReq *w = t->head; + t->head = w->next; + sf_write_req_free(w); + } + sf_write_req_free(t->in_flight); + scr_promise_release(t->close_request); + scr_promise_release(t->in_flight_close); + scr_promise_release(t->pending_abort); + scr_dyn_release(t->pending_abort_reason); + scr_dyn_release(t->abort_settle_error); + scr_promise_release(t->bp_change); + scr_promise_release(t->finish); + scr_dyn_release(t->write_error); + scr_dyn_release(t->cancel_reason); + free(t); +} + +static void *sf_transform_retain_v(void *p) { + return sf_transform_retain((SfTransform *)p); +} +static void sf_transform_release_v(void *p) { + sf_transform_release((SfTransform *)p); +} + +static void *sf_writer_handle_retain_v(void *p) { + SfWriter *w = p; + w->rc++; + w->handle_refs++; + if (w->stream) sf_transform_retain(w->stream); + return w; +} + +static void sf_writer_handle_release_v(void *p) { + SfWriter *w = p; + SfTransform *t = w->stream; + if (w->handle_refs > 0) w->handle_refs--; + sf_writer_release(w); + sf_transform_release(t); +} + +static ScrBox *sf_transform_box(SfTransform *t) { + ScrBox *box = scr_box_new_obj(&sf_transform_retain_v, + &sf_transform_release_v, NULL); + scr_box_set_ref(box, sf_transform_retain(t)); + return box; +} + +static void sf_invalid_state(const char *message) { + char buf[96]; + int n = snprintf(buf, sizeof buf, "Invalid state: %s", message); + scr_throw_error_msg_code(SCR_ERR_TYPE, buf, (size_t)n, "ERR_INVALID_STATE"); +} + +static ScrPromise *sf_promise_invalid_state(const char *message) { + ScrPromise *p = scr_promise_new(); + sf_invalid_state(message); + scr_promise_reject_pending(p); + return p; +} + +static ScrPromise *sf_promise_rejected_with(ScrDyn *reason) { + ScrPromise *p = scr_promise_new(); + sf_reject_promise_reason(p, reason); + return p; +} + +static ScrPromise *sf_promise_resolved(void) { + ScrPromise *p = scr_promise_new(); + scr_promise_fulfill_void(p); + return p; +} + +/* Deferred steps: the reaction `op` after `ticks` microtask turns. */ +enum { + SFT_WRITE_DONE, + SFT_WRITE_CONTINUE, + SFT_CLOSE_DONE, + SFT_ABORT_DONE, + SFT_START, + SFT_CANCEL_DONE, + SFT_ABORT_SETTLE, +}; + +static void sf_transform_step(SfTransform *t, int op); + +typedef struct SfTransformTask { + size_t rc; + SfTransform *t; + int op; + int ticks; +} SfTransformTask; + +static void *sf_transform_task_retain_v(void *p) { + ((SfTransformTask *)p)->rc++; + return p; +} + +static void sf_transform_task_release_v(void *p) { + SfTransformTask *task = p; + if (--task->rc > 0) return; + sf_transform_release(task->t); + free(task); +} + +static void sf_transform_tick_task(ScrClosure *cb) { + SfTransformTask *task = (SfTransformTask *)scr_box_get_ref(cb->caps[0]); + if (--task->ticks > 0) { + ScrClosure *next = scr_closure_new((void *)&sf_transform_tick_task, 1); + next->caps[0] = scr_box_new_obj(&sf_transform_task_retain_v, + &sf_transform_task_release_v, NULL); + scr_box_set_ref(next->caps[0], sf_transform_task_retain_v(task)); + scr_queue_microtask(next); + } else { + sf_transform_step(task->t, task->op); + } + sf_transform_task_release_v(task); +} + +static void sf_transform_defer(SfTransform *t, int op, int ticks) { + SfTransformTask *task = calloc(1, sizeof *task); + if (!task) sf_oom(); + task->rc = 1; + task->t = sf_transform_retain(t); + task->op = op; + task->ticks = ticks; + ScrClosure *cb = scr_closure_new((void *)&sf_transform_tick_task, 1); + cb->caps[0] = scr_box_new_obj(&sf_transform_task_retain_v, + &sf_transform_task_release_v, NULL); + scr_box_set_ref(cb->caps[0], task); /* the box takes the reference */ + scr_queue_microtask(cb); +} + +/* ── WritableStream abstract operations ── */ + +static bool sfw_close_queued_or_in_flight(const SfTransform *t) { + return t->close_request || t->in_flight_close; +} + +static bool sfw_has_operation_in_flight(const SfTransform *t) { + return t->in_flight || t->in_flight_close; +} + +static void sfw_writer_ensure_ready_rejected(SfWriter *w, ScrDyn *reason) { + if (!scr_promise_is_pending(w->ready)) { + scr_promise_release(w->ready); + w->ready = scr_promise_new(); + } + sf_reject_promise_reason(w->ready, reason); + scr_promise_mark_handled(w->ready); +} + +static void sfw_writer_ensure_closed_rejected(SfWriter *w, ScrDyn *reason) { + if (!scr_promise_is_pending(w->closed)) { + scr_promise_release(w->closed); + w->closed = scr_promise_new(); + } + sf_reject_promise_reason(w->closed, reason); + scr_promise_mark_handled(w->closed); +} + +static void sfw_update_backpressure(SfTransform *t, bool backpressure) { + SfWriter *w = t->writer; + if (w && backpressure != t->backpressure) { + if (backpressure) { + scr_promise_release(w->ready); + w->ready = scr_promise_new(); + scr_promise_mark_handled(w->ready); + } else { + scr_promise_fulfill_void(w->ready); + } + } + t->backpressure = backpressure; +} + +static void sfw_reject_close_and_closed(SfTransform *t) { + if (t->close_request) { + sf_reject_promise_reason(t->close_request, t->stored_error); + scr_promise_release(t->close_request); + t->close_request = NULL; + } + if (t->writer) { + sf_reject_promise_reason(t->writer->closed, t->stored_error); + scr_promise_mark_handled(t->writer->closed); + } +} + +static void sft_abort_algorithm(SfTransform *t, ScrDyn *reason); + +static void sfw_finish_erroring(SfTransform *t) { + t->state = SFW_ERRORED; + /* The controller's queue resets; every queued write rejects. */ + while (t->head) { + SfWriteReq *w = t->head; + t->head = w->next; + sf_reject_promise_reason(w->promise, t->stored_error); + sf_write_req_free(w); + } + t->tail = NULL; + t->queue_size = 0; + t->close_queued = false; + if (!t->has_pending_abort) { + sfw_reject_close_and_closed(t); + return; + } + if (t->pending_abort_was_erroring) { + t->has_pending_abort = false; + sf_reject_promise_reason(t->pending_abort, t->stored_error); + scr_promise_release(t->pending_abort); + t->pending_abort = NULL; + scr_dyn_release(t->pending_abort_reason); + t->pending_abort_reason = NULL; + sfw_reject_close_and_closed(t); + return; + } + /* The sink's abort runs; its promise settles the abort request. */ + sft_abort_algorithm(t, t->pending_abort_reason); +} + +static void sfw_start_erroring(SfTransform *t, ScrDyn *reason) { + t->state = SFW_ERRORING; + scr_dyn_release(t->stored_error); + t->stored_error = scr_dyn_retain(reason); + if (t->writer) sfw_writer_ensure_ready_rejected(t->writer, reason); + if (!sfw_has_operation_in_flight(t) && t->started) sfw_finish_erroring(t); +} + +static void sfw_deal_with_rejection(SfTransform *t, ScrDyn *error) { + if (t->state == SFW_WRITABLE) { + sfw_start_erroring(t, error); + return; + } + sfw_finish_erroring(t); +} + +static void sfw_error_if_needed(SfTransform *t, ScrDyn *error) { + if (t->state == SFW_WRITABLE) sfw_start_erroring(t, error); +} + +static void sft_set_backpressure(SfTransform *t, bool backpressure); +static void sft_sink_write(SfTransform *t); +static void sft_sink_close(SfTransform *t); + +static void sfw_advance_queue(SfTransform *t) { + if (!t->started || t->in_flight || t->in_flight_close) return; + if (t->state == SFW_ERRORING) { + sfw_finish_erroring(t); + return; + } + if (t->head) { + SfWriteReq *w = t->head; + t->head = w->next; + if (!t->head) t->tail = NULL; + w->next = NULL; + t->in_flight = w; + sft_sink_write(t); + return; + } + if (t->close_queued) { + t->close_queued = false; + t->in_flight_close = t->close_request; + t->close_request = NULL; + sft_sink_close(t); + } +} + +static void sfw_finish_write(SfTransform *t, ScrDyn *error) { + SfWriteReq *w = t->in_flight; + t->in_flight = NULL; + if (error) { + sf_reject_promise_reason(w->promise, error); + sf_write_req_free(w); + sfw_deal_with_rejection(t, error); + return; + } + scr_promise_fulfill_void(w->promise); + sf_write_req_free(w); + if (t->queue_size) t->queue_size--; + if (!sfw_close_queued_or_in_flight(t) && t->state == SFW_WRITABLE) { + sfw_update_backpressure(t, t->queue_size >= 1); + } + sfw_advance_queue(t); +} + +static void sfw_finish_close(SfTransform *t, ScrDyn *error) { + ScrPromise *p = t->in_flight_close; + t->in_flight_close = NULL; + if (error) { + sf_reject_promise_reason(p, error); + scr_promise_release(p); + if (t->has_pending_abort) { + t->has_pending_abort = false; + sf_reject_promise_reason(t->pending_abort, error); + scr_promise_release(t->pending_abort); + t->pending_abort = NULL; + scr_dyn_release(t->pending_abort_reason); + t->pending_abort_reason = NULL; + } + sfw_deal_with_rejection(t, error); + return; + } + scr_promise_fulfill_void(p); + scr_promise_release(p); + if (t->state == SFW_ERRORING) { + scr_dyn_release(t->stored_error); + t->stored_error = NULL; + if (t->has_pending_abort) { + t->has_pending_abort = false; + scr_promise_fulfill_void(t->pending_abort); + scr_promise_release(t->pending_abort); + t->pending_abort = NULL; + scr_dyn_release(t->pending_abort_reason); + t->pending_abort_reason = NULL; + } + } + t->state = SFW_CLOSED; + if (t->writer) scr_promise_fulfill_void(t->writer->closed); +} + +/* WritableStreamAbort: +1 promise. */ +static ScrPromise *sfw_abort(SfTransform *t, ScrDyn *reason) { + if (t->state == SFW_CLOSED || t->state == SFW_ERRORED) { + return sf_promise_resolved(); + } + if (t->has_pending_abort) return scr_promise_retain(t->pending_abort); + bool was_erroring = t->state == SFW_ERRORING; + ScrPromise *p = scr_promise_new(); + t->has_pending_abort = true; + t->pending_abort = scr_promise_retain(p); + t->pending_abort_reason = + scr_dyn_retain(was_erroring ? scr_dyn_undefined() : reason); + t->pending_abort_was_erroring = was_erroring; + if (!was_erroring) sfw_start_erroring(t, reason); + return p; +} + +/* WritableStreamClose: +1 promise. */ +static ScrPromise *sfw_close(SfTransform *t) { + if (t->state == SFW_CLOSED || t->state == SFW_ERRORED) { + char message[64]; + snprintf(message, sizeof message, "WritableStream is %s", + t->state == SFW_CLOSED ? "closed" : "errored"); + return sf_promise_invalid_state(message); + } + ScrPromise *p = scr_promise_new(); + t->close_request = scr_promise_retain(p); + if (t->writer && t->backpressure && t->state == SFW_WRITABLE) { + scr_promise_fulfill_void(t->writer->ready); + } + t->close_queued = true; + sfw_advance_queue(t); + return p; +} + +/* WritableStreamDefaultWriterWrite: +1 promise. */ +static ScrPromise *sfw_writer_write(SfWriter *w, ScrDyn *chunk) { + SfTransform *t = w->stream; + if (!t) return sf_promise_invalid_state("Writer is not bound to a WritableStream"); + if (t->state == SFW_ERRORED || t->state == SFW_ERRORING) { + return sf_promise_rejected_with(t->stored_error); + } + if (sfw_close_queued_or_in_flight(t) || t->state == SFW_CLOSED) { + return sf_promise_invalid_state("WritableStream is closed"); + } + SfWriteReq *req = calloc(1, sizeof *req); + if (!req) sf_oom(); + req->chunk = scr_dyn_retain(chunk); + req->promise = scr_promise_new(); + ScrPromise *p = scr_promise_retain(req->promise); + if (t->tail) t->tail->next = req; + else t->head = req; + t->tail = req; + t->queue_size++; + sfw_update_backpressure(t, t->queue_size >= 1); + sfw_advance_queue(t); + return p; +} + +static SfWriter *sfw_get_writer(SfTransform *t) { + if (t->writer) { + sf_invalid_state("WritableStream is locked"); + return NULL; + } + SfWriter *w = calloc(1, sizeof *w); + if (!w) sf_oom(); + w->rc = 1; + w->stream = t; + w->ready = scr_promise_new(); + w->closed = scr_promise_new(); + scr_promise_mark_handled(w->ready); + scr_promise_mark_handled(w->closed); + switch (t->state) { + case SFW_WRITABLE: + if (sfw_close_queued_or_in_flight(t) || !t->backpressure) { + scr_promise_fulfill_void(w->ready); + } + break; + case SFW_ERRORING: + sf_reject_promise_reason(w->ready, t->stored_error); + break; + case SFW_CLOSED: + scr_promise_fulfill_void(w->ready); + scr_promise_fulfill_void(w->closed); + break; + default: + sf_reject_promise_reason(w->ready, t->stored_error); + sf_reject_promise_reason(w->closed, t->stored_error); + break; + } + t->writer = w; + w->rc++; /* the stream's ownership */ + return w; +} + +static void sfw_writer_release_lock(SfWriter *w) { + SfTransform *t = w->stream; + if (!t) return; + sf_invalid_state("Writer has been released"); + ScrCaught *caught = scr_exc_take(); + ScrDyn *released = scr_caught_to_dyn(caught); + scr_caught_release(caught); + sfw_writer_ensure_ready_rejected(w, released); + sfw_writer_ensure_closed_rejected(w, released); + scr_dyn_release(released); + size_t handle_refs = w->handle_refs; + t->writer = NULL; + w->stream = NULL; + sf_writer_release(w); /* the stream's ownership */ + while (handle_refs-- > 0) sf_transform_release(t); +} + +/* ── TransformStream algorithms (identity transformer) ── */ + +static void sft_set_backpressure(SfTransform *t, bool backpressure) { + if (t->bp_change) { + scr_promise_fulfill_void(t->bp_change); + scr_promise_release(t->bp_change); + } + t->bp_change = scr_promise_new(); + t->t_backpressure = backpressure; + if (t->write_waiting) { + t->write_waiting = false; + sf_transform_defer(t, SFT_WRITE_CONTINUE, 1); + } +} + +/* After an enqueue: whether a read is still waiting. A fetch upload's read + * was just satisfied by the chunk, and it reads again through the next + * pull, as a reader's next read() does. */ +static bool sft_readable_wants_chunk(const SfStream *r) { + return r->collector || (r->reader && r->reader->pending_head); +} + +/* The readable side is closed or errored: a pull still waiting on + * bp_change can no longer receive a chunk, so complete it. */ +static void sft_finish_pull(SfTransform *t) { + if (t->bp_change && scr_promise_is_pending(t->bp_change)) scr_promise_fulfill_void(t->bp_change); +} + +static bool sft_readable_can_enqueue(const SfStream *r) { + return !r->close_requested && !r->closed && !r->error; +} + +/* TransformStreamErrorWritableAndUnblockWrite */ +static void sft_error_writable(SfTransform *t, ScrDyn *error) { + sfw_error_if_needed(t, error); + if (t->t_backpressure) sft_set_backpressure(t, false); +} + +/* The identity transform's enqueue: NULL on success, else +1 error. */ +static ScrDyn *sft_perform_transform(SfTransform *t, ScrDyn *chunk) { + SfStream *r = t->readable; + if (!sft_readable_can_enqueue(r)) { + sf_invalid_state("Unable to enqueue"); + ScrCaught *caught = scr_exc_take(); + ScrDyn *error = scr_caught_to_dyn(caught); + scr_caught_release(caught); + sft_error_writable(t, error); + return error; + } + sf_stream_enqueue_value(r, chunk); + if (scr_exc_pending()) { + ScrCaught *caught = scr_exc_take(); + ScrDyn *error = scr_caught_to_dyn(caught); + scr_caught_release(caught); + sf_stream_error(r, error); + sft_error_writable(t, error); + return error; + } + bool backpressure = !sft_readable_wants_chunk(r); + if (backpressure != t->t_backpressure) sft_set_backpressure(t, true); + return NULL; +} + +static void sft_sink_write(SfTransform *t) { + if (t->t_backpressure) { + t->write_waiting = true; + return; + } + ScrDyn *error = sft_perform_transform(t, t->in_flight->chunk); + if (error) { + scr_dyn_release(t->write_error); + t->write_error = error; + } + sf_transform_defer(t, SFT_WRITE_DONE, 2); +} + +static void sft_sink_close(SfTransform *t) { + if (!t->finish) { + t->finish = scr_promise_new(); + scr_promise_mark_handled(t->finish); + sf_transform_defer(t, SFT_CLOSE_DONE, 1); + return; + } + /* A cancel or abort already finished the transform. */ + sf_transform_defer(t, SFT_CLOSE_DONE, 1); +} + +static void sft_abort_algorithm(SfTransform *t, ScrDyn *reason) { + (void)reason; + t->abort_joins_finish = t->finish != NULL; + if (!t->finish) { + t->finish = scr_promise_new(); + scr_promise_mark_handled(t->finish); + } + sf_transform_defer(t, SFT_ABORT_DONE, 1); +} + +static void sf_transform_step(SfTransform *t, int op) { + switch (op) { + case SFT_WRITE_CONTINUE: { + if (!t->in_flight) return; + if (t->state == SFW_ERRORING) { + ScrDyn *error = scr_dyn_retain(t->stored_error); + scr_dyn_release(t->write_error); + t->write_error = error; + } else { + ScrDyn *error = sft_perform_transform(t, t->in_flight->chunk); + if (error) { + scr_dyn_release(t->write_error); + t->write_error = error; + } + } + sf_transform_defer(t, SFT_WRITE_DONE, 3); + return; + } + case SFT_WRITE_DONE: { + if (!t->in_flight) return; + ScrDyn *error = t->write_error; + t->write_error = NULL; + sfw_finish_write(t, error); + scr_dyn_release(error); + return; + } + case SFT_CLOSE_DONE: { + SfStream *r = t->readable; + ScrDyn *error = NULL; + if (r->error) { + error = scr_dyn_retain(r->error); + } else { + sf_stream_close(r); + } + sft_finish_pull(t); + if (error) sf_reject_promise_reason(t->finish, error); + else scr_promise_fulfill_void(t->finish); + if (t->in_flight_close) sfw_finish_close(t, error); + scr_dyn_release(error); + return; + } + case SFT_ABORT_DONE: { + SfStream *r = t->readable; + scr_dyn_release(t->abort_settle_error); + t->abort_settle_error = NULL; + if (t->abort_joins_finish && !scr_promise_is_pending(t->finish)) { + /* The cancel or close that created the finish promise settled + * it; the abort request takes its result. */ + if (scr_promise_is_rejected(t->finish)) + t->abort_settle_error = scr_promise_reason_dyn(t->finish); + } else { + ScrDyn *reason = t->pending_abort_reason + ? scr_dyn_retain(t->pending_abort_reason) + : scr_dyn_retain(scr_dyn_undefined()); + bool readable_errored = r->error != NULL; + if (!readable_errored) sf_stream_error(r, reason); + sft_finish_pull(t); + /* The sink abort algorithm's finish promise, which a + * readable.cancel() after the abort answers too: rejected with the + * readable side's stored error when it had already errored, else + * resolved. */ + if (t->finish && scr_promise_is_pending(t->finish)) { + if (readable_errored) sf_reject_promise_reason(t->finish, r->error); + else scr_promise_fulfill_void(t->finish); + } + if (readable_errored) t->abort_settle_error = scr_dyn_retain(r->error); + scr_dyn_release(reason); + } + /* WritableStreamFinishErroring settles the abort request in a + * reaction to the sink abort's promise, two turns after it. */ + sf_transform_defer(t, SFT_ABORT_SETTLE, 2); + return; + } + case SFT_ABORT_SETTLE: { + ScrDyn *error = t->abort_settle_error; + t->abort_settle_error = NULL; + if (t->has_pending_abort) { + t->has_pending_abort = false; + if (error) sf_reject_promise_reason(t->pending_abort, error); + else scr_promise_fulfill_void(t->pending_abort); + scr_promise_release(t->pending_abort); + t->pending_abort = NULL; + scr_dyn_release(t->pending_abort_reason); + t->pending_abort_reason = NULL; + } + sfw_reject_close_and_closed(t); + scr_dyn_release(error); + return; + } + case SFT_CANCEL_DONE: { + ScrDyn *reason = t->cancel_reason; + t->cancel_reason = NULL; + if (t->state == SFW_ERRORED) { + sf_reject_promise_reason(t->finish, t->stored_error); + } else { + sfw_error_if_needed(t, reason ? reason : scr_dyn_undefined()); + if (t->t_backpressure) sft_set_backpressure(t, false); + sft_finish_pull(t); + scr_promise_fulfill_void(t->finish); + } + scr_dyn_release(reason); + return; + } + case SFT_START: { + /* Both sides start one turn after construction, the writable side + * first (Node registers its start reaction first). */ + t->started = true; + sfw_advance_queue(t); + SfStream *r = t->readable; + r->started = true; + if (!r->close_requested && !r->closed && !r->error) { + sf_stream_drain(r); + sf_stream_pull(r); + } + return; + } + } +} + +/* The readable side's pull: release backpressure; the pull completes when + * backpressure changes again. */ +static ScrDyn *sft_pull_thunk(ScrClosure *cb, ScrDyn *const *args, + size_t argc) { + (void)args; + (void)argc; + SfTransform *t = (SfTransform *)scr_box_get_ref(cb->caps[0]); + sft_set_backpressure(t, false); + ScrDyn *out = scr_dyn_new_promise(t->bp_change); + sf_transform_release(t); + return out; +} + +/* The readable side's cancel (TransformStreamDefaultSourceCancelAlgorithm): + * errors the writable side with the reason and unblocks a waiting write. */ +static ScrDyn *sft_cancel_thunk(ScrClosure *cb, ScrDyn *const *args, + size_t argc) { + SfTransform *t = (SfTransform *)scr_box_get_ref(cb->caps[0]); + ScrDyn *reason = argc ? args[0] : scr_dyn_undefined(); + if (!t->finish) { + t->finish = scr_promise_new(); + scr_promise_mark_handled(t->finish); + scr_dyn_release(t->cancel_reason); + t->cancel_reason = scr_dyn_retain(reason); + /* The (identity) cancel algorithm's resolved promise reacts a turn + * later. */ + sf_transform_defer(t, SFT_CANCEL_DONE, 1); + } + ScrDyn *out = scr_dyn_new_promise(t->finish); + sf_transform_release(t); + return out; +} + +static ScrDyn *sft_func(SfTransform *t, ScrDynThunk thunk, const char *name) { + ScrClosure *cb = scr_closure_new((void *)thunk, 1); + cb->caps[0] = sf_transform_box(t); + return scr_dyn_new_func(cb, thunk, 1, "(controller)", name); +} + +ScrDyn *scr_fetch_transform_new(void) { + SfTransform *t = calloc(1, sizeof *t); + if (!t) sf_oom(); + t->rc = 1; + t->state = SFW_WRITABLE; + SfStream *r = sf_stream_new_native(); + r->started = false; + r->hwm_zero = true; + r->pull_cb = sft_func(t, &sft_pull_thunk, "pull"); + r->cancel_cb = sft_func(t, &sft_cancel_thunk, "cancel"); + r->source_this = scr_dyn_retain(scr_dyn_undefined()); + sf_stream_track_callbacks(r); + t->readable = r; + /* The writable side's high-water mark of 1 leaves room: no backpressure. + * The readable side's 0 does: writes wait for a read. */ + t->backpressure = false; + sft_set_backpressure(t, true); + sf_transform_defer(t, SFT_START, 1); + ScrDyn *out = scr_dyn_new_handle(t, SCR_DYNH_WEB_TRANSFORM); + sf_transform_release(t); + return out; +} + +static ScrDyn *sf_transform_get(void *ptr, const char *key, size_t len) { + SfTransform *t = ptr; + if (sf_name(key, len, "readable")) { + return scr_dyn_new_handle(t->readable, SCR_DYNH_WEB_STREAM); + } + if (sf_name(key, len, "writable")) { + return scr_dyn_new_handle(t, SCR_DYNH_WEB_WRITABLE); + } + return NULL; +} + +static ScrDyn *sf_transform_invoke(void *ptr, ScrDyn *self, + const char *method, ScrDyn *const *args, + size_t argc, const char *what) { + (void)ptr; + (void)self; + (void)args; + (void)argc; + (void)method; + sf_not_function(what); + return NULL; +} + +static ScrDyn *sf_promise_dyn(ScrPromise *p) { + ScrDyn *out = scr_dyn_new_promise(p); + scr_promise_release(p); + return out; +} + +static ScrDyn *sf_writable_invoke(void *ptr, ScrDyn *self, + const char *method, ScrDyn *const *args, + size_t argc, const char *what) { + SfTransform *t = ptr; + (void)self; + if (strcmp(method, "getWriter") == 0) { + SfWriter *w = sfw_get_writer(t); + if (!w) return NULL; + ScrDyn *out = scr_dyn_new_handle(w, SCR_DYNH_WEB_WRITER); + sf_writer_release(w); + return out; + } + if (strcmp(method, "abort") == 0) { + if (t->writer) { + return sf_promise_dyn( + sf_promise_invalid_state("WritableStream is locked")); + } + return sf_promise_dyn(sfw_abort(t, argc ? args[0] : scr_dyn_undefined())); + } + if (strcmp(method, "close") == 0) { + if (t->writer) { + return sf_promise_dyn( + sf_promise_invalid_state("WritableStream is locked")); + } + if (sfw_close_queued_or_in_flight(t)) { + return sf_promise_dyn( + sf_promise_invalid_state("WritableStream is closed")); + } + return sf_promise_dyn(sfw_close(t)); + } + sf_not_function(what); + return NULL; +} + +static ScrDyn *sf_writable_get(void *ptr, const char *key, size_t len) { + SfTransform *t = ptr; + if (sf_name(key, len, "locked")) return scr_dyn_new_bool(t->writer != NULL); + return NULL; +} + +static ScrDyn *sf_writer_invoke(void *ptr, ScrDyn *self, const char *method, + ScrDyn *const *args, size_t argc, + const char *what) { + SfWriter *w = ptr; + (void)self; + if (strcmp(method, "write") == 0) { + return sf_promise_dyn( + sfw_writer_write(w, argc ? args[0] : scr_dyn_undefined())); + } + if (strcmp(method, "close") == 0) { + SfTransform *t = w->stream; + if (!t) { + return sf_promise_dyn(sf_promise_invalid_state( + "Writer is not bound to a WritableStream")); + } + if (sfw_close_queued_or_in_flight(t)) { + return sf_promise_dyn( + sf_promise_invalid_state("WritableStream is closed")); + } + return sf_promise_dyn(sfw_close(t)); + } + if (strcmp(method, "abort") == 0) { + SfTransform *t = w->stream; + if (!t) { + return sf_promise_dyn(sf_promise_invalid_state( + "Writer is not bound to a WritableStream")); + } + return sf_promise_dyn(sfw_abort(t, argc ? args[0] : scr_dyn_undefined())); + } + if (strcmp(method, "releaseLock") == 0) { + sfw_writer_release_lock(w); + return scr_dyn_retain(scr_dyn_undefined()); + } + sf_not_function(what); + return NULL; +} + +static ScrDyn *sf_writer_get(void *ptr, const char *key, size_t len) { + SfWriter *w = ptr; + if (sf_name(key, len, "ready")) return scr_dyn_new_promise(w->ready); + if (sf_name(key, len, "closed")) return scr_dyn_new_promise(w->closed); + if (sf_name(key, len, "desiredSize")) { + SfTransform *t = w->stream; + if (!t) { + sf_invalid_state("Writer is not bound to a WritableStream"); + return NULL; + } + if (t->state == SFW_ERRORED || t->state == SFW_ERRORING) { + return scr_dyn_new_null(); + } + if (t->state == SFW_CLOSED) return scr_dyn_new_num(0); + return scr_dyn_new_num(1.0 - (double)t->queue_size); + } + return NULL; +} + /* ── Response ────────────────────────────────────────────────────── */ static SfResponse *sf_response_retain(SfResponse *r) { @@ -2660,6 +3883,7 @@ static void sf_response_release(SfResponse *r) { sf_headers_release(r->headers); scr_str_release(r->url); scr_str_release(r->status_text); + scr_arr_release(r->light_record); free(r); } @@ -2670,6 +3894,13 @@ static void sf_response_release_v(void *p) { sf_response_release((SfResponse *)p); } +/* A read the package's lightweight Response answers from its native + * response materializes it; status, ok and headers keep it light, and a + * headers read turns the header form into a Headers object. */ +static void sf_response_materialize(SfResponse *r) { + r->light = 0; +} + static ScrPromise *sf_response_collect(SfResponse *r, int mode) { if (!r->null_body) return sf_stream_collect(r->body, mode); @@ -2703,11 +3934,54 @@ static ScrPromise *sf_response_collect(SfResponse *r, int mode) { return p; } +/* The body and metadata copy of Response.clone and Request.clone (undici + * cloneResponse/cloneBody): the body tees, so the original and the clone + * each keep one branch, and the clone's headers are a copy with the same + * guard. NULL with a TypeError pending when the body is unusable. */ +static SfResponse *sf_response_clone(SfResponse *r, const char *unusable) { + if (!r->null_body && + (r->body->disturbed || r->body->reader || r->body->internal_lock)) { + sf_type_error(unusable); + return NULL; + } + SfResponse *c = calloc(1, sizeof *c); + if (!c) sf_oom(); + c->rc = 1; + if (r->null_body) { + c->body = sf_stream_new_native(); + sf_stream_close(c->body); + } else { + SfStream *branches[2]; + sf_stream_tee(r->body, branches); + sf_stream_release(r->body); + r->body = branches[0]; + c->body = branches[1]; + } + c->headers = sf_headers_new(scr_arr_slice(r->headers->pairs, 0, INFINITY), + r->headers->immutable); + c->url = scr_str_retain(r->url); + c->status_text = scr_str_retain(r->status_text); + c->status = r->status; + c->redirected = r->redirected; + c->null_body = r->null_body; + c->light_length = r->light_length; + return c; +} + static ScrDyn *sf_response_invoke(void *ptr, ScrDyn *self, const char *method, ScrDyn *const *args, size_t argc, const char *what) { SfResponse *r = ptr; + sf_response_materialize(r); + if (strcmp(method, "clone") == 0) { + SfResponse *c = sf_response_clone( + r, "Response.clone: Body has already been consumed."); + if (!c) return NULL; + ScrDyn *out = scr_dyn_new_handle(c, SCR_DYNH_FETCH_RESPONSE); + sf_response_release(c); + return out; + } int mode = -1; if (strcmp(method, "json") == 0) mode = SF_COLLECT_JSON; else if (strcmp(method, "text") == 0) mode = SF_COLLECT_TEXT; @@ -2731,6 +4005,19 @@ static ScrDyn *sf_response_get(void *ptr, const char *key, size_t len) { if (sf_name(key, len, "ok")) { return scr_dyn_new_bool(r->status >= 200 && r->status <= 299); } + if (sf_name(key, len, "headers")) { + if (r->light == 1) { + r->light = 2; + r->light_default_ct = !r->null_body; + } else if (r->light == 3) { + r->light = 2; + } + return scr_dyn_new_handle(r->headers, SCR_DYNH_FETCH_HEADERS); + } + if (sf_name(key, len, "body") || sf_name(key, len, "bodyUsed") || + sf_name(key, len, "url") || sf_name(key, len, "redirected") || + sf_name(key, len, "statusText")) + sf_response_materialize(r); if (sf_name(key, len, "bodyUsed")) { return scr_dyn_new_bool(!r->null_body && r->body->disturbed); } @@ -2746,9 +4033,6 @@ static ScrDyn *sf_response_get(void *ptr, const char *key, size_t len) { if (sf_name(key, len, "statusText")) { return scr_dyn_new_str(r->status_text); } - if (sf_name(key, len, "headers")) { - return scr_dyn_new_handle(r->headers, SCR_DYNH_FETCH_HEADERS); - } return NULL; } @@ -2760,6 +4044,7 @@ static ScrPromise *sf_response_collect_dyn(ScrDyn *response, int mode) { &scr_dyn_release_v, NULL); } SfResponse *r = response->v.handle.ptr; + sf_response_materialize(r); return sf_response_collect(r, mode); } @@ -3285,14 +4570,13 @@ static bool sf_pairs_have(ScrArr *pairs, const char *name) { size_t n = (size_t)scr_arr_len(pairs); size_t want = strlen(name); for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = (ScrStr *)scr_arr_get_ref(pairs, (double)i); + ScrStr *key = sf_pair_at(pairs, i); bool same = key->len == want; for (size_t j = 0; j < want && same; j++) { char c = key->data[j]; if (c >= 'A' && c <= 'Z') c = (char)(c - 'A' + 'a'); same = c == name[j]; } - scr_str_release(key); if (same) return true; } return false; @@ -3320,6 +4604,10 @@ static bool sf_header_value_ok(const ScrStr *value) { } static ScrStr *sf_header_value_bytestring(ScrStr *value) { + /* An ASCII value is its own ByteString: each byte is one UTF-16 unit. */ + size_t ascii = 0; + while (ascii < value->len && (unsigned char)value->data[ascii] < 0x80) ascii++; + if (ascii == value->len) return scr_str_retain(value); size_t len = (size_t)scr_str_utf16_len(value); char *bytes = malloc(len ? len : 1); if (!bytes) sf_oom(); @@ -3388,6 +4676,21 @@ static bool sf_request_header_ok( !sf_header_name_ci(name, name_len, "expect"); } +/* Whether every value in `pairs` is already trimmed (sf_trim_header_value + * would keep it): the form a copy can take unchanged (SfHeaders.canonical). + * Combining duplicate names with an empty value leaves the separator's + * space at the end of the combined value, which a copy trims again. */ +static bool sf_header_values_trimmed(ScrArr *pairs) { + size_t n = (size_t)scr_arr_len(pairs); + for (size_t i = 1; i < n; i += 2) { + const ScrStr *value = sf_pair_at(pairs, i); + if (value->len == 0) continue; + char first = value->data[0], last = value->data[value->len - 1]; + if (first == ' ' || first == '\t' || last == ' ' || last == '\t') return false; + } + return true; +} + static ScrStr *sf_trim_header_value(const ScrStr *value) { size_t start = 0; size_t end = value->len; @@ -3399,11 +4702,14 @@ static ScrStr *sf_trim_header_value(const ScrStr *value) { (value->data[end - 1] == ' ' || value->data[end - 1] == '\t')) { end--; } + if (start == 0 && end == value->len) return scr_str_retain((ScrStr *)value); return scr_str_new(value->data + start, end - start); } -static bool sf_push_header(ScrArr *pairs, const char *name, size_t name_len, +static bool sf_push_header(ScrArr *pairs, ScrStr *name_str, ScrStr *value, bool request_guard) { + const char *name = name_str->data; + size_t name_len = name_str->len; if (!sf_header_name_ok(name, name_len)) { ScrStr *display = scr_str_new(name, name_len); sf_headers_invalid("append", display, "name"); @@ -3429,7 +4735,7 @@ static bool sf_push_header(ScrArr *pairs, const char *name, size_t name_len, if (!sf_header_name_ci(name, name_len, "set-cookie")) { size_t n = (size_t)scr_arr_len(pairs); for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *key = scr_arr_get_ref(pairs, (double)i); + ScrStr *key = sf_pair_at(pairs, i); bool match = key->len == name_len; for (size_t j = 0; j < name_len && match; j++) { unsigned char a = (unsigned char)key->data[j]; @@ -3438,11 +4744,9 @@ static bool sf_push_header(ScrArr *pairs, const char *name, size_t name_len, if (b >= 'A' && b <= 'Z') b = (unsigned char)(b - 'A' + 'a'); match = a == b; } - scr_str_release(key); if (!match) continue; - ScrStr *previous = - scr_arr_get_ref(pairs, (double)(i + 1)); + ScrStr *previous = sf_pair_at(pairs, i + 1); const char *separator = sf_header_name_ci(name, name_len, "cookie") ? "; " : ", "; size_t separator_len = 2; @@ -3456,7 +4760,6 @@ static bool sf_push_header(ScrArr *pairs, const char *name, size_t name_len, normalized->data, normalized->len); ScrStr *joined = scr_str_new(joined_data, joined_len); free(joined_data); - scr_str_release(previous); scr_str_release(normalized); if (request_guard && !sf_request_header_ok(name, name_len, joined)) { scr_str_release(joined); @@ -3475,8 +4778,8 @@ static bool sf_push_header(ScrArr *pairs, const char *name, size_t name_len, scr_arr_push_ref( pairs, request_guard - ? scr_str_new(name, name_len) - : sf_header_name_lower(name, name_len)); + ? scr_str_retain(name_str) + : sf_header_name_lower_str(name_str)); scr_arr_push_ref(pairs, normalized); return true; } @@ -3503,10 +4806,13 @@ static void sf_headers_pair_length_error(size_t len) { * leaves header-name/value syntax validation for initialization. Response * construction converts this snapshot before status/statusText and only * validates it after body extraction, matching Undici's two phases. */ -static ScrArr *sf_headers_convert_init(const ScrDyn *headers) { +/* *canonical_out reports that the result copies a canonical Headers + * object's pairs (SfHeaders.canonical). */ +static ScrArr *sf_headers_convert_init(const ScrDyn *headers, bool *canonical_out) { + *canonical_out = false; if (headers && headers->kind == SCR_DYN_TYPED_REF) { ScrDyn *materialized = scr_dyn_typed_ref_materialize(headers); - ScrArr *converted = sf_headers_convert_init(materialized); + ScrArr *converted = sf_headers_convert_init(materialized, canonical_out); scr_dyn_release(materialized); return converted; } @@ -3597,13 +4903,9 @@ static ScrArr *sf_headers_convert_init(const ScrDyn *headers) { headers->v.handle.tag == SCR_DYNH_FETCH_HEADERS) { SfHeaders *source = (SfHeaders *)headers->v.handle.ptr; size_t n = (size_t)scr_arr_len(source->pairs); - for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *name = scr_arr_get_ref(source->pairs, (double)i); - ScrStr *value = - scr_arr_get_ref(source->pairs, (double)(i + 1)); - scr_arr_push_ref(converted, name); - scr_arr_push_ref(converted, value); - } + for (size_t i = 0; i < n - n % 2; i++) + scr_arr_push_ref(converted, scr_str_retain(sf_pair_at(source->pairs, i))); + *canonical_out = source->canonical; return converted; } /* Functions are WebIDL objects and an ordinary function has no @@ -3618,25 +4920,29 @@ fail: } static bool sf_add_converted_headers( - ScrArr *pairs, ScrArr *converted, bool request_guard) { + ScrArr *pairs, ScrArr *converted, bool request_guard, bool canonical) { size_t n = (size_t)scr_arr_len(converted); + /* Pushing canonical pairs into an empty list one by one through + * sf_push_header would reproduce them exactly. */ + if (canonical && !request_guard && scr_arr_len(pairs) == 0) { + for (size_t i = 0; i < n; i++) + scr_arr_push_ref(pairs, scr_str_retain(sf_pair_at(converted, i))); + return true; + } for (size_t i = 0; i + 1 < n; i += 2) { - ScrStr *name = scr_arr_get_ref(converted, (double)i); - ScrStr *value = scr_arr_get_ref(converted, (double)(i + 1)); - bool ok = sf_push_header( - pairs, name->data, name->len, value, request_guard); - scr_str_release(name); - scr_str_release(value); - if (!ok) return false; + if (!sf_push_header(pairs, sf_pair_at(converted, i), + sf_pair_at(converted, i + 1), request_guard)) + return false; } return true; } static bool sf_add_headers( ScrArr *pairs, const ScrDyn *headers, bool request_guard) { - ScrArr *converted = sf_headers_convert_init(headers); + bool canonical = false; + ScrArr *converted = sf_headers_convert_init(headers, &canonical); if (!converted) return false; - bool ok = sf_add_converted_headers(pairs, converted, request_guard); + bool ok = sf_add_converted_headers(pairs, converted, request_guard, canonical); scr_arr_release(converted); return ok; } @@ -4752,11 +6058,71 @@ static ScrDyn *sf_response_json_serialize(const ScrDyn *value) { return serialized; } +/* The lightweight header form of a constructor's init headers (see + * SfResponse.light): absent headers are form 1 (Response.json's plain + * {"content-type": "application/json"} record is form 3), a Headers object + * or pair list form 2, and a plain record form 3, whose names and values + * *record receives in order (an array value contributes one entry per + * element, as writeHead writes it). Response.json with init headers is + * form 2 whatever their shape: the package's Response.json passes them on + * as a Headers object, adding the JSON content type when they lack one. + * 0 for any other init. */ +static int8_t sf_light_header_form(const ScrDyn *headers, bool json_body, + ScrArr **record) { + if (!headers || headers->kind == SCR_DYN_UNDEF) { + if (!json_body) return 1; + *record = scr_arr_new(SCR_ELEM_STR, 2); + sf_push_header_text(*record, "content-type", "application/json", 16); + return 3; + } + if (json_body) return 2; + if (headers->kind == SCR_DYN_ARR || + (headers->kind == SCR_DYN_HANDLE && + headers->v.handle.tag == SCR_DYNH_FETCH_HEADERS)) + return 2; + ScrDyn *materialized = NULL; + if (headers->kind == SCR_DYN_TYPED_REF) { + materialized = scr_dyn_typed_ref_materialize(headers); + if (!materialized) return 0; + headers = materialized; + } + int8_t form = 0; + if (headers->kind == SCR_DYN_OBJ) { + ScrArr *out = scr_arr_new(SCR_ELEM_STR, 2 * headers->v.obj.len); + form = 3; + for (size_t i = 0; i < headers->v.obj.len && form == 3; i++) { + const ScrDynEntry *e = &headers->v.obj.entries[i]; + const ScrDyn *value = e->value; + size_t count = value->kind == SCR_DYN_ARR ? value->v.arr.len : 1; + for (size_t j = 0; j < count; j++) { + const ScrDyn *item = value->kind == SCR_DYN_ARR ? value->v.arr.items[j] : value; + if (item->kind != SCR_DYN_STR && item->kind != SCR_DYN_NUM && + item->kind != SCR_DYN_BOOL) { + form = 0; + break; + } + ScrStr *text = scr_dyn_string_coerce_js(item); + if (!text) { + form = 0; + break; + } + scr_arr_push_ref(out, scr_str_new(e->key, e->key_len)); + scr_arr_push_ref(out, text); + } + } + if (form == 3) *record = out; + else scr_arr_release(out); + } + scr_dyn_release(materialized); + return form; +} + static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, const ScrDyn *json_data, SfJsonSerialize serialize) { ScrArr *header_pairs = NULL; ScrArr *converted_headers = NULL; + bool converted_canonical = false; ScrBytes *body_bytes = NULL; ScrStr *status_text = NULL; SfStream *stream = NULL; @@ -4766,6 +6132,23 @@ static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, bool text_body = false; int status = 200; bool status_range_error = false; + /* The lightweight-response classification (see SfResponse.light): a + * Response init qualifies only while it is itself still light. */ + bool light_ok = !(init && init->kind == SCR_DYN_HANDLE && + init->v.handle.tag == SCR_DYNH_FETCH_RESPONSE && + ((SfResponse *)init->v.handle.ptr)->light == 0); + if (!json_body && !null_body) { + if (body->kind == SCR_DYN_BYTES) + light_ok = light_ok && body->v.bytes->elem == SCR_BYTES_U8 && + !body->v.bytes->is_data_view; + else if (!(body->kind == SCR_DYN_STR || + (body->kind == SCR_DYN_HANDLE && + body->v.handle.tag == SCR_DYNH_WEB_STREAM))) + light_ok = false; + } + int8_t light_form = 0; + ScrArr *light_record = NULL; + size_t light_length_bytes = 0; /* WebIDL converts constructor arguments from left to right. BodyInit's * string arm therefore runs before ResponseInit's dictionary conversion, @@ -4803,7 +6186,8 @@ static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, * coercions precede status and statusText. Header syntax validation is a * later response-initialization step, after body extraction. */ ScrDyn *init_headers = sf_response_init_get(init, "headers", 7); - converted_headers = sf_headers_convert_init(init_headers); + if (light_ok) light_form = sf_light_header_form(init_headers, json_body, &light_record); + converted_headers = sf_headers_convert_init(init_headers, &converted_canonical); scr_dyn_release(init_headers); if (!converted_headers) goto fail; @@ -4859,6 +6243,7 @@ static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, scr_str_release(encoding); scr_dyn_release(serialized); stream = sf_stream_new_native(); + light_length_bytes = body_bytes->len; sf_stream_enqueue_bytes(stream, body_bytes); sf_stream_close(stream); scr_bytes_release(body_bytes); @@ -4877,6 +6262,7 @@ static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, sf_stream_retain(stream); } else { stream = sf_stream_new_native(); + light_length_bytes = body_bytes->len; sf_stream_enqueue_bytes(stream, body_bytes); sf_stream_close(stream); scr_bytes_release(body_bytes); @@ -4896,7 +6282,7 @@ static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, header_pairs = scr_arr_new(SCR_ELEM_STR, 8); if (!sf_add_converted_headers( - header_pairs, converted_headers, false)) goto fail; + header_pairs, converted_headers, false, converted_canonical)) goto fail; scr_arr_release(converted_headers); converted_headers = NULL; @@ -4910,9 +6296,11 @@ static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, sf_type_error(message); goto fail; } + bool auto_ct = false; if (text_body && !sf_pairs_have(header_pairs, "content-type")) { sf_push_header_text(header_pairs, "content-type", "text/plain;charset=UTF-8", 24); + auto_ct = true; } if (json_body && !sf_pairs_have(header_pairs, "content-type")) { static const char json_type[16] = "application/json"; @@ -4925,11 +6313,20 @@ static ScrDyn *sf_response_construct(ScrDyn *body, ScrDyn *init, response->rc = 1; response->body = stream; response->headers = sf_headers_new(header_pairs, false); + response->headers->canonical = sf_header_values_trimmed(header_pairs); response->url = scr_str_new("", 0); response->status_text = status_text; response->status = status; response->redirected = false; response->null_body = null_body; + response->light = light_form; + response->auto_ct = auto_ct; + response->light_record = light_record; + light_record = NULL; + response->light_length = + null_body ? -1.0 + : (!json_body && body->kind == SCR_DYN_HANDLE) ? -2.0 + : (double)light_length_bytes; ScrDyn *out = scr_dyn_new_handle(response, SCR_DYNH_FETCH_RESPONSE); sf_response_release(response); @@ -4941,9 +6338,43 @@ fail: scr_str_release(status_text); scr_arr_release(header_pairs); scr_arr_release(converted_headers); + scr_arr_release(light_record); return NULL; } +/* @hono/node-server's built-in implementation reads a Response's + * lightweight state to write it as the package does: [form, flags, + * byte length, ...raw record name/value pairs], with form as in + * SfResponse.light ("0" alone when the response is not light), flags "d" + * (the package's Headers hold its default content type) and "a" (the + * constructor added the text content type), and the length -1 for a null + * body or -2 for a stream. */ +ScrArr *scr_fetch_response_light(ScrDyn *response) { + ScrArr *out = scr_arr_new(SCR_ELEM_STR, 4); + if (!response || response->kind != SCR_DYN_HANDLE || + response->v.handle.tag != SCR_DYNH_FETCH_RESPONSE) { + scr_arr_push_ref(out, scr_str_new("0", 1)); + return out; + } + SfResponse *r = response->v.handle.ptr; + char buf[32]; + int n = snprintf(buf, sizeof buf, "%d", (int)r->light); + scr_arr_push_ref(out, scr_str_new(buf, (size_t)n)); + if (r->light == 0) return out; + char flags[3]; + size_t f = 0; + if (r->light_default_ct) flags[f++] = 'd'; + if (r->auto_ct) flags[f++] = 'a'; + scr_arr_push_ref(out, scr_str_new(flags, f)); + n = snprintf(buf, sizeof buf, "%.0f", r->light_length); + scr_arr_push_ref(out, scr_str_new(buf, (size_t)n)); + if (r->light == 3 && r->light_record) { + for (size_t i = 0; i < r->light_record->len; i++) + scr_arr_push_ref(out, scr_arr_get_ref(r->light_record, (double)i)); + } + return out; +} + ScrDyn *scr_fetch_response_new(ScrDyn *body, ScrDyn *init) { return sf_response_construct(body, init, NULL, NULL); } @@ -4990,21 +6421,52 @@ static ScrDyn *sf_request_get(void *ptr, const char *key, size_t len) { if (default_value) { ScrStr *s = scr_str_new(default_value, strlen(default_value)); ScrDyn *v = scr_dyn_new_str(s); scr_str_release(s); return v; } if (sf_name(key, len, "keepalive") || sf_name(key, len, "isReloadNavigation") || sf_name(key, len, "isHistoryNavigation")) return scr_dyn_new_bool(false); if (sf_name(key, len, "body") || sf_name(key, len, "bodyUsed") || sf_name(key, len, "headers")) return sf_response_get(r->payload, key, len); - return scr_dyn_undefined(); + return NULL; +} +/* Request.clone: the payload clones as a Response does, and the clone's + * signal follows the original's (undici: an AbortController aborted with + * the original signal's reason, at once when it already aborted). */ +static ScrDyn *sf_request_clone(SfRequest *r) { + SfResponse *payload = sf_response_clone(r->payload, "unusable"); + if (!payload) return NULL; + ScrDyn *sources = scr_dyn_new_arr(); + scr_dyn_arr_push(sources, + scr_dyn_new_handle(r->signal, SCR_DYNH_ABORT_SIGNAL)); + ScrDyn *follower = scr_fetch_abort_any(sources); + scr_dyn_release(sources); + if (!follower) { + sf_response_release(payload); + return NULL; + } + SfRequest *c = calloc(1, sizeof *c); + if (!c) sf_oom(); + c->rc = 1; + c->payload = payload; + c->url = scr_str_retain(r->url); + c->method = scr_str_retain(r->method); + c->redirect = scr_str_retain(r->redirect); + c->signal = sf_signal_retain((SfSignal *)follower->v.handle.ptr); + scr_dyn_release(follower); + ScrDyn *out = scr_dyn_new_handle(c, SCR_DYNH_FETCH_REQUEST); + sf_request_release_v(c); + return out; } static ScrDyn *sf_request_invoke(void *ptr, ScrDyn *self, const char *method, ScrDyn *const *args, size_t argc, const char *what) { SfRequest *r = ptr; + if (strcmp(method, "clone") == 0) return sf_request_clone(r); return sf_response_invoke(r->payload, self, method, args, argc, what); } ScrDyn *scr_fetch_headers_new(ScrDyn *init) { - ScrArr *converted = sf_headers_convert_init(init); + bool canonical = false; + ScrArr *converted = sf_headers_convert_init(init, &canonical); if (!converted) return NULL; ScrArr *pairs = scr_arr_new(SCR_ELEM_STR, 8); - bool ok = sf_add_converted_headers(pairs, converted, false); + bool ok = sf_add_converted_headers(pairs, converted, false, canonical); scr_arr_release(converted); if (!ok) { scr_arr_release(pairs); return NULL; } SfHeaders *headers = sf_headers_new(pairs, false); + headers->canonical = sf_header_values_trimmed(pairs); ScrDyn *result = scr_dyn_new_handle(headers, SCR_DYNH_FETCH_HEADERS); sf_headers_release(headers); return result; @@ -5495,33 +6957,94 @@ static bool sf_no_set(void *ptr, const char *key, size_t len, return false; } +/* The members invoke answers, as values (ScrDynHandleMethod), with Node's + * lengths and the brand check each one runs on another receiver. The + * signal's EventTarget methods are EventTarget's own (sf_signal_get). */ +static const ScrDynHandleMethod sf_signal_methods[] = { + {"throwIfAborted", 0, false}, {NULL, 0, false}}; +static const ScrDynHandleMethod sf_stream_methods[] = { + {"cancel", 0, true}, {"getReader", 0, false}, {NULL, 0, false}}; +static const ScrDynHandleMethod sf_reader_methods[] = { + {"read", 0, true}, {"releaseLock", 0, false}, {"cancel", 0, true}, + {NULL, 0, false}}; +static const ScrDynHandleMethod sf_body_methods[] = { + {"json", 0, true}, {"text", 0, true}, {"bytes", 0, true}, + {"arrayBuffer", 0, true}, {"clone", 0, false}, {NULL, 0, false}}; +static const ScrDynHandleMethod sf_headers_methods[] = { + {"append", 2, false}, {"delete", 1, false}, {"entries", 0, false}, + {"forEach", 1, false}, {"get", 1, false}, {"getSetCookie", 0, false}, + {"has", 1, false}, {"keys", 0, false}, {"set", 2, false}, + {"values", 0, false}, {NULL, 0, false}}; + +static const ScrDynHandleMethod sf_writable_methods[] = { + {"abort", 0, true}, {"close", 0, true}, {"getWriter", 0, false}, + {NULL, 0, false}}; +static const ScrDynHandleMethod sf_writer_methods[] = { + {"abort", 0, true}, {"close", 0, true}, {"releaseLock", 0, false}, + {"write", 0, true}, {NULL, 0, false}}; + +static ScrDyn *sf_rejected(void) { + ScrPromise *p = scr_promise_settled_ref(NULL, &scr_dyn_retain_v, + &scr_dyn_release_v, NULL); + ScrDyn *out = scr_dyn_new_promise(p); + scr_promise_release(p); + return out; +} + static const ScrDynHandleOps sf_signal_ops = { "AbortSignal", &sf_signal_retain_v, &sf_signal_release_v, - &sf_signal_invoke, &sf_signal_get, &sf_signal_set, NULL}; + &sf_signal_invoke, &sf_signal_get, &sf_signal_set, NULL, + .methods = sf_signal_methods, + .invalid_this = "Value of \"this\" must be of type AbortSignal"}; static const ScrDynHandleOps sf_abort_controller_ops = { "AbortController", &sf_signal_retain_v, &sf_signal_release_v, &sf_abort_controller_invoke, &sf_abort_controller_get, &sf_no_set, NULL}; static const ScrDynHandleOps sf_stream_ops = { "ReadableStream", &sf_stream_retain_v, &sf_stream_release_v, - &sf_stream_invoke, &sf_stream_get, &sf_no_set, NULL}; + &sf_stream_invoke, &sf_stream_get, &sf_no_set, NULL, + .methods = sf_stream_methods, + .invalid_this = "Value of \"this\" must be of type ReadableStream", + .rejected = &sf_rejected}; static const ScrDynHandleOps sf_reader_ops = { "ReadableStreamDefaultReader", &sf_reader_handle_retain_v, &sf_reader_handle_release_v, &sf_reader_invoke, &sf_reader_get, - &sf_no_set, NULL}; + &sf_no_set, NULL, .methods = sf_reader_methods, + .invalid_this = + "Value of \"this\" must be of type ReadableStreamDefaultReader", + .rejected = &sf_rejected}; static const ScrDynHandleOps sf_controller_ops = { "ReadableStreamDefaultController", &sf_stream_retain_v, &sf_stream_release_v, &sf_controller_invoke, &sf_controller_get, &sf_no_set, NULL}; static const ScrDynHandleOps sf_request_ops = { "Request", &sf_request_retain_v, &sf_request_release_v, - &sf_request_invoke, &sf_request_get, &sf_no_set, NULL}; + &sf_request_invoke, &sf_request_get, &sf_no_set, NULL, + .methods = sf_body_methods, .rejected = &sf_rejected}; static const ScrDynHandleOps sf_response_ops = { "Response", &sf_response_retain_v, &sf_response_release_v, - &sf_response_invoke, &sf_response_get, &sf_no_set, NULL}; + &sf_response_invoke, &sf_response_get, &sf_no_set, NULL, + .methods = sf_body_methods, .rejected = &sf_rejected}; static const ScrDynHandleOps sf_headers_ops = { "Headers", &sf_headers_retain_v, &sf_headers_release_v, &sf_headers_invoke, &sf_headers_get, &sf_no_set, NULL, NULL, - &sf_headers_step}; + &sf_headers_step, .methods = sf_headers_methods}; + +static const ScrDynHandleOps sf_transform_ops = { + "TransformStream", &sf_transform_retain_v, &sf_transform_release_v, + &sf_transform_invoke, &sf_transform_get, &sf_no_set, NULL}; +static const ScrDynHandleOps sf_writable_ops = { + "WritableStream", &sf_transform_retain_v, &sf_transform_release_v, + &sf_writable_invoke, &sf_writable_get, &sf_no_set, NULL, + .methods = sf_writable_methods, + .invalid_this = "Value of \"this\" must be of type WritableStream", + .rejected = &sf_rejected}; +static const ScrDynHandleOps sf_writer_ops = { + "WritableStreamDefaultWriter", &sf_writer_handle_retain_v, + &sf_writer_handle_release_v, &sf_writer_invoke, &sf_writer_get, + &sf_no_set, NULL, .methods = sf_writer_methods, + .invalid_this = + "Value of \"this\" must be of type WritableStreamDefaultWriter", + .rejected = &sf_rejected}; static void sf_teardown(void) { while (sf_live) { @@ -5562,6 +7085,9 @@ void scr_fetch_install(void) { scr_dyn_handle_install(SCR_DYNH_FETCH_REQUEST, &sf_request_ops); scr_dyn_handle_install(SCR_DYNH_FETCH_RESPONSE, &sf_response_ops); scr_dyn_handle_install(SCR_DYNH_FETCH_HEADERS, &sf_headers_ops); + scr_dyn_handle_install(SCR_DYNH_WEB_TRANSFORM, &sf_transform_ops); + scr_dyn_handle_install(SCR_DYNH_WEB_WRITABLE, &sf_writable_ops); + scr_dyn_handle_install(SCR_DYNH_WEB_WRITER, &sf_writer_ops); scr_et_signal_core_fn = sf_signal_core_of; scr_atexit(sf_teardown); } @@ -5572,9 +7098,13 @@ static bool sf_no_set(void *ptr, const char *key, size_t len, const ScrDyn *valu (void)ptr; (void)key; (void)len; (void)value; return false; } +static const ScrDynHandleMethod sf_signal_methods[] = { + {"throwIfAborted", 0, false}, {NULL, 0, false}}; static const ScrDynHandleOps sf_signal_ops = { "AbortSignal", &sf_signal_retain_v, &sf_signal_release_v, - &sf_signal_invoke, &sf_signal_get, &sf_signal_set, NULL }; + &sf_signal_invoke, &sf_signal_get, &sf_signal_set, NULL, + .methods = sf_signal_methods, + .invalid_this = "Value of \"this\" must be of type AbortSignal" }; static const ScrDynHandleOps sf_abort_controller_ops = { "AbortController", &sf_signal_retain_v, &sf_signal_release_v, &sf_abort_controller_invoke, &sf_abort_controller_get, &sf_no_set, NULL }; diff --git a/packages/runtime/src/scr_generator.h b/packages/runtime/src/scr_generator.h index c3dac861d..c73ceb900 100644 --- a/packages/runtime/src/scr_generator.h +++ b/packages/runtime/src/scr_generator.h @@ -181,6 +181,7 @@ ScrDyn *scr_fetch_headers_new(ScrDyn *init); ScrPromise *scr_fetch_static(ScrStr *url, ScrDyn *init); /* +1 promise */ ScrDyn *scr_fetch_response_new(ScrDyn *body, ScrDyn *init); /* borrowed args; +1 Response handle or NULL pending */ ScrDyn *scr_fetch_response_json_static(ScrDyn *data, ScrDyn *init); /* Response.json; borrowed args; +1 Response handle or NULL pending */ +ScrArr *scr_fetch_response_light(ScrDyn *response); /* @hono/node-server shim: borrowed; +1 string[] */ ScrPromise *scr_fetch_response_json(ScrDyn *response); /* +1 promise */ ScrPromise *scr_fetch_response_text(ScrDyn *response); /* +1 promise */ ScrPromise *scr_fetch_response_array_buffer(ScrDyn *response); /* +1 promise */ @@ -191,6 +192,7 @@ ScrDyn *scr_fetch_abort_timeout(ScrDyn *delay); ScrDyn *scr_fetch_abort_now(ScrDyn *reason); /* borrowed reason; +1 handle */ ScrDyn *scr_fetch_abort_any(ScrDyn *signals); /* borrowed array; +1 handle or NULL pending */ ScrDyn *scr_fetch_stream_new(ScrDyn *source); /* borrowed source; +1 handle or NULL pending */ +ScrDyn *scr_fetch_transform_new(void); /* +1 identity TransformStream handle */ ScrDyn *scr_fetch_stream_from(ScrDyn *iterable); /* borrowed iterable; +1 handle or NULL pending */ ScrDyn *scr_fetch_stream_from_array( ScrArr *iterable, diff --git a/packages/runtime/src/scr_http.c b/packages/runtime/src/scr_http.c index cb88297bf..29eb52ef7 100644 --- a/packages/runtime/src/scr_http.c +++ b/packages/runtime/src/scr_http.c @@ -239,10 +239,11 @@ struct ScrHttpReq { bool close_queued; bool close_emitted; /* settled: err/close listeners dropped */ bool join_dup; /* joinDuplicateHeaders: repeated names read joined ", " */ - /* pause(): delivery holds — arrived body bytes buffer in pend (the - * parser keeps consuming; memory is the buffer, a documented bound) - * and 'end' defers (end_pending) until resume() drains through the - * emit queue, never the resuming stack. */ + /* pause(): delivery holds — arrived body bytes buffer in pend and + * 'end' defers (end_pending) until resume() drains through the emit + * queue, never the resuming stack. A held HTTP/1 server request stops + * its socket's reads once pend reaches SCR_HTTP_REQ_HOLD_BYTES + * (sock_held), as Node's readStop does. */ bool paused; bool end_pending; bool drain_queued; @@ -257,6 +258,7 @@ struct ScrHttpReq { bool received; /* the whole body arrived (req.complete) */ bool body_seen; /* body bytes arrived */ bool destroyed; /* req.destroy()/the teardown ran — req.destroyed */ + bool sock_held; /* this request stopped its socket's reads */ ScrNetLs aborted_ls; /* req.on('aborted') — the h2 compat event */ }; @@ -330,7 +332,10 @@ void scr_http_req_release(ScrHttpReq *r) { scr_http_client_release(r->pipe_client); scr_net_sock_release(r->pipe_sock); free(r->pend); - if (r->sock) scr_net_sock_release(r->sock); + if (r->sock) { + if (r->sock_held) scr_net_sock_hold(r->sock, false); + scr_net_sock_release(r->sock); + } if (r->h2_stream) scr_http_h2_ops->release(r->h2_stream); #ifdef SCR_RC_AUDIT scr_http_live--; @@ -638,6 +643,22 @@ static bool scr_http_req_holding(const ScrHttpReq *r) { return r->paused || (r->gated && !r->flowing); } +/* IncomingMessage's readableHighWaterMark (its socket's, 64 KiB): a held + * request buffers this many body bytes before its socket stops reading. */ +#define SCR_HTTP_REQ_HOLD_BYTES 65536u + +/* Stops the socket's reads while an HTTP/1 server request holds a full + * buffer, and restarts them once it flows, ends or is dumped, as Node's + * readStop when push() returns false. Client responses and h2 compat + * requests keep reading. */ +static void scr_http_req_update_hold(ScrHttpReq *r) { + bool hold = r->sock != NULL && r->status == -1 && !r->http2 && !r->ended && + scr_http_req_holding(r) && r->pend_len >= SCR_HTTP_REQ_HOLD_BYTES; + if (hold == r->sock_held) return; + r->sock_held = hold; + scr_net_sock_hold(r->sock, hold); +} + static void scr_http_req_queue_drain(ScrHttpReq *r) { if (r->drain_queued) return; r->drain_queued = true; @@ -650,6 +671,7 @@ static void scr_http_req_flow(ScrHttpReq *r) { r->paused = false; r->flowing = true; if (r->pend_len > 0 || r->end_pending) scr_http_req_queue_drain(r); + scr_http_req_update_hold(r); } /* A 'data' listener or pipe() starts the flow unless pause() stopped it, @@ -658,9 +680,7 @@ static void scr_http_req_consume(ScrHttpReq *r) { if (r->gated && !r->paused && !r->flowing) scr_http_req_flow(r); } -/* resume(): releases what pause() or the initial paused state held. This - * parser always consumes body bytes from the socket (SEMANTICS.md); held - * bytes buffer in memory. */ +/* resume(): releases what pause() or the initial paused state held. */ void scr_http_req_resume(ScrHttpReq *r) { if (!scr_http_req_holding(r)) return; scr_http_req_flow(r); @@ -676,6 +696,7 @@ static void scr_http_req_dump(ScrHttpReq *r) { r->pend_len = r->pend_cap = 0; r->paused = false; r->flowing = true; + scr_http_req_update_hold(r); } /* pause(): delivery holds until resume() (scr_http_req_deliver buffers; @@ -759,6 +780,7 @@ static void scr_http_req_finish(ScrHttpReq *r, bool fire) { return; } r->ended = true; + scr_http_req_update_hold(r); r->end_consumed = fire && (r->data_ls.n > 0 || r->pipe_res || r->pipe_client || r->pipe_sock); if (fire) scr_net_fire0_this(&r->end_ls, r, SCR_DYNH_HTTP_REQ); scr_net_ls_drop(&r->data_ls); @@ -2305,6 +2327,7 @@ static void scr_http_proto_sweep(void) { req->pend_len = req->pend_cap = 0; if (held_len > 0) scr_http_req_deliver(req, held, held_len); free(held); + scr_http_req_update_hold(req); if (!scr_exc_pending() && !req->paused && req->end_pending) { req->end_pending = false; scr_http_req_finish(req, true); @@ -2758,8 +2781,8 @@ static void scr_http_req_deliver(ScrHttpReq *r, const char *data, size_t n) { if (!r || n == 0 || r->ended) return; r->body_seen = true; if (scr_http_req_holding(r)) { - /* paused: the parser keeps consuming (memory is the buffer — a - * documented bound), delivery waits for the drain */ + /* held: delivery waits for the drain; a full buffer stops the + * socket's reads */ if (r->pend_len + n > r->pend_cap) { size_t cap = r->pend_cap ? r->pend_cap : 4096; while (cap < r->pend_len + n) cap *= 2; @@ -2769,6 +2792,7 @@ static void scr_http_req_deliver(ScrHttpReq *r, const char *data, size_t n) { } memcpy(r->pend + r->pend_len, data, n); r->pend_len += n; + scr_http_req_update_hold(r); return; } bool piped = r->pipe_res != NULL || r->pipe_client != NULL || r->pipe_sock != NULL; diff --git a/packages/runtime/src/scr_js.h b/packages/runtime/src/scr_js.h index 6fa20fdb8..1d874b31e 100644 --- a/packages/runtime/src/scr_js.h +++ b/packages/runtime/src/scr_js.h @@ -195,10 +195,13 @@ bool scr_js_truthy(ScrJsVal value); * index and which keys an object lacks. * * Each property site of compiled dynamic-tier code owns one zeroed, - * thread-local ScrJsIc. The code at the site is fixed: it tests entry 0 - * inline and calls a probe for everything else. Probes never run user - * code: they answer from plain data properties or return a miss (NULL / - * false) with nothing done, and the site then runs its generic path. + * thread-local ScrJsIc with static storage: the reset that frees a + * thread's shapes (at thread exit, and when a library session ends) clears + * every record that holds entries. The code at the site is fixed: it tests + * entry 0 inline and calls a probe for everything else. Probes never run + * user code: they answer from plain data properties or return a miss + * (NULL / false) with nothing done, and the site then runs its generic + * path. * * load entry: shape = receiver shape; target = NULL for an own property * (slot indexes the receiver's entries), else the shape of diff --git a/packages/runtime/src/scr_js_object.c b/packages/runtime/src/scr_js_object.c index 63929ab57..0e0ec0908 100644 --- a/packages/runtime/src/scr_js_object.c +++ b/packages/runtime/src/scr_js_object.c @@ -18,6 +18,7 @@ #include "scr_key.h" #include +#include /* The compiler's inline cache tests read these in place (64-bit targets; * backend/llvm/js-ic.ts). */ @@ -37,10 +38,10 @@ _Static_assert(sizeof(ScrJsIcEntry) == 40 && sizeof(ScrJsIc) == 168, "cache layo * take the generic path and its hash index. */ #define SCR_JS_SHAPE_MAX 64 -/* Shapes live until the thread exits, so a thread creates at most this - * many, holding at most this many key bytes. Once either budget is spent, - * objects whose key sequence has no shape yet stay unshaped; shapes that - * exist keep serving. */ +/* Shapes live until the thread exits or a library session ends, so a thread + * creates at most this many, holding at most this many key bytes. Once + * either budget is spent, objects whose key sequence has no shape yet stay + * unshaped; shapes that exist keep serving. */ #define SCR_JS_SHAPE_BUDGET ((size_t)1 << 16) #define SCR_JS_SHAPE_KEY_BYTES_BUDGET ((size_t)4 << 20) @@ -63,12 +64,25 @@ static SCR_TL size_t scr_js_transition_count; static SCR_TL size_t scr_js_shape_key_bytes; static SCR_TL bool scr_js_shapes_registered; +/* The cache records that hold entries. Entries name shapes and prototype + * objects, so the reset that frees the shapes also clears these records: a + * library's next session runs the same sites with the same records. */ +static SCR_TL ScrJsIc **scr_js_filled_ics; +static SCR_TL size_t scr_js_filled_count; +static SCR_TL size_t scr_js_filled_capacity; + /* The shape word of an object whose key sequence has no shape within the * budgets. No cache entry names it, so its accesses take the probes, * which answer with a miss until the key set changes. */ static ScrJsShape scr_js_unshaped; static void scr_js_shapes_reset(void) { + for (size_t i = 0; i < scr_js_filled_count; i++) + memset(scr_js_filled_ics[i], 0, sizeof *scr_js_filled_ics[i]); + free(scr_js_filled_ics); + scr_js_filled_ics = NULL; + scr_js_filled_count = 0; + scr_js_filled_capacity = 0; while (scr_js_all_shapes) { ScrJsShape *shape = scr_js_all_shapes; scr_js_all_shapes = shape->all_next; @@ -192,6 +206,16 @@ static size_t scr_js_shape_find(const ScrJsShape *shape, const char *key, size_t /* Fills a cache entry: `stale` (a matching entry whose hit failed) when * there is one, else a free or replaceable one. */ static void scr_js_ic_add(ScrJsIc *ic, uint32_t stale, ScrJsIcEntry entry) { + if (ic->count == 0) { + if (scr_js_filled_count == scr_js_filled_capacity) { + size_t capacity = scr_js_filled_capacity ? scr_js_filled_capacity * 2 : 64; + ScrJsIc **grown = realloc(scr_js_filled_ics, capacity * sizeof *grown); + if (!grown) scr_trap("scriptc: out of memory\n"); + scr_js_filled_ics = grown; + scr_js_filled_capacity = capacity; + } + scr_js_filled_ics[scr_js_filled_count++] = ic; + } uint32_t index; if (stale < ic->count) { index = stale; diff --git a/packages/runtime/src/scr_js_object.test.c b/packages/runtime/src/scr_js_object.test.c index a14aa9ff6..6b6934dea 100644 --- a/packages/runtime/src/scr_js_object.test.c +++ b/packages/runtime/src/scr_js_object.test.c @@ -24,11 +24,17 @@ static ScrDyn *object_with(const ScrStr *key, double n) { return object; } +/* Cache records have static storage, as the compiled sites' do: the reset + * at exit clears every record that holds entries. Each check starts from + * an empty record. */ +static ScrJsIc load_ic; +static ScrJsIc store_ic; + /* Whether a load probe answers `object.key` and returns `n`. */ static bool probe_reads(ScrDyn *object, const ScrStr *key, double n) { - ScrJsIc ic; - memset(&ic, 0, sizeof ic); - ScrDyn *value = scr_js_load_probe(object, key, &ic); + ScrJsIc *ic = &load_ic; + memset(ic, 0, sizeof *ic); + ScrDyn *value = scr_js_load_probe(object, key, ic); if (!value) return false; assert(value->kind == SCR_DYN_NUM && value->v.num == n); scr_dyn_release(value); @@ -66,14 +72,13 @@ static void budget_bounds_shapes(size_t key_len, size_t budget) { /* A store probe that would add a new shape leaves the add to the generic * store; one that replaces through an existing shape still runs. */ - ScrJsIc ic; - memset(&ic, 0, sizeof ic); + memset(&store_ic, 0, sizeof store_ic); ScrStr *added = scr_str_new("added", 5); ScrDyn *value = scr_dyn_new_num(8); - assert(!scr_js_store_probe(object, added, value, &ic)); + assert(!scr_js_store_probe(object, added, value, &store_ic)); assert(object->v.obj.len == 1); - memset(&ic, 0, sizeof ic); - assert(scr_js_store_probe(object, first, value, &ic)); + memset(&store_ic, 0, sizeof store_ic); + assert(scr_js_store_probe(object, first, value, &store_ic)); assert(probe_reads(object, first, 8)); scr_dyn_release(value); scr_str_release(added); @@ -90,10 +95,9 @@ static void immortal_objects_stay_unshaped(void) { object->rc = SIZE_MAX; assert(!probe_reads(object, key, 3)); assert(object->v.obj.shape == NULL); - ScrJsIc ic; - memset(&ic, 0, sizeof ic); + memset(&store_ic, 0, sizeof store_ic); ScrDyn *value = scr_dyn_new_num(4); - assert(!scr_js_store_probe(object, key, value, &ic)); + assert(!scr_js_store_probe(object, key, value, &store_ic)); assert(object->v.obj.shape == NULL); scr_dyn_release(value); scr_str_release(key); diff --git a/packages/runtime/src/scr_json.c b/packages/runtime/src/scr_json.c index 28ce987ff..35e1d7674 100644 --- a/packages/runtime/src/scr_json.c +++ b/packages/runtime/src/scr_json.c @@ -1911,7 +1911,10 @@ static ScrDyn *scr_error_ctor_call(ScrClosure *closure, ScrDyn *const *args, siz ScrDyn *scr_dyn_error_constructor(const char *name) { for (size_t i = 0; i < SCR_ERROR_CTORS; i++) { - if (!scr_error_ctor_names[i] || strcmp(scr_error_ctor_names[i], name) != 0) continue; + /* The first byte rules out most names without a call. */ + if (!scr_error_ctor_names[i] || scr_error_ctor_names[i][0] != name[0] || + strcmp(scr_error_ctor_names[i], name) != 0) + continue; if (!scr_error_ctors_registered) { scr_error_ctors_registered = true; scr_atexit(scr_error_ctors_cleanup); @@ -2313,6 +2316,79 @@ ScrDyn *scr_dyn_set_prototype(ScrDyn *object, ScrDyn *prototype) { return scr_dyn_retain(object); } +/* The native record a record capsule refers to, or NULL for any other + * value. */ +static void *scr_dyn_typed_record(const ScrDyn *d) { + if (d->kind != SCR_DYN_TYPED_REF || d->v.typed_ref.type_key_len < 7 || + memcmp(d->v.typed_ref.type_key, "record:", 7) != 0) + return NULL; + return d->v.typed_ref.ptr; +} + +/* The native array an array capsule refers to, or NULL for any other + * value. */ +static ScrArr *scr_dyn_typed_array(const ScrDyn *d) { + if (d->kind != SCR_DYN_TYPED_REF || d->v.typed_ref.type_key_len < 6 || + memcmp(d->v.typed_ref.type_key, "array<", 6) != 0) + return NULL; + return d->v.typed_ref.ptr; +} + +/* The native Map or Set a handle holds, or NULL for any other value. */ +static ScrMap *scr_dyn_native_collection(const ScrDyn *d) { + if (d->kind != SCR_DYN_HANDLE || (d->v.handle.tag != SCR_DYNH_MAP && d->v.handle.tag != SCR_DYNH_SET)) + return NULL; + return d->v.handle.ptr; +} + +/* Whether a capsule's native record or array is frozen. */ +static bool scr_dyn_typed_ref_frozen(const ScrDyn *d) { + void *record = scr_dyn_typed_record(d); + if (record) return scr_rc_frozen(record); + ScrArr *array = scr_dyn_typed_array(d); + return array && scr_arr_frozen(array); +} + +/* Freezes the checked view of a frozen record or array: its fields or + * elements, and every property added through the view. */ +static void scr_dyn_record_view_freeze(ScrDyn *view) { + view->non_extensible = true; + if (view->kind == SCR_DYN_OBJ) { + for (size_t i = 0; i < view->v.obj.len; i++) { + ScrDynEntry *entry = &view->v.obj.entries[i]; + entry->configurable = false; + if (!entry->accessor) entry->writable = false; + } + } else if (view->kind == SCR_DYN_ARR) { + view->v.arr.sealed = true; + view->v.arr.frozen = true; + if (view->v.arr.properties) scr_dyn_release(scr_dyn_freeze_data(view->v.arr.properties)); + } + if (view->symbol_properties) scr_dyn_release(scr_dyn_freeze_data(view->symbol_properties)); +} + +void scr_record_freeze(void *record) { + size_t *rc = record; + if (*rc != SIZE_MAX) *rc |= SCR_RC_FROZEN; +} + +void scr_record_throw_frozen(ScrStr *key, ScrStr *owner, double operation) { + ScrJsonBuf buffer; + scr_jb_init(&buffer); + if (operation == 1) { + scr_jb_puts(&buffer, "Cannot add property "); + scr_jb_write(&buffer, key->data, key->len); + scr_jb_puts(&buffer, ", object is not extensible"); + } else { + scr_jb_puts(&buffer, operation == 2 ? "Cannot delete property '" : "Cannot assign to read only property '"); + scr_jb_write(&buffer, key->data, key->len); + scr_jb_puts(&buffer, operation == 2 ? "' of " : "' of object '"); + scr_jb_write(&buffer, owner->data, owner->len); + if (operation != 2) scr_jb_puts(&buffer, "'"); + } + scr_throw_error(SCR_ERR_TYPE, scr_jb_finish(&buffer)); +} + /* A class instance's integrity state lives in its native layout, which only * statically typed calls can reach (the compiler adds the state to the * hierarchies they name). Through an untyped value the operation throws. */ @@ -2321,7 +2397,9 @@ static bool scr_dyn_class_integrity_fence(const ScrDyn *object, const char *oper ScrJsonBuf buffer; scr_jb_init(&buffer); scr_jb_puts(&buffer, operation); - scr_jb_puts(&buffer, " of a class instance held in an untyped value is not supported yet"); + scr_jb_puts(&buffer, scr_dyn_typed_record(object) ? " of an object held in an untyped value is not supported yet" + : scr_dyn_typed_array(object) ? " of an array held in an untyped value is not supported yet" + : " of a class instance held in an untyped value is not supported yet"); ScrStr *message = scr_jb_finish(&buffer); scr_throw_error_msg_code(SCR_ERR_ERROR, message->data, message->len, "SC2020"); scr_str_release(message); @@ -2369,6 +2447,11 @@ static ScrDyn *scr_dyn_fn_existing_table(const ScrDyn *function) { ScrDyn *scr_dyn_prevent_extensions(ScrDyn *object) { if (scr_dyn_class_reflection_fence(object) || scr_dyn_error_prototype_fence(object)) return NULL; if (object->kind == SCR_DYN_PROXY) { scr_dyn_proxy_unsupported("Object.preventExtensions"); return NULL; } + ScrMap *collection = scr_dyn_native_collection(object); + if (collection) { + scr_map_prevent_extensions(collection); + return scr_dyn_retain(object); + } if (scr_dyn_class_integrity_fence(object, "Object.preventExtensions")) return NULL; ScrDyn *table = scr_dyn_fn_integrity_table(object); if (table) { @@ -2388,6 +2471,8 @@ bool scr_dyn_is_extensible(ScrDyn *object) { if (object->kind == SCR_DYN_PROXY) { scr_dyn_proxy_unsupported("Object.isExtensible"); return false; } bool answer; if (scr_dyn_typed_ref_view_answer(object, scr_dyn_is_extensible_view, &answer)) return answer; + ScrMap *collection = scr_dyn_native_collection(object); + if (collection) return scr_map_is_extensible(collection); if (scr_dyn_plain_fn(object)) { ScrDyn *table = scr_dyn_fn_existing_table(object); answer = !table || !table->non_extensible; @@ -2400,6 +2485,11 @@ bool scr_dyn_is_extensible(ScrDyn *object) { ScrDyn *scr_dyn_seal(ScrDyn *object) { if (scr_dyn_class_reflection_fence(object) || scr_dyn_error_prototype_fence(object)) return NULL; if (object->kind == SCR_DYN_PROXY) { scr_dyn_proxy_unsupported("Object.seal"); return NULL; } + ScrMap *collection = scr_dyn_native_collection(object); + if (collection) { + scr_map_prevent_extensions(collection); + return scr_dyn_retain(object); + } if (scr_dyn_class_integrity_fence(object, "Object.seal")) return NULL; if (object->symbol_properties) scr_dyn_release(scr_dyn_seal(object->symbol_properties)); ScrDyn *table = scr_dyn_fn_integrity_table(object); @@ -2424,6 +2514,8 @@ bool scr_dyn_is_sealed(const ScrDyn *object) { if (object->kind == SCR_DYN_PROXY) { scr_dyn_proxy_unsupported("Object.isSealed"); return false; } bool answer; if (scr_dyn_typed_ref_view_answer((ScrDyn *)object, scr_dyn_is_sealed, &answer)) return answer; + ScrMap *collection = scr_dyn_native_collection(object); + if (collection) return !scr_map_is_extensible(collection); if (object->symbol_properties && !scr_dyn_is_sealed(object->symbol_properties)) return false; if (scr_dyn_plain_fn(object)) { ScrDyn *table = scr_dyn_fn_existing_table(object); @@ -3217,6 +3309,13 @@ void scr_record_source_spread(void *dst, const void *src, const char *const *nam if (scr_record_source_declared(&probe, entry->key, entry->key_len)) scr_dyn_obj_set(view, entry->key, entry->key_len, scr_dyn_retain(scr_dyn_undefined())); } + /* The keys src lacks follow in the table's order. The view lists them + * too, so an integer key among them comes first in its own-key order, as + * the literal creates it. */ + for (size_t i = 0; i < count; i++) { + if (scr_dyn_obj_get(view, names[i], lens[i])) continue; + scr_dyn_obj_set(view, names[i], lens[i], scr_dyn_retain(scr_dyn_undefined())); + } ScrRecordSource *old = scr_record_source_find(dst); if (old) scr_record_source_drop(old); probe.view = view; @@ -3466,7 +3565,7 @@ static ScrDyn *scr_dyn_typed_ref_new( return scr_dyn_retain(value); } } - if (type_key_len >= (size_t)1 << 28) scr_trap("scriptc: typed reference key too long\n"); + if (type_key_len >= (size_t)1 << 27) scr_trap("scriptc: typed reference key too long\n"); ScrDyn *d = scr_dyn_alloc(SCR_DYN_TYPED_REF); d->v.typed_ref.ptr = retain(ptr); d->v.typed_ref.retain = retain; @@ -3489,6 +3588,7 @@ static ScrDyn *scr_dyn_typed_ref_new( d->v.typed_ref.observed = observed; d->v.typed_ref.record = false; d->v.typed_ref.klass = false; + d->v.typed_ref.bag_class = false; scr_typed_ref_slots[slot] = d; /* the empty slot that ended the probe */ scr_typed_ref_count++; return d; @@ -3680,12 +3780,14 @@ ScrDyn *scr_dyn_typed_ref_materialize(const ScrDyn *d) { fresh->rc = fresh_rc; scr_dyn_release(fresh); } + if (scr_dyn_typed_ref_frozen(capsule)) scr_dyn_record_view_freeze(capsule->v.typed_ref.materialized); return scr_dyn_retain(capsule->v.typed_ref.materialized); } void scr_dyn_typed_ref_commit(ScrDyn *d) { + /* A frozen record's or array's view cannot have changed its contents. */ if (d && d->kind == SCR_DYN_TYPED_REF && d->v.typed_ref.commit && - d->v.typed_ref.materialized) { + d->v.typed_ref.materialized && !scr_dyn_typed_ref_frozen(d)) { d->v.typed_ref.commit(d->v.typed_ref.ptr, d->v.typed_ref.materialized); } @@ -3704,6 +3806,7 @@ void scr_dyn_typed_ref_bind_array(ScrDyn *d, const ScrDynTypedArrayOps *ops) { d->v.typed_ref.array = ops; d->v.typed_ref.record = false; d->v.typed_ref.klass = false; + d->v.typed_ref.bag_class = false; } } @@ -3722,6 +3825,7 @@ void scr_dyn_typed_ref_bind_record(ScrDyn *d, const ScrDynTypedRecordOps *ops) { d->v.typed_ref.array = (const ScrDynTypedArrayOps *)(const void *)ops; d->v.typed_ref.record = true; d->v.typed_ref.klass = false; + d->v.typed_ref.bag_class = false; } } @@ -3730,9 +3834,75 @@ void scr_dyn_typed_ref_bind_class(ScrDyn *d, const ScrDynTypedClassOps *ops) { d->v.typed_ref.array = (const ScrDynTypedArrayOps *)(const void *)ops; d->v.typed_ref.record = false; d->v.typed_ref.klass = true; + d->v.typed_ref.bag_class = false; } } +void scr_dyn_typed_ref_bind_bag_class(ScrDyn *d, const ScrDynTypedBagClassOps *ops) { + if (d && d->kind == SCR_DYN_TYPED_REF) { + d->v.typed_ref.array = (const ScrDynTypedArrayOps *)(const void *)ops; + d->v.typed_ref.record = false; + d->v.typed_ref.klass = false; + d->v.typed_ref.bag_class = true; + } +} + +/* The view's own string keys are the bag's (ScrDynTypedBagClassOps). An + * own data property answers as the view would hold it: a declared field + * from its native slot (the materializer's ordinary write of that value, + * which a read-only or accessor entry would refuse, so those take the + * view), another property with the bag's value. A key the bag lacks is + * looked up on the prototype the view has: the cached view's, which keeps + * prototype changes made through the view, or for a capsule without one, + * the prototype a fresh view takes. The walk is scr_dyn_obj_read_receiver's + * from the view's prototype on, answering only when it meets an ordinary + * data property before anything that needs more (an accessor, a function, + * array or capsule link, Object.prototype). */ +ScrDyn *scr_dyn_typed_ref_bag_method(const ScrDyn *d, const char *key, size_t key_len) { + if (d->kind != SCR_DYN_TYPED_REF || !d->v.typed_ref.bag_class || d->prototype) return NULL; + const ScrDynTypedBagClassOps *ops = (const ScrDynTypedBagClassOps *)(const void *)d->v.typed_ref.array; + void *instance = d->v.typed_ref.ptr; + const ScrDyn *bag = ops->bag(instance); + if (bag && bag->kind != SCR_DYN_UNDEF) { + if (bag->kind != SCR_DYN_OBJ) return NULL; + const ScrDynEntry *own = scr_dyn_find_entry(bag, key, key_len); + if (own) { + for (size_t i = 0; i < ops->count; i++) { + const ScrStr *name = ops->names[i]; + if (name->len != key_len || memcmp(name->data, key, key_len) != 0) continue; + return own->accessor || !own->writable ? NULL : ops->field(instance, i); + } + return own->accessor ? NULL : scr_dyn_retain(own->value); + } + } + ScrDyn *prototype; + const ScrDyn *cached = d->v.typed_ref.materialized; + if (cached) { + if (cached->kind != SCR_DYN_OBJ || !cached->prototype) return NULL; + prototype = scr_dyn_retain(cached->prototype); + } else { + prototype = ops->prototype(instance); + if (scr_exc_pending()) { + scr_dyn_release(prototype); + return NULL; + } + if (!prototype) return NULL; + } + ScrDyn *found = NULL; + for (const ScrDyn *current = prototype; current; current = current->prototype) { + if (current == scr_builtin_object_prototype || current->kind == SCR_DYN_FUNC || + current->kind == SCR_DYN_ARR || current->kind == SCR_DYN_TYPED_REF) + break; + if (current->kind != SCR_DYN_OBJ) continue; + ScrDynEntry *entry = scr_dyn_find_entry(current, key, key_len); + if (!entry) continue; + if (!entry->accessor) found = scr_dyn_retain(entry->value); + break; + } + scr_dyn_release(prototype); + return found; +} + static const ScrDynTypedClassOps *scr_typed_ref_class_ops(const ScrDyn *d) { if (!d->v.typed_ref.klass) return NULL; const ScrDynTypedClassOps *ops = (const ScrDynTypedClassOps *)(const void *)d->v.typed_ref.array; @@ -3924,6 +4094,8 @@ static bool scr_typed_ref_array_has_own(const ScrDyn *d, const ScrStr *key) { static bool scr_typed_ref_array_set(ScrDyn *d, const ScrStr *key, const ScrDyn *value) { const ScrDynTypedArrayOps *ops = scr_dyn_typed_ref_array_ops(d); if (!ops || !ops->set) return false; + /* A frozen array's view refuses the write. */ + if (scr_arr_frozen(d->v.typed_ref.ptr)) return false; /* Numeric properties live in the native array's property table, which * holds values of the element type. */ double number; @@ -5007,19 +5179,103 @@ ScrDyn *scr_dyn_promise_key_get(const ScrDyn *d, const ScrStr *k) { return method ? method : scr_dyn_retain(scr_dyn_undefined()); } +/* ── handle methods as values (ScrDynHandleMethod) ────────────────── + * One function per (tag, method), as Node's prototype methods are shared + * by every instance. The function calls invoke on its `this` when that is + * a handle of its tag, and otherwise raises the class's ERR_INVALID_THIS + * TypeError (or answers a promise rejected with it), as the WebIDL brand + * checks do. closure->fn holds the method entry, caps[0] the tag. */ +static SCR_TL ScrDyn **scr_dynh_method_values[SCR_DYNH_CAPACITY]; +static SCR_TL bool scr_dynh_method_values_registered; + +static void scr_dynh_method_values_cleanup(void) { + for (size_t tag = 0; tag < SCR_DYNH_CAPACITY; tag++) { + ScrDyn **values = scr_dynh_method_values[tag]; + if (!values) continue; + scr_dynh_method_values[tag] = NULL; + const ScrDynHandleMethod *methods = scr_dynh_ops[tag]->methods; + for (size_t i = 0; methods[i].name; i++) + if (values[i]) scr_dyn_release(values[i]); + free(values); + } +} + +static ScrDyn *scr_dynh_method_call(ScrClosure *closure, ScrDyn *const *args, size_t argc) { + const ScrDynHandleMethod *method = closure->fn; + ScrDynHandleTag tag = (ScrDynHandleTag)scr_box_get_f64(closure->caps[0]); + const ScrDynHandleOps *ops = scr_dyn_handle_ops(tag); + ScrDyn *self = scr_dyn_this_get(); + ScrDyn *result; + if (self->kind == SCR_DYN_HANDLE && self->v.handle.tag == tag) { + result = ops->invoke(self->v.handle.ptr, self, method->name, args, argc, method->name); + } else { + const char *message = method->invalid_this ? method->invalid_this + : ops->invalid_this ? ops->invalid_this + : "Illegal invocation"; + scr_throw_error_msg_code(SCR_ERR_TYPE, message, strlen(message), "ERR_INVALID_THIS"); + result = method->rejects && ops->rejected ? ops->rejected() : NULL; + } + scr_dyn_release(self); + return result; +} + +/* The method `key` of the handle's class as a function value (+1), or NULL + * when the class has no such method. */ +static ScrDyn *scr_dynh_method_value(ScrDynHandleTag tag, const ScrDynHandleOps *ops, const char *key, size_t len) { + const ScrDynHandleMethod *methods = ops->methods; + if (!methods || !ops->invoke) return NULL; + for (size_t i = 0; methods[i].name; i++) { + if (strlen(methods[i].name) != len || memcmp(methods[i].name, key, len) != 0) continue; + if (!scr_dynh_method_values[tag]) { + size_t count = i + 1; + while (methods[count].name) count++; + scr_dynh_method_values[tag] = calloc(count, sizeof(ScrDyn *)); + if (!scr_dynh_method_values[tag]) scr_trap("scriptc: out of memory\n"); + if (!scr_dynh_method_values_registered) { + scr_dynh_method_values_registered = true; + scr_atexit(scr_dynh_method_values_cleanup); + } + } + ScrDyn **slot = &scr_dynh_method_values[tag][i]; + if (!*slot) { + ScrClosure *closure = scr_closure_new((void *)&methods[i], 1); + closure->caps[0] = scr_box_new(SCR_BOX_F64); + scr_box_set_f64(closure->caps[0], (double)tag); + *slot = scr_dyn_new_func(closure, scr_dynh_method_call, methods[i].length, "", methods[i].name); + } + return scr_dyn_retain(*slot); + } + return NULL; +} + ScrDyn *scr_dyn_handle_key_get(const ScrDyn *d, const ScrStr *k) { - const ScrDynHandleOps *ops = scr_dyn_handle_ops(d->v.handle.tag); + ScrDynHandleTag tag = d->v.handle.tag; + const ScrDynHandleOps *ops = scr_dyn_handle_ops(tag); ScrDyn *r = ops->get(d->v.handle.ptr, k->data, k->len); if (scr_exc_pending()) { scr_dyn_release(r); return NULL; } + if (!r) r = scr_dynh_method_value(tag, ops, k->data, k->len); /* Unmodeled names answer undefined — the checked-dynamic tree's own-property stance * (real-but-unmodeled members throw "not supported yet" inside ops->get instead; * SEMANTICS.md documents the remainder). */ return r ? r : scr_dyn_retain(scr_dyn_undefined()); } +/* `key in handle`: the class's methods, and the properties its get models. */ +static bool scr_dyn_handle_in(const ScrDyn *d, const ScrStr *k) { + const ScrDynHandleOps *ops = scr_dyn_handle_ops(d->v.handle.tag); + const ScrDynHandleMethod *methods = ops->invoke ? ops->methods : NULL; + for (size_t i = 0; methods && methods[i].name; i++) + if (strlen(methods[i].name) == k->len && memcmp(methods[i].name, k->data, k->len) == 0) return true; + if (!methods) return false; + ScrDyn *value = ops->get(d->v.handle.ptr, k->data, k->len); + if (!value) return scr_dyn_object_proto_name(k); + scr_dyn_release(value); + return true; +} + /* ── the ambient receiver (scr_runtime.h's design note) ─────────────── * A strictly nested push/pop stack: firing sites bind the owner around * each listener call, dyn OBJ method dispatch binds the object, and the @@ -5111,6 +5367,8 @@ ScrDyn *scr_dyn_this_get(void) { scr_dyn_typed_ref_bind_record(e->dv, (const ScrDynTypedRecordOps *)r->ops); else if (r->view == 3) scr_dyn_typed_ref_bind_class(e->dv, (const ScrDynTypedClassOps *)r->ops); + else if (r->view == 4) + scr_dyn_typed_ref_bind_bag_class(e->dv, (const ScrDynTypedBagClassOps *)r->ops); } if (e->dv) return scr_dyn_retain(e->dv); /* An uninstalled tag never binds: a unit that fires without its dyn @@ -5596,6 +5854,23 @@ static void scr_native_iterator_release(void *ptr) { * next() calls still receive a fresh, independently mutable result object. */ static ScrDyn *scr_native_iterator_value(ScrNativeIterator *iterator, bool *done_out) { ScrDyn *source = iterator->source; + /* An own element of an untyped array, or its end: no getter can run, so + * the general path below is not needed. */ + if (source && source->kind == SCR_DYN_ARR && iterator->kind == 0 && !iterator->array_like && + iterator->selection == 0) { + if (iterator->index >= (double)source->v.arr.len) { + iterator->source = NULL; + scr_dyn_release(source); + *done_out = true; + return scr_dyn_retain(scr_dyn_undefined()); + } + if (scr_dyn_arr_has_index(source, (size_t)iterator->index)) { + size_t index = (size_t)iterator->index; + iterator->index++; + *done_out = false; + return scr_dyn_retain(source->v.arr.items[index]); + } + } /* An inherited indexed getter can advance this same iterator recursively. * Keep the source alive even if that nested call exhausts the iterator. */ if (source) scr_dyn_retain(source); @@ -6740,6 +7015,12 @@ ScrDyn *scr_dyn_iterator(const ScrDyn *source, const ScrStr *spell) { scr_dyn_release(method); return scr_dyn_not_iterable(source, spell); } + /* An array whose iterator method is the builtin values(): what calling + * it with the array as `this` returns, without the call. */ + if (source->kind == SCR_DYN_ARR && method == scr_iterator_functions[1]) { + scr_dyn_release(method); + return scr_native_iterator_new((ScrDyn *)source); + } ScrDyn *iterator = scr_iterator_call(method, source, "value[Symbol.iterator]"); scr_dyn_release(method); if (!iterator) return NULL; @@ -9228,6 +9509,7 @@ bool scr_dyn_in(const ScrDyn *v, const ScrStr *key) { } if (v->kind == SCR_DYN_PROXY) return scr_dyn_proxy_has(v, key); if (v->kind == SCR_DYN_BYTES) return scr_dyn_bytes_in(v, key); + if (v->kind == SCR_DYN_HANDLE) return scr_dyn_handle_in(v, key); if (scr_dyn_has_key(v, key)) return true; if (scr_exc_pending()) return false; if (v->kind == SCR_DYN_OBJ) return scr_dyn_builtin_inherited(v, key); @@ -9245,10 +9527,32 @@ bool scr_dyn_in(const ScrDyn *v, const ScrStr *key) { return false; } +/* Node's TypeErrors for assigning a read-only property and deleting a + * non-configurable one of a checked object: the object reads as + * #, or [object Object] without a prototype. */ +static void scr_dyn_throw_property(const ScrDyn *object, const char *key, size_t key_len, bool deleting) { + const char *name = scr_dyn_class_name(object); + ScrJsonBuf buffer; + scr_jb_init(&buffer); + scr_jb_puts(&buffer, deleting ? "Cannot delete property '" : "Cannot assign to read only property '"); + scr_jb_write(&buffer, key, key_len); + scr_jb_puts(&buffer, deleting ? "' of " : "' of object '"); + if (object->null_proto) scr_jb_puts(&buffer, "[object Object]"); + else { + scr_jb_puts(&buffer, "#<"); + scr_jb_puts(&buffer, name ? name : "Object"); + scr_jb_puts(&buffer, ">"); + } + if (!deleting) scr_jb_puts(&buffer, "'"); + scr_throw_error(SCR_ERR_TYPE, scr_jb_finish(&buffer)); +} + +/* One lookup of the own property decides the assignment: an own property + * is written (or runs its setter, or refuses the write); an absent one + * consults the prototype chain and is then added. */ static void scr_dyn_object_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value, const ScrDyn *receiver) { - for (size_t i = 0; i < recv->v.obj.len; i++) { - ScrDynEntry *entry = &recv->v.obj.entries[i]; - if (entry->key_len != key->len || memcmp(entry->key, key->data, key->len) != 0) continue; + ScrDynEntry *entry = scr_dyn_entry(recv, key); + if (entry) { if (entry->accessor && entry->setter) { ScrDyn *args[] = {value}; ScrDyn *setter = scr_dyn_retain(entry->setter); @@ -9260,12 +9564,17 @@ static void scr_dyn_object_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value, con return; } if (entry->accessor || !entry->writable) { - static const char msg[] = "Cannot assign to read only property"; - scr_throw_error_msg(SCR_ERR_TYPE, msg, sizeof msg - 1); + scr_dyn_throw_property(receiver, key->data, key->len, false); return; } + scr_dyn_region_store(recv, value); + ScrDyn *old = entry->value; + entry->value = scr_dyn_retain(value); + scr_dyn_release(old); + scr_error_sync_cause(recv, key); + return; } - if (!scr_dyn_entry(recv, key)) { + { for (const ScrDyn *current = scr_dyn_proto_next(recv); current;) { ScrDyn *table = current->kind == SCR_DYN_FUNC ? scr_dyn_fn_properties(current) : NULL; const ScrDyn *object = table ? table : current; @@ -9288,8 +9597,7 @@ static void scr_dyn_object_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value, con return; } if (inherited->accessor || !inherited->writable) { - static const char message[] = "Cannot assign to read only property"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); + scr_dyn_throw_property(receiver, key->data, key->len, false); scr_dyn_release(table); return; } @@ -9297,7 +9605,7 @@ static void scr_dyn_object_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value, con break; } } - if (recv->non_extensible && !scr_dyn_entry(recv, key)) { + if (recv->non_extensible) { ScrJsonBuf buffer; scr_jb_init(&buffer); scr_jb_puts(&buffer, "Cannot add property "); @@ -9306,9 +9614,8 @@ static void scr_dyn_object_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value, con scr_throw_error(SCR_ERR_TYPE, scr_jb_finish(&buffer)); return; } - bool created = scr_dyn_entry(recv, key) == NULL; scr_dyn_obj_set(recv, key->data, key->len, scr_dyn_retain(value)); - if (created && key->len == 5 && memcmp(key->data, "stack", 5) == 0 && + if (key->len == 5 && memcmp(key->data, "stack", 5) == 0 && scr_errdyn_find(recv, false) != SIZE_MAX) { /* An error's own `stack` exists from construction as a non-enumerable * data property, so assigning it adds no enumerable key. */ @@ -9369,8 +9676,7 @@ void scr_dyn_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value) { if (recv->kind == SCR_DYN_ARR) { if (key->len == 6 && memcmp(key->data, "length", 6) == 0) { if (recv->v.arr.frozen) { - static const char message[] = "Cannot assign to read only property"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); + scr_array_frozen_error(3, recv->v.arr.len, 0, 0, 0); return; } double number; @@ -9389,8 +9695,7 @@ void scr_dyn_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value) { while (recv->v.arr.len < length) scr_dyn_arr_push_hole(recv); while (recv->v.arr.len > length) { if (recv->v.arr.sealed && scr_dyn_arr_has_index(recv, recv->v.arr.len - 1)) { - static const char message[] = "Cannot delete non-configurable property"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); + scr_array_frozen_error(5, recv->v.arr.len, (double)(recv->v.arr.len - 1), 0, 0); return; } scr_dyn_release(recv->v.arr.items[--recv->v.arr.len]); @@ -9408,14 +9713,13 @@ void scr_dyn_key_set(ScrDyn *recv, ScrStr *key, ScrDyn *value) { } if (is_index && idx < 4294967295ULL) { if (scr_dyn_array_prototype_element_fence(recv)) return; - if (recv->v.arr.frozen && idx < recv->v.arr.len) { - static const char message[] = "Cannot assign to read only property"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); + if (recv->v.arr.frozen && idx < recv->v.arr.len && scr_dyn_arr_has_index(recv, idx)) { + scr_array_frozen_error(0, recv->v.arr.len, (double)idx, 0, 0); return; } if (recv->non_extensible && !scr_dyn_arr_has_index(recv, idx)) { - static const char message[] = "Cannot add property, object is not extensible"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); + /* An absent index, a hole included, is a property to add. */ + scr_array_frozen_error(0, 0, (double)idx, 0, 0); return; } if (idx > recv->v.arr.len + 1000000) { @@ -11244,10 +11548,7 @@ void scr_dyn_key_delete(ScrDyn *recv, const ScrStr *key, bool strict) { if (recv->kind == SCR_DYN_OBJ) { ScrDynEntry *entry = scr_dyn_entry(recv, key); if (entry && !entry->configurable) { - if (strict) { - static const char message[] = "Cannot delete non-configurable property"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); - } + if (strict) scr_dyn_throw_property(recv, key->data, key->len, true); return; } scr_json_delete_member(recv, key); @@ -11259,10 +11560,7 @@ void scr_dyn_key_delete(ScrDyn *recv, const ScrStr *key, bool strict) { for (size_t i = 0; i < key->len; i++) index = index * 10 + (size_t)(key->data[i] - '0'); if (!scr_dyn_arr_has_index(recv, index)) return; if (recv->v.arr.sealed) { - if (strict) { - static const char message[] = "Cannot delete non-configurable property"; - scr_throw_error_msg(SCR_ERR_TYPE, message, sizeof message - 1); - } + if (strict) scr_array_frozen_error(5, recv->v.arr.len, (double)index, 0, 0); return; } if (!recv->v.arr.presence) { @@ -13774,19 +14072,74 @@ ScrDyn *scr_dyn_object_rest(const ScrDyn *source, const ScrDyn *excluded) { /* Compiler-owned native class views retain every property and its flags, * including nonenumerable data, without invoking accessors. */ -ScrDyn *scr_dyn_copy_property_descriptors(ScrDyn *target, const ScrDyn *src) { - if (!src || src->kind == SCR_DYN_UNDEF) return scr_dyn_retain(target); - ScrDyn *keys = scr_dyn_obj_own_keys(src); - if (!keys) return NULL; - for (size_t i = 0; i < keys->v.arr.len; i++) { - ScrDyn *key = keys->v.arr.items[i]; - ScrDyn *descriptor = scr_dyn_own_descriptor(src, key->v.str); +static void scr_dyn_copy_property_descriptor(ScrDyn *target, const ScrDyn *src, + const ScrDynEntry *from) { + /* Exactly scr_dyn_define_property(target, key, own descriptor of key) + * when `target` has no such property and accepts new ones: the entry + * takes the source's attributes, value or accessor pair. */ + ScrDynEntry *prior = scr_dyn_find_entry(target, from->key, from->key_len); + if (prior || target->non_extensible) { + ScrStr *name = scr_str_new(from->key, from->key_len); + ScrDyn *key = scr_dyn_new_str(name); + ScrDyn *descriptor = scr_dyn_own_descriptor(src, name); ScrDyn *result = descriptor ? scr_dyn_define_property(target, key, descriptor) : NULL; scr_dyn_release(descriptor); scr_dyn_release(result); - if (scr_exc_pending()) break; + scr_dyn_release(key); + scr_str_release(name); + return; + } + /* The names whose definition may move into a native Error's slots + * (scr_error_sync_slots). */ + static const char *const slots[] = {"cause", "stack", "name", "message", "code"}; + bool slot = false; + for (size_t i = 0; i < sizeof slots / sizeof *slots && !slot; i++) + slot = from->key_len == strlen(slots[i]) && memcmp(from->key, slots[i], from->key_len) == 0; + bool accessor = from->accessor, writable = !accessor && from->writable; + bool enumerable = from->enumerable, configurable = from->configurable; + ScrDyn *value = scr_dyn_retain(accessor ? scr_dyn_undefined() : from->value); + ScrDyn *getter = accessor && from->getter ? scr_dyn_retain(from->getter) : NULL; + ScrDyn *setter = accessor && from->setter ? scr_dyn_retain(from->setter) : NULL; + ScrStr *name = slot ? scr_str_new(from->key, from->key_len) : NULL; + scr_dyn_obj_set(target, from->key, from->key_len, value); + ScrDynEntry *entry = &target->v.obj.entries[target->v.obj.len - 1]; + entry->accessor = accessor; + entry->getter = getter; + entry->setter = setter; + entry->writable = writable; + entry->enumerable = enumerable; + entry->configurable = configurable; + if (name) { + scr_error_sync_cause(target, name); + scr_str_release(name); + } +} + +ScrDyn *scr_dyn_copy_property_descriptors(ScrDyn *target, const ScrDyn *src) { + if (!src || src->kind == SCR_DYN_UNDEF) return scr_dyn_retain(target); + if (src->kind == SCR_DYN_OBJ && target->kind == SCR_DYN_OBJ && src != target && + src != scr_builtin_object_prototype && target != scr_builtin_error_prototype) { + /* Plain objects on both sides: no fence, proxy trap or accessor runs, + * so each property is copied from its entry without building the key + * and descriptor objects the general path goes through. */ + if (src->v.obj.len) scr_dyn_region_store(target, NULL); + size_t *order = scr_dyn_obj_key_order(src); + for (size_t i = 0; i < src->v.obj.len && !scr_exc_pending(); i++) + scr_dyn_copy_property_descriptor(target, src, &src->v.obj.entries[order ? order[i] : i]); + free(order); + } else { + ScrDyn *keys = scr_dyn_obj_own_keys(src); + if (!keys) return NULL; + for (size_t i = 0; i < keys->v.arr.len; i++) { + ScrDyn *key = keys->v.arr.items[i]; + ScrDyn *descriptor = scr_dyn_own_descriptor(src, key->v.str); + ScrDyn *result = descriptor ? scr_dyn_define_property(target, key, descriptor) : NULL; + scr_dyn_release(descriptor); + scr_dyn_release(result); + if (scr_exc_pending()) break; + } + scr_dyn_release(keys); } - scr_dyn_release(keys); if (!scr_exc_pending()) { ScrDyn *symbols = scr_dyn_get_own_property_symbols(src); if (!symbols) return NULL; @@ -14930,6 +15283,7 @@ static int scr_weak_key(const ScrDyn *key, void **ptr, unsigned *kind) { key->v.handle.tag != SCR_DYNH_WEAK_SET && key->v.handle.tag != SCR_DYNH_FINALIZATION_REGISTRY && key->v.handle.tag != SCR_DYNH_STDIO && key->v.handle.tag != SCR_DYNH_FETCH_REQUEST && key->v.handle.tag != SCR_DYNH_FETCH_RESPONSE && + key->v.handle.tag != SCR_DYNH_WEB_STREAM && key->v.handle.tag != SCR_DYNH_MAP && key->v.handle.tag != SCR_DYNH_SET && key->v.handle.tag != SCR_DYNH_NET_SOCKET && key->v.handle.tag != SCR_DYNH_NET_SERVER) return -1; *ptr = key->v.handle.ptr; *kind = 4 + key->v.handle.tag; return 1; @@ -15697,8 +16051,29 @@ ScrStr *scr_dyn_to_string_method(const ScrDyn *d, const ScrStr *enc, const ScrSt return scr_dyn_to_string_method_args(d, enc, what, NULL, 0); } +/* Freezing a record or array capsule freezes the native value, whose + * emitted writes then refuse to change it, and its checked view. A Map or + * Set handle only stops being extensible: its entries stay writable. Answers + * NULL for every other value. */ +static ScrDyn *scr_dyn_freeze_native(ScrDyn *value) { + void *record = scr_dyn_typed_record(value); + ScrArr *array = record ? NULL : scr_dyn_typed_array(value); + ScrMap *collection = scr_dyn_native_collection(value); + if (collection) { + scr_map_prevent_extensions(collection); + return scr_dyn_retain(value); + } + if (!record && !array) return NULL; + if (record) scr_record_freeze(record); + else scr_arr_freeze(array); + if (value->v.typed_ref.materialized) scr_dyn_record_view_freeze(value->v.typed_ref.materialized); + return scr_dyn_retain(value); +} + ScrDyn *scr_dyn_freeze(ScrDyn *value) { if (scr_dyn_class_reflection_fence(value) || scr_dyn_error_prototype_fence(value)) return NULL; + ScrDyn *native = scr_dyn_freeze_native(value); + if (native) return native; if (scr_dyn_class_integrity_fence(value, "Object.freeze")) return NULL; ScrDyn *table = scr_dyn_fn_integrity_table(value); if (!table) return scr_dyn_freeze_data(value); @@ -15710,6 +16085,8 @@ ScrDyn *scr_dyn_freeze(ScrDyn *value) { ScrDyn *scr_dyn_freeze_data(ScrDyn *value) { if (scr_dyn_class_reflection_fence(value) || scr_dyn_error_prototype_fence(value)) return NULL; + ScrDyn *native = scr_dyn_freeze_native(value); + if (native) return native; if (scr_dyn_class_integrity_fence(value, "Object.freeze")) return NULL; if (value->symbol_properties) scr_dyn_release(scr_dyn_freeze_data(value->symbol_properties)); if (value->kind == SCR_DYN_ARR) { @@ -15740,6 +16117,8 @@ bool scr_dyn_is_frozen(const ScrDyn *value) { if (scr_dyn_class_reflection_fence(value)) return false; bool answer; if (scr_dyn_typed_ref_view_answer((ScrDyn *)value, scr_dyn_is_frozen, &answer)) return answer; + ScrMap *collection = scr_dyn_native_collection(value); + if (collection) return !scr_map_is_extensible(collection); if (value->symbol_properties && !scr_dyn_is_frozen(value->symbol_properties)) return false; if (scr_dyn_plain_fn(value)) { ScrDyn *table = scr_dyn_fn_existing_table(value); @@ -16247,7 +16626,6 @@ void scr_dyn_decl_become(ScrDyn *n, const ScrDyn *capsule) { * does not have it yet. Answers false, changing nothing, for any other bag, * whose creation takes the general defineProperty path. Calls only * exported functions. */ -bool scr_dyn_bag_create_field(ScrDyn *bag, const ScrStr *key); bool scr_dyn_bag_create_field(ScrDyn *bag, const ScrStr *key) { if (!bag || bag->kind != SCR_DYN_OBJ || bag->non_extensible || bag->v.obj.source_access) return false; @@ -16259,7 +16637,6 @@ bool scr_dyn_bag_create_field(ScrDyn *bag, const ScrStr *key) { /* Whether a value is a capsule of a compiled class instance (its type key * names an object type): the class tables' fast exit for every other value, * including record and array capsules. Calls only libc. */ -bool scr_dyn_is_class_capsule(const ScrDyn *d); bool scr_dyn_is_class_capsule(const ScrDyn *d) { return d && d->kind == SCR_DYN_TYPED_REF && d->v.typed_ref.type_key_len > 7 && memcmp(d->v.typed_ref.type_key, "object:", 7) == 0; diff --git a/packages/runtime/src/scr_json.test.c b/packages/runtime/src/scr_json.test.c index 736eca540..a28759876 100644 --- a/packages/runtime/src/scr_json.test.c +++ b/packages/runtime/src/scr_json.test.c @@ -802,8 +802,21 @@ static void json_schema_union_members(void) { assert(test_opts_live == 0); } +/* Freezing a record sets the frozen bit beside its count; an immortal + * record keeps its word, which does not read as frozen. */ +static void record_frozen_bit(void) { + size_t record[2] = {3, 0}; + assert(!scr_rc_frozen(record)); + scr_record_freeze(record); + assert(scr_rc_frozen(record) && SCR_RC_REFS(record) == 3); + size_t immortal[2] = {SIZE_MAX, 0}; + scr_record_freeze(immortal); + assert(immortal[0] == SIZE_MAX && !scr_rc_frozen(immortal)); +} + int main(void) { json_string_boundaries(); + record_frozen_bit(); json_indent_ownership(); scr_init(); checked_storage(); diff --git a/packages/runtime/src/scr_map.c b/packages/runtime/src/scr_map.c index 1a6a3fc00..1cb77b0f6 100644 --- a/packages/runtime/src/scr_map.c +++ b/packages/runtime/src/scr_map.c @@ -988,6 +988,12 @@ double scr_map_size(const ScrMap *m) { return (double)m->nlive; } * table. Regrowth is amortized against the insertions that need it. * A map without entries has empty tables already: clearing it does no * work and keeps its storage for the next fill. */ +void scr_map_prevent_extensions(ScrMap *m) { + if (m->rc != SIZE_MAX) m->flags |= SCR_MAP_NOT_EXTENSIBLE; +} + +bool scr_map_is_extensible(const ScrMap *m) { return (m->flags & SCR_MAP_NOT_EXTENSIBLE) == 0; } + void scr_map_clear(ScrMap *m) { size_t walked = m->nentries; if (walked == 0) return; diff --git a/packages/runtime/src/scr_map.h b/packages/runtime/src/scr_map.h index d1ca387bf..a581cc1c7 100644 --- a/packages/runtime/src/scr_map.h +++ b/packages/runtime/src/scr_map.h @@ -116,6 +116,10 @@ typedef struct ScrMapOps { * whose object has a null prototype (Object.groupBy results). Compiled * code sets and tests the byte; the runtime never copies it. */ #define SCR_MAP_NULL_PROTO 1u +/* Object.freeze, Object.seal or Object.preventExtensions of a Map or Set: + * it has no own properties, so it is then frozen, sealed and not + * extensible at once. Its entries stay writable, as in Node. */ +#define SCR_MAP_NOT_EXTENSIBLE 2u typedef struct ScrMap { size_t rc; /* SIZE_MAX = immortal (unused for maps; kept per convention) */ @@ -272,6 +276,10 @@ void scr_map_release_v(void *m); void scr_map_trace_v(void *m, ScrTraceVisit visit, void *ctx); double scr_map_size(const ScrMap *m); /* live entries (Map.size) */ +/* Object.freeze, seal and preventExtensions of a Map or Set + * (SCR_MAP_NOT_EXTENSIBLE); immortal maps are left as they are. */ +void scr_map_prevent_extensions(ScrMap *m); +bool scr_map_is_extensible(const ScrMap *m); void scr_map_clear(ScrMap *m); bool scr_map_has_f64(const ScrMap *m, double key); diff --git a/packages/runtime/src/scr_net.c b/packages/runtime/src/scr_net.c index 29a4af393..4d5a37f77 100644 --- a/packages/runtime/src/scr_net.c +++ b/packages/runtime/src/scr_net.c @@ -556,6 +556,10 @@ struct ScrNetSocket { * unconsumed stream). resume() clears user_paused and sets flowing. */ bool user_paused; bool flowing; + /* scr_net_sock_hold: a protocol consumer (an HTTP/1 server request + * holding unread body bytes) stops reads without touching the user's + * pause state */ + bool hold; /* EOF arrived while the socket buffered for a consumer it does not have * yet: 'end' waits until one drains the buffer (Node's readable state) */ bool eof_pending; @@ -978,7 +982,7 @@ static ScrNetSocket *scr_net_sock_new(void) { #define SCR_NET_READ_AHEAD 65536u static void scr_net_sock_update_read(ScrNetSocket *s) { - bool want = s->fd >= 0 && !s->connecting && !s->rd_eof && !s->user_paused && + bool want = s->fd >= 0 && !s->connecting && !s->rd_eof && !s->user_paused && !s->hold && (s->data_ls.n > 0 || s->pipe_dst != NULL || s->native_data != NULL || (s->tunnel != NULL && !s->tunnel->send_only) /* the proxy's CONNECT answer */ || s->flowing /* resume(): flow (and discard sans listeners) */ || @@ -1303,7 +1307,7 @@ static void scr_net_sock_read(ScrNetSocket *s) { if (s->tunnel != NULL) return; /* readiness feeds the tunnel pump instead */ if (s->tops && !s->t_est) return; /* readiness feeds the handshake pump instead */ for (;;) { - if (s->fd < 0 || s->user_paused) return; + if (s->fd < 0 || s->user_paused || s->hold) return; bool flowing = s->data_ls.n > 0 || s->pipe_dst || s->native_data || s->flowing; /* buffered (peeked/unshifted) bytes first — they precede the * kernel's in the stream; a TLS engine drains them through its bio */ @@ -3102,6 +3106,15 @@ ScrNetSocket *scr_net_sock_pause(ScrNetSocket *s) { return scr_net_sock_retain(s); } +/* Stops (hold) or restarts reads for a protocol consumer that has + * buffered as much as it accepts, as Node's readStop when push() returns + * false. Separate from pause()/resume(), which stay the user's. */ +void scr_net_sock_hold(ScrNetSocket *s, bool hold) { + if (s->hold == hold) return; + s->hold = hold; + scr_net_sock_update_read(s); +} + /* socket.resume(): flowing mode — a consumer even with no 'data' * listener (arrived bytes discard, so 'end' can be reached, Node's * resumed-but-unconsumed stream). Buffered bytes deliver from the next @@ -5271,7 +5284,7 @@ static bool scr_net_flags_pending(void) { * plaintext inside a transport, or unshifted bytes in the receive * buffer: the poller can't re-signal them, the sweep must deliver */ bool consumers = (s->data_ls.n > 0 || s->pipe_dst || s->native_data || s->flowing) && - !s->user_paused; + !s->user_paused && !s->hold; if (consumers && s->fd >= 0) { if (s->tops && s->t_est && s->tops->pending && s->tops->pending(s->tctx)) return true; if (!s->tops && s->rlen > s->rhead) return true; @@ -5377,7 +5390,7 @@ static void scr_net_sweep(void) { while (sock) { ScrNetSocket *next = sock->next; /* may unregister below */ scr_net_sock_retain(sock); /* callbacks may drop every other ref */ - if (sock->fd >= 0 && !sock->user_paused && + if (sock->fd >= 0 && !sock->user_paused && !sock->hold && (sock->data_ls.n > 0 || sock->pipe_dst || sock->native_data || sock->flowing) && ((sock->tops && sock->t_est && sock->tops->pending && sock->tops->pending(sock->tctx)) || (!sock->tops && sock->rlen > sock->rhead))) { diff --git a/packages/runtime/src/scr_net.h b/packages/runtime/src/scr_net.h index ffa9b2f72..6df6945ea 100644 --- a/packages/runtime/src/scr_net.h +++ b/packages/runtime/src/scr_net.h @@ -232,6 +232,7 @@ void scr_net_sock_destroy(ScrNetSocket *s); * counters/flags the compat surface reads. */ ScrNetSocket *scr_net_sock_pause(ScrNetSocket *s); /* +1: chaining */ ScrNetSocket *scr_net_sock_resume(ScrNetSocket *s); /* +1: chaining */ +void scr_net_sock_hold(ScrNetSocket *s, bool hold); /* protocol read hold (not pause) */ ScrNetSocket *scr_net_sock_set_nodelay(ScrNetSocket *s, bool enable); /* +1: chaining */ void scr_net_sock_set_keepalive(ScrNetSocket *s, bool enable, double delay_ms); void scr_net_sock_destroy_soon(ScrNetSocket *s); diff --git a/packages/runtime/src/scr_pki.c b/packages/runtime/src/scr_pki.c index 54aadc06b..94a0000e7 100644 --- a/packages/runtime/src/scr_pki.c +++ b/packages/runtime/src/scr_pki.c @@ -5,8 +5,9 @@ * asymmetricKeyDetails, export(), equals()), crypto.sign and crypto.verify, * and the Sign and Verify objects of createSign and createVerify. Keys read * and write PEM and DER as PKCS#1, SEC1, PKCS#8 and SubjectPublicKeyInfo, - * and JWK; the EC curves are prime256v1, secp384r1, secp521r1 and - * secp256k1, and the Ed25519 arithmetic is scr_ed25519.c. Argument checks + * JWK, and the raw-public and raw-private encodings; the EC curves are + * prime256v1, secp384r1, secp521r1 and secp256k1, and the Ed25519 + * arithmetic is scr_ed25519.c. Argument checks * and errors follow Node 24's lib/internal/crypto/keys.js, keygen.js and * sig.js. Encrypted keys and other key types throw a coded SC2020 error. * All arguments are borrowed. */ @@ -68,14 +69,25 @@ static ScrStr *sp_received(const ScrDyn *got, bool range) { } /* ERR_INVALID_ARG_VALUE for an option ("The property 'options.type' is - * invalid. Received 'spki'"). */ -static void sp_property_invalid(const char *name, const ScrDyn *got) { + * invalid. Received 'spki'"), with another reason than "is invalid". */ +static void sp_property_invalid_because(const char *name, const ScrDyn *got, const char *reason) { ScrStr *received = sp_received(got, false); - scr_pki_throw_fmt(SCR_ERR_TYPE, "ERR_INVALID_ARG_VALUE", "The property '%s' is invalid. Received %.*s", name, + scr_pki_throw_fmt(SCR_ERR_TYPE, "ERR_INVALID_ARG_VALUE", "The property '%s' %s. Received %.*s", name, reason, (int)received->len, received->data); scr_str_release(received); } +static void sp_property_invalid(const char *name, const ScrDyn *got) { + sp_property_invalid_because(name, got, "is invalid"); +} + +/* ERR_CRYPTO_INCOMPATIBLE_KEY_OPTIONS ("The selected key encoding pkcs1 + * can only be used for RSA keys."). */ +static void sp_incompatible(const char *encoding, const char *reason) { + scr_pki_throw_fmt(SCR_ERR_ERROR, "ERR_CRYPTO_INCOMPATIBLE_KEY_OPTIONS", "The selected key encoding %s %s.", encoding, + reason); +} + static void sp_unsupported_input(void) { sp_throw(SCR_ERR_ERROR, "ERR_OSSL_UNSUPPORTED", "error:1E08010C:DECODER routines::unsupported"); } @@ -867,6 +879,8 @@ static ScrBytes *sp_source_bytes(const ScrDyn *v, const ScrDyn *encoding) { return NULL; } +static SpKey *sp_key_from_raw(const ScrDyn *input, const ScrDyn *data, const ScrDyn *format, bool private_input); + static const char sp_key_input_types[] = "of type string or an instance of ArrayBuffer, Buffer, TypedArray, DataView, or URL"; @@ -918,6 +932,14 @@ static ScrDyn *sp_create_key(ScrDyn *input, bool private_key) { SpKey *k = sp_key_from_jwk(data, private_key); return k ? sp_key_box(k) : NULL; } + if (sp_is_str(format, "raw-public") || sp_is_str(format, "raw-private") || sp_is_str(format, "raw-seed")) { + SpKey *k = sp_key_from_raw(input, data, format, private_key); + if (!k || private_key || !k->private_key) return k ? sp_key_box(k) : NULL; + SpKey *pub = sp_public_of(k); + sp_key_release(k); + if (!pub) sp_unsupported_input(); + return pub ? sp_key_box(pub) : NULL; + } if (!sp_absent(format) && !sp_is_str(format, "pem") && !sp_is_str(format, "der")) { sp_property_invalid("key.format", format); return NULL; @@ -948,6 +970,183 @@ ScrDyn *scr_crypto_create_private_key(ScrDyn *key) { return sp_create_key(key, t ScrDyn *scr_crypto_create_public_key(ScrDyn *key) { return sp_create_key(key, false); } +/* ── raw keys ─────────────────────────────────────────────────────────── */ + +/* A namedCurve as Node reads it: the OpenSSL short name or the NIST name, + * matched exactly. NULL for other names. */ +static const SpCurve *sp_curve_named(const ScrDyn *name) { + for (size_t i = 0; i < sizeof sp_curves / sizeof sp_curves[0]; i++) + if (sp_is_str(name, sp_curves[i].name) || (sp_curves[i].nist && sp_is_str(name, sp_curves[i].nist))) + return &sp_curves[i]; + return NULL; +} + +static void sp_incompatible_key_type(void) { + sp_throw(SCR_ERR_ERROR, "ERR_CRYPTO_INCOMPATIBLE_KEY_OPTIONS", + "The selected key encoding is incompatible with the key type"); +} + +static void sp_invalid_key_data(void) { sp_throw(SCR_ERR_TYPE, "ERR_INVALID_ARG_VALUE", "Invalid key data"); } + +/* A key's raw encoding: an EC public point, compressed when asked, or the + * private scalar at the curve's size, or an Ed25519 public key or seed. + * raw-seed, and every encoding of an RSA key, throw + * ERR_CRYPTO_INCOMPATIBLE_KEY_OPTIONS. NULL with the error pending. */ +static ScrDyn *sp_export_raw(SpKey *k, bool private_part, bool seed, bool compressed) { + if (seed || sp_is_rsa(k)) { + sp_incompatible_key_type(); + return NULL; + } + if (sp_is_ed(k)) return sp_dyn_buffer(private_part ? k->ed_seed : k->ed_pub, 32); + mbedtls_ecp_keypair *ec = mbedtls_pk_ec(k->pk); + const mbedtls_ecp_group *grp = &ec->MBEDTLS_PRIVATE(grp); + unsigned char buf[1 + 2 * 66]; + size_t n = (grp->nbits + 7) / 8; + int rc = private_part ? mbedtls_mpi_write_binary(&ec->MBEDTLS_PRIVATE(d), buf, n) + : mbedtls_ecp_point_write_binary( + grp, &ec->MBEDTLS_PRIVATE(Q), + compressed ? MBEDTLS_ECP_PF_COMPRESSED : MBEDTLS_ECP_PF_UNCOMPRESSED, &n, buf, sizeof buf); + ScrDyn *out = rc == 0 ? sp_dyn_buffer(buf, n) : NULL; + mbedtls_platform_zeroize(buf, sizeof buf); + if (!out) sp_unsupported_input(); + return out; +} + +/* Node's named key types that this runtime does not generate or read: + * the other OKP curves and the post-quantum types. */ +static const char *const sp_other_named_types[] = { + "ed448", + "x25519", + "x448", + "ml-dsa-44", + "ml-dsa-65", + "ml-dsa-87", + "ml-kem-512", + "ml-kem-768", + "ml-kem-1024", + "slh-dsa-sha2-128f", + "slh-dsa-sha2-128s", + "slh-dsa-sha2-192f", + "slh-dsa-sha2-192s", + "slh-dsa-sha2-256f", + "slh-dsa-sha2-256s", + "slh-dsa-shake-128f", + "slh-dsa-shake-128s", + "slh-dsa-shake-192f", + "slh-dsa-shake-192s", + "slh-dsa-shake-256f", + "slh-dsa-shake-256s", +}; + +/* createPrivateKey / createPublicKey of { key, format: "raw-public" | + * "raw-private" | "raw-seed", asymmetricKeyType, namedCurve }: an EC point + * (uncompressed or compressed) or scalar, or an Ed25519 public key or + * seed. The argument checks are Node's prepareAsymmetricKey order, then + * the key type, the seed format, the curve and the data. NULL with the + * error pending. */ +static SpKey *sp_key_from_raw(const ScrDyn *input, const ScrDyn *data, const ScrDyn *format, bool private_input) { + bool raw_public = sp_is_str(format, "raw-public"); + bool seed = sp_is_str(format, "raw-seed"); + if (private_input && raw_public) { + sp_property_invalid("key.format", format); + return NULL; + } + if (data->kind != SCR_DYN_BYTES && !scr_buffer_storage_is(data)) { + scr_dyn_arg_type_fail("key.key", "an instance of ArrayBuffer, Buffer, TypedArray, or DataView", data); + return NULL; + } + const ScrDyn *type = sp_opt(input, "asymmetricKeyType"); + if (type->kind != SCR_DYN_STR) { + scr_dyn_prop_type_fail("key.asymmetricKeyType", "of type string", type); + return NULL; + } + bool ec = sp_is_str(type, "ec"); + bool ed = sp_is_str(type, "ed25519"); + const ScrDyn *curve_name = sp_opt(input, "namedCurve"); + if (ec && curve_name->kind != SCR_DYN_STR) { + scr_dyn_prop_type_fail("key.namedCurve", "of type string", curve_name); + return NULL; + } + if (sp_is_str(type, "rsa") || sp_is_str(type, "rsa-pss") || sp_is_str(type, "dsa") || sp_is_str(type, "dh")) { + sp_incompatible_key_type(); + return NULL; + } + bool other = false; + for (size_t i = 0; i < sizeof sp_other_named_types / sizeof sp_other_named_types[0]; i++) + other = other || sp_is_str(type, sp_other_named_types[i]); + if (!ec && !ed && !other) { + ScrStr *s = type->v.str; + scr_pki_throw_fmt(SCR_ERR_TYPE, "ERR_INVALID_ARG_VALUE", "Invalid asymmetricKeyType: %.*s", (int)s->len, s->data); + return NULL; + } + bool ml = !ec && !ed && type->v.str->len > 3 && memcmp(type->v.str->data, "ml-", 3) == 0; + if (seed && !ml) { + sp_incompatible_key_type(); + return NULL; + } + if (other) { + sp_not_supported("raw keys of this asymmetricKeyType"); + return NULL; + } + const SpCurve *curve = NULL; + if (ec && !(curve = sp_curve_named(curve_name))) { + sp_throw(SCR_ERR_TYPE, "ERR_CRYPTO_INVALID_CURVE", "Invalid EC curve name"); + return NULL; + } + ScrBytes *bytes = scr_bytes_buffer_source(data); + if (!bytes) return NULL; + const unsigned char *p = bytes->data; + size_t len = (size_t)scr_bytes_byte_len(bytes); + SpKey *k = sp_key_alloc(!raw_public); + bool ok = false; + if (ed) { + k->ed25519 = true; + if (len == 32 && raw_public) { + memcpy(k->ed_pub, p, 32); + ok = true; + } else if (len == 32) { + const ScrPkiEd25519 *eds = sp_ed(); + if (!eds) { + sp_key_release(k); + scr_bytes_release(bytes); + return NULL; + } + memcpy(k->ed_seed, p, 32); + eds->public_key(k->ed_pub, k->ed_seed); + ok = true; + } + } else if (raw_public && len == 1 && p[0] == 0) { + sp_key_release(k); + scr_bytes_release(bytes); + sp_not_supported("EC public keys at the point at infinity"); + return NULL; + } else if (mbedtls_pk_setup(&k->pk, mbedtls_pk_info_from_type(MBEDTLS_PK_ECKEY)) == 0) { + mbedtls_ecp_keypair *kp = mbedtls_pk_ec(k->pk); + mbedtls_ecp_group *grp = &kp->MBEDTLS_PRIVATE(grp); + ok = mbedtls_ecp_group_load(grp, curve->id) == 0; + if (ok && raw_public) { + ok = mbedtls_ecp_point_read_binary(grp, &kp->MBEDTLS_PRIVATE(Q), p, len) == 0 && + mbedtls_ecp_check_pubkey(grp, &kp->MBEDTLS_PRIVATE(Q)) == 0; + } else if (ok) { + ok = len == (grp->nbits + 7) / 8 && mbedtls_mpi_read_binary(&kp->MBEDTLS_PRIVATE(d), p, len) == 0; + if (ok && mbedtls_ecp_check_privkey(grp, &kp->MBEDTLS_PRIVATE(d)) != 0) { + sp_key_release(k); + scr_bytes_release(bytes); + sp_not_supported("EC private keys outside 1 to n - 1"); + return NULL; + } + ok = ok && mbedtls_ecp_mul(grp, &kp->MBEDTLS_PRIVATE(Q), &kp->MBEDTLS_PRIVATE(d), &grp->G, sp_random, NULL) == 0; + } + } + scr_bytes_release(bytes); + if (!ok) { + sp_key_release(k); + sp_invalid_key_data(); + return NULL; + } + return k; +} + /* ── export ───────────────────────────────────────────────────────────── */ /* export({ format, type }) per Node's parseKeyEncoding: format "pem", @@ -964,11 +1163,37 @@ static ScrDyn *sp_export(SpKey *k, const ScrDyn *options) { } static ScrDyn *sp_export_plain(SpKey *k, const ScrDyn *options) { + const ScrDyn *format = sp_opt(options, "format"); + /* A private key's passphrase applies only to PEM and DER. */ + if (k->private_key && !sp_absent(sp_opt(options, "passphrase")) && !sp_is_str(format, "pem") && + !sp_is_str(format, "der")) { + ScrStr *text = scr_dyn_string_coerce_js(format); + if (!text) return NULL; + scr_pki_throw_fmt(SCR_ERR_ERROR, "ERR_CRYPTO_INCOMPATIBLE_KEY_OPTIONS", + "The selected key encoding %.*s does not support encryption.", (int)text->len, text->data); + scr_str_release(text); + return NULL; + } + if (!k->private_key && sp_is_str(format, "raw-public")) { + const ScrDyn *type = sp_opt(options, "type"); + bool compressed = sp_is_str(type, "compressed"); + /* Only an EC point has a compressed form. */ + if (sp_is_ec(k) && !sp_absent(type) && !compressed && !sp_is_str(type, "uncompressed")) { + ScrStr *received = sp_received(type, false); + scr_pki_throw_fmt(SCR_ERR_TYPE, "ERR_INVALID_ARG_VALUE", + "The property 'options.type' must be one of: 'compressed', 'uncompressed'. Received %.*s", + (int)received->len, received->data); + scr_str_release(received); + return NULL; + } + return sp_export_raw(k, false, false, sp_is_ec(k) && compressed); + } + if (k->private_key && (sp_is_str(format, "raw-private") || sp_is_str(format, "raw-seed"))) + return sp_export_raw(k, true, sp_is_str(format, "raw-seed"), false); if (options == NULL || !sp_is_object(options)) { scr_dyn_arg_type_fail("options", "of type object", options ? options : scr_dyn_undefined()); return NULL; } - const ScrDyn *format = sp_opt(options, "format"); if (sp_is_str(format, "jwk")) return sp_export_jwk(k); bool pem = sp_is_str(format, "pem"); if (!pem && !sp_is_str(format, "der")) { @@ -989,13 +1214,11 @@ static ScrDyn *sp_export_plain(SpKey *k, const ScrDyn *options) { return NULL; } if (pkcs1 && !sp_is_rsa(k)) { - sp_throw(SCR_ERR_ERROR, "ERR_CRYPTO_INCOMPATIBLE_KEY_OPTIONS", - "The selected key encoding pkcs1 can only be used for RSA keys."); + sp_incompatible("pkcs1", "can only be used for RSA keys"); return NULL; } if (sec1 && !sp_is_ec(k)) { - sp_throw(SCR_ERR_ERROR, "ERR_CRYPTO_INCOMPATIBLE_KEY_OPTIONS", - "The selected key encoding sec1 can only be used for EC keys."); + sp_incompatible("sec1", "can only be used for EC keys"); return NULL; } unsigned char *der = malloc(SP_DER_MAX); @@ -1175,132 +1398,266 @@ static void sp_key_install(void) { /* ── generateKeyPairSync ──────────────────────────────────────────────── */ /* A key in the encoding an options.publicKeyEncoding/privateKeyEncoding - * object asks for, or the KeyObject when it is undefined. Consumes `k`. */ + * object asks for, or the KeyObject when it is undefined or null. + * Consumes `k`. */ static ScrDyn *sp_encoded(SpKey *k, const ScrDyn *encoding) { - if (sp_absent(encoding)) return sp_key_box(k); + if (sp_absent(encoding) || encoding->kind == SCR_DYN_NULL) return sp_key_box(k); ScrDyn *result = sp_export(k, encoding); sp_key_release(k); return result; } -static bool sp_check_encoding_plain(const ScrDyn *encoding, const char *name, bool private_key); - -static bool sp_check_encoding(const ScrDyn *options, const char *name, bool private_key) { - ScrDyn *encoding = sp_plain(sp_opt(options, name)); - if (!encoding) return false; - bool ok = sp_check_encoding_plain(encoding, name, private_key); - scr_dyn_release(encoding); - return ok; +/* validateObject: an object other than null or an array (typeof + * "object"); false with ERR_INVALID_ARG_TYPE pending. */ +static bool sp_check_object(const ScrDyn *v, const char *name) { + if (v->kind != SCR_DYN_NULL && v->kind != SCR_DYN_ARR && scr_dyn_is_object(v)) return true; + scr_dyn_arg_type_fail(name, "of type object", v); + return false; } -static bool sp_check_encoding_plain(const ScrDyn *encoding, const char *name, bool private_key) { - if (sp_absent(encoding)) return true; - char full[64]; - snprintf(full, sizeof full, "options.%s", name); - if (!sp_is_object(encoding)) { - sp_property_invalid(full, encoding); +/* validateUint32 for an option: a number, then an integer, then the + * range; -0 reads as 0. False with the error pending. */ +static bool sp_check_uint32(const ScrDyn *v, const char *name, double *out) { + if (v->kind != SCR_DYN_NUM) { + scr_dyn_prop_type_fail(name, "of type number", v); return false; } - const ScrDyn *format = sp_opt(encoding, "format"); - if (!sp_is_str(format, "pem") && !sp_is_str(format, "der") && !sp_is_str(format, "jwk")) { - snprintf(full, sizeof full, "options.%s.format", name); - sp_property_invalid(full, format); - return false; - } - if (sp_is_str(format, "jwk")) return true; - const ScrDyn *type = sp_opt(encoding, "type"); - bool valid = private_key ? sp_is_str(type, "pkcs1") || sp_is_str(type, "pkcs8") || sp_is_str(type, "sec1") - : sp_is_str(type, "pkcs1") || sp_is_str(type, "spki"); - if (!valid) { - snprintf(full, sizeof full, "options.%s.type", name); - sp_property_invalid(full, type); + double x = v->v.num; + bool integer = isfinite(x) && trunc(x) == x; + if (!integer || x < 0 || x > 4294967295.0) { + ScrStr *received = sp_received(v, true); + scr_pki_throw_fmt(SCR_ERR_RANGE, "ERR_OUT_OF_RANGE", + "The value of \"%s\" is out of range. It must be %s. Received %.*s", name, + integer ? ">= 0 && <= 4294967295" : "an integer", (int)received->len, received->data); + scr_str_release(received); return false; } + *out = x + 0.0; return true; } -/* Generates the private key for (type, options), validated in Node's - * order; NULL with the error pending. */ -static SpKey *sp_generate(const ScrDyn *type, const ScrDyn *options) { +static bool sp_check_encoding_plain(const ScrDyn *enc, const char *name, const ScrDyn *type, bool private_key, + const char **refusal); + +/* Node's parseKeyEncoding for options.publicKeyEncoding or + * options.privateKeyEncoding of a `type` key: true when the encoding is + * undefined, null or valid, false with the error pending. A valid + * encrypted encoding, which this runtime does not write, sets *refusal + * when it is unset. */ +static bool sp_check_encoding(const ScrDyn *options, const char *key, const ScrDyn *type, bool private_key, + const char **refusal) { + ScrDyn *enc = sp_plain(sp_opt(options, key)); + if (!enc) return false; + char name[64]; + snprintf(name, sizeof name, "options.%s", key); + bool ok = sp_check_encoding_plain(enc, name, type, private_key, refusal); + scr_dyn_release(enc); + return ok; +} + +static bool sp_check_encoding_plain(const ScrDyn *enc, const char *name, const ScrDyn *type, bool private_key, + const char **refusal) { + if (sp_absent(enc) || enc->kind == SCR_DYN_NULL) return true; + if (!scr_dyn_is_object(enc)) { + sp_property_invalid(name, enc); + return false; + } + if (!sp_check_object(enc, "options")) return false; + /* Node's isPublic: true for a public key encoding of a non-empty type, + * false for a private key encoding, and unset otherwise. */ + bool is_public = !private_key && type->v.str->len > 0; + char full[96]; + const ScrDyn *format = sp_opt(enc, "format"); + const ScrDyn *kind = sp_opt(enc, "type"); + bool raw_public = sp_is_str(format, "raw-public"); + bool raw_private = sp_is_str(format, "raw-private") || sp_is_str(format, "raw-seed"); + bool jwk = sp_is_str(format, "jwk"); + bool der = sp_is_str(format, "der"); + if ((!raw_public && !raw_private && !jwk && !der && !sp_is_str(format, "pem")) || (raw_public && private_key) || + (raw_private && is_public)) { + snprintf(full, sizeof full, "%s.format", name); + sp_property_invalid(full, format); + return false; + } + bool pkcs1 = false, sec1 = false; + if (raw_public || raw_private) { + if (!sp_absent(kind) && + !(raw_public && (sp_is_str(kind, "uncompressed") || sp_is_str(kind, "compressed")))) { + snprintf(full, sizeof full, "%s.type", name); + sp_property_invalid(full, kind); + return false; + } + } else { + pkcs1 = sp_is_str(kind, "pkcs1"); + sec1 = !is_public && sp_is_str(kind, "sec1"); + bool valid = (jwk && sp_absent(kind)) || pkcs1 || sec1 || (!private_key && sp_is_str(kind, "spki")) || + (!is_public && sp_is_str(kind, "pkcs8")); + if (!valid) { + snprintf(full, sizeof full, "%s.type", name); + sp_property_invalid(full, kind); + return false; + } + if (pkcs1 && !sp_is_str(type, "rsa")) { + sp_incompatible("pkcs1", "can only be used for RSA keys"); + return false; + } + if (sec1 && !sp_is_str(type, "ec")) { + sp_incompatible("sec1", "can only be used for EC keys"); + return false; + } + } + bool encrypted = false; + if (!is_public) { + const ScrDyn *cipher = sp_opt(enc, "cipher"); + const ScrDyn *passphrase = sp_opt(enc, "passphrase"); + encrypted = !sp_absent(cipher) && cipher->kind != SCR_DYN_NULL; + if (raw_private) { + if (encrypted || !sp_absent(passphrase)) { + sp_incompatible("raw format", "does not support encryption"); + return false; + } + } else if (encrypted) { + if (cipher->kind != SCR_DYN_STR) { + snprintf(full, sizeof full, "%s.cipher", name); + sp_property_invalid(full, cipher); + return false; + } + if (der && (pkcs1 || sec1)) { + sp_incompatible(pkcs1 ? "pkcs1" : "sec1", "does not support encryption"); + return false; + } + if (passphrase->kind != SCR_DYN_STR && passphrase->kind != SCR_DYN_BYTES && !scr_buffer_storage_is(passphrase)) { + snprintf(full, sizeof full, "%s.passphrase", name); + sp_property_invalid(full, passphrase); + return false; + } + } else if (!sp_absent(passphrase)) { + snprintf(full, sizeof full, "%s.cipher", name); + sp_property_invalid_because(full, cipher, "is required when a passphrase is specified"); + return false; + } + } + if (!*refusal && encrypted) *refusal = "encrypted private key export"; + return true; +} + +/* The key generateKeyPair makes, as sp_keygen_check reads it. */ +typedef struct { + enum { SP_GEN_RSA, SP_GEN_EC, SP_GEN_ED25519 } kind; + double bits, exponent; /* RSA */ + const SpCurve *curve; /* EC */ +} SpKeyGen; + + +/* createJob's checks for generateKeyPair(Sync)(type, options) in Node's + * order: the type, the options object, the public and private key + * encodings, then the options of the type. False with the error pending. + * The forms this runtime does not generate throw a coded SC2020 error + * after all of these checks. */ +static bool sp_keygen_check(const ScrDyn *type, const ScrDyn *options, SpKeyGen *gen) { if (type->kind != SCR_DYN_STR) { scr_dyn_arg_type_fail("type", "of type string", type); - return NULL; + return false; } - bool rsa = sp_is_str(type, "rsa"); - bool ec = sp_is_str(type, "ec"); - bool ed = sp_is_str(type, "ed25519"); - static const char *const known[] = {"rsa-pss", "dsa", "ed448", "x25519", "x448", "dh"}; - bool other = false; - for (size_t i = 0; i < sizeof known / sizeof known[0]; i++) other = other || sp_is_str(type, known[i]); - if (!rsa && !ec && !ed && !other) { - ScrStr *received = sp_received(type, false); - scr_pki_throw_fmt(SCR_ERR_TYPE, "ERR_INVALID_ARG_VALUE", - "The argument 'type' must be a supported key type. Received %.*s", (int)received->len, - received->data); - scr_str_release(received); - return NULL; + if (!sp_absent(options) && !sp_check_object(options, "options")) return false; + const char *refusal = NULL; + if (!sp_check_encoding(options, "publicKeyEncoding", type, false, &refusal) || + !sp_check_encoding(options, "privateKeyEncoding", type, true, &refusal)) + return false; + bool pss = sp_is_str(type, "rsa-pss"); + if (sp_is_str(type, "rsa") || pss) { + if (!sp_check_object(options, "options")) return false; + gen->kind = SP_GEN_RSA; + if (!sp_check_uint32(sp_opt(options, "modulusLength"), "options.modulusLength", &gen->bits)) return false; + const ScrDyn *exponent = sp_opt(options, "publicExponent"); + gen->exponent = 65537; + if (!sp_absent(exponent) && exponent->kind != SCR_DYN_NULL && + !sp_check_uint32(exponent, "options.publicExponent", &gen->exponent)) + return false; + if (pss && !refusal) refusal = "generateKeyPairSync for this key type"; + } else if (sp_is_str(type, "dsa") || sp_is_str(type, "dh")) { + if (!sp_check_object(options, "options")) return false; + if (!refusal) refusal = "generateKeyPairSync for this key type"; + } else if (sp_is_str(type, "ec")) { + if (!sp_check_object(options, "options")) return false; + gen->kind = SP_GEN_EC; + const ScrDyn *curve_name = sp_opt(options, "namedCurve"); + if (curve_name->kind != SCR_DYN_STR) { + scr_dyn_prop_type_fail("options.namedCurve", "of type string", curve_name); + return false; + } + const ScrDyn *param = sp_opt(options, "paramEncoding"); + if (sp_is_str(param, "explicit")) { + if (!refusal) refusal = "generateKeyPair with paramEncoding 'explicit'"; + } else if (!sp_absent(param) && param->kind != SCR_DYN_NULL && !sp_is_str(param, "named")) { + sp_property_invalid("options.paramEncoding", param); + return false; + } + gen->curve = NULL; + for (size_t i = 0; i < sizeof sp_curves / sizeof sp_curves[0]; i++) + if (sp_is_str(curve_name, sp_curves[i].name) || + (sp_curves[i].nist && sp_is_str(curve_name, sp_curves[i].nist))) + gen->curve = &sp_curves[i]; + if (!gen->curve) { + sp_throw(SCR_ERR_TYPE, "ERR_CRYPTO_INVALID_CURVE", "Invalid EC curve name"); + return false; + } + } else if (sp_is_str(type, "ed25519")) { + gen->kind = SP_GEN_ED25519; + } else { + bool named = false; + for (size_t i = 0; i < sizeof sp_other_named_types / sizeof sp_other_named_types[0]; i++) + named = named || sp_is_str(type, sp_other_named_types[i]); + if (!named) { + ScrStr *received = sp_received(type, false); + scr_pki_throw_fmt(SCR_ERR_TYPE, "ERR_INVALID_ARG_VALUE", + "The argument 'type' must be a supported key type. Received %.*s", (int)received->len, + received->data); + scr_str_release(received); + return false; + } + if (!refusal) refusal = "generateKeyPairSync for this key type"; } - /* The ed25519 options are optional. */ - if (!sp_is_object(options) && !(ed && sp_absent(options))) { - scr_dyn_arg_type_fail("options", "of type object", options); - return NULL; - } - if (other) { - sp_not_supported("generateKeyPairSync for this key type"); - return NULL; + if (refusal) { + sp_not_supported(refusal); + return false; } + return gen->kind != SP_GEN_ED25519 || sp_ed() != NULL; +} + +/* OpenSSL's error for an RSA modulus length or public exponent it does not + * generate, as Node reports it, or NULL. */ +static const char *sp_rsa_keygen_failure(double bits, double exponent) { + if (bits < 512) return "error:1C8000AB:Provider routines::key size too small"; + if (exponent == 0) return "Key generation job failed"; + if (bits > 2147483647.0) return "error:02000078:rsa routines::key size too small"; + if (exponent == 1 || fmod(exponent, 2) == 0) return "error:020000B2:rsa routines::pub exponent out of range"; + return NULL; +} + +/* Generates the key `gen` describes: NULL with the error pending, and + * *job_error set for the errors generateKeyPair passes to its callback. */ +static SpKey *sp_keygen_run(const SpKeyGen *gen, bool *job_error) { + *job_error = false; SpKey *k = sp_key_alloc(true); - if (ed) { - if (!sp_check_encoding(options, "publicKeyEncoding", false) || - !sp_check_encoding(options, "privateKeyEncoding", true)) { - sp_key_release(k); - return NULL; - } - const ScrPkiEd25519 *ed = sp_ed(); - if (!ed) { - sp_key_release(k); - return NULL; - } + if (gen->kind == SP_GEN_ED25519) { k->ed25519 = true; sp_random(NULL, k->ed_seed, 32); - ed->public_key(k->ed_pub, k->ed_seed); + scr_pki_ed25519->public_key(k->ed_pub, k->ed_seed); return k; } - if (rsa) { - const ScrDyn *bits = sp_opt(options, "modulusLength"); - if (bits->kind != SCR_DYN_NUM) { + if (gen->kind == SP_GEN_RSA) { + const char *failure = sp_rsa_keygen_failure(gen->bits, gen->exponent); + if (failure) { sp_key_release(k); - scr_dyn_prop_type_fail("options.modulusLength", "of type number", bits); + *job_error = true; + scr_pki_throw_fmt(SCR_ERR_ERROR, NULL, "%s", failure); return NULL; } - double b = bits->v.num; - /* validateUint32: an integer first, then the range. */ - bool integer = isfinite(b) && trunc(b) == b; - if (!integer || b < 0 || b > 4294967295.0) { - sp_key_release(k); - ScrStr *received = sp_received(bits, true); - scr_pki_throw_fmt(SCR_ERR_RANGE, "ERR_OUT_OF_RANGE", - "The value of \"options.modulusLength\" is out of range. It must be %s. Received %.*s", - integer ? ">= 0 && <= 4294967295" : "an integer", (int)received->len, received->data); - scr_str_release(received); - return NULL; - } - const ScrDyn *exp = sp_opt(options, "publicExponent"); - double e = 65537; - if (!sp_absent(exp)) { - if (exp->kind != SCR_DYN_NUM) { - sp_key_release(k); - scr_dyn_prop_type_fail("options.publicExponent", "of type number", exp); - return NULL; - } - e = exp->v.num; - } - if (!sp_check_encoding(options, "publicKeyEncoding", false) || - !sp_check_encoding(options, "privateKeyEncoding", true)) { - sp_key_release(k); - return NULL; - } - if (b < 512 || b > 16384 || e < 3 || e > 2147483647.0 || e != (double)(int64_t)e || ((int64_t)e & 1) == 0 || + /* mbedTLS generates even modulus lengths from 1024 to 8192 bits and, + * past the checks above, odd exponents up to INT_MAX. */ + double b = gen->bits, e = gen->exponent; + if (b < MBEDTLS_RSA_GEN_KEY_MIN_BITS || b > MBEDTLS_MPI_MAX_BITS || fmod(b, 2) != 0 || e > 2147483647.0 || mbedtls_pk_setup(&k->pk, mbedtls_pk_info_from_type(MBEDTLS_PK_RSA)) != 0 || mbedtls_rsa_gen_key(mbedtls_pk_rsa(k->pk), sp_random, NULL, (unsigned)b, (int)e) != 0) { sp_key_release(k); @@ -1309,29 +1666,8 @@ static SpKey *sp_generate(const ScrDyn *type, const ScrDyn *options) { } return k; } - const ScrDyn *curve_name = sp_opt(options, "namedCurve"); - if (curve_name->kind != SCR_DYN_STR) { - sp_key_release(k); - scr_dyn_prop_type_fail("options.namedCurve", "of type string", curve_name); - return NULL; - } - if (!sp_check_encoding(options, "publicKeyEncoding", false) || - !sp_check_encoding(options, "privateKeyEncoding", true)) { - sp_key_release(k); - return NULL; - } - const SpCurve *curve = NULL; - for (size_t i = 0; i < sizeof sp_curves / sizeof sp_curves[0]; i++) - if (sp_is_str(curve_name, sp_curves[i].name) || - (sp_curves[i].nist && sp_is_str(curve_name, sp_curves[i].nist))) - curve = &sp_curves[i]; - if (!curve) { - sp_key_release(k); - sp_throw(SCR_ERR_TYPE, "ERR_CRYPTO_INVALID_CURVE", "Invalid EC curve name"); - return NULL; - } if (mbedtls_pk_setup(&k->pk, mbedtls_pk_info_from_type(MBEDTLS_PK_ECKEY)) != 0 || - mbedtls_ecp_gen_key(curve->id, mbedtls_pk_ec(k->pk), sp_random, NULL) != 0) { + mbedtls_ecp_gen_key(gen->curve->id, mbedtls_pk_ec(k->pk), sp_random, NULL) != 0) { sp_key_release(k); sp_unsupported_input(); return NULL; @@ -1339,17 +1675,14 @@ static SpKey *sp_generate(const ScrDyn *type, const ScrDyn *options) { return k; } -/* generateKeyPairSync(type, options): { publicKey, privateKey }, each a - * KeyObject or the encoding the options ask for. */ -ScrDyn *scr_crypto_generate_key_pair_sync(ScrDyn *type, ScrDyn *options) { - if (options->kind == SCR_DYN_TYPED_REF) { - ScrDyn *value = scr_dyn_typed_ref_materialize(options); - if (!value) return NULL; - ScrDyn *result = scr_crypto_generate_key_pair_sync(type, value); - scr_dyn_release(value); - return result; - } - SpKey *k = sp_generate(type, options); +/* The { publicKey, privateKey } pair for (type, options), each a + * KeyObject or the encoding the options ask for; NULL with the error + * pending, and *job_error as sp_keygen_run sets it. */ +static ScrDyn *sp_key_pair(const ScrDyn *type, const ScrDyn *options, bool *job_error) { + *job_error = false; + SpKeyGen gen; + if (!sp_keygen_check(type, options, &gen)) return NULL; + SpKey *k = sp_keygen_run(&gen, job_error); if (!k) return NULL; SpKey *pub = sp_public_of(k); if (!pub) { @@ -1357,14 +1690,18 @@ ScrDyn *scr_crypto_generate_key_pair_sync(ScrDyn *type, ScrDyn *options) { sp_unsupported_input(); return NULL; } + /* An encoding the key type has no form for (a raw encoding of an RSA + * key, raw-seed) fails the job, as in Node. */ ScrDyn *public_value = sp_encoded(pub, sp_opt(options, "publicKeyEncoding")); if (!public_value) { sp_key_release(k); + *job_error = true; return NULL; } ScrDyn *private_value = sp_encoded(k, sp_opt(options, "privateKeyEncoding")); if (!private_value) { scr_dyn_release(public_value); + *job_error = true; return NULL; } ScrDyn *pair = scr_dyn_new_obj(); @@ -1373,6 +1710,16 @@ ScrDyn *scr_crypto_generate_key_pair_sync(ScrDyn *type, ScrDyn *options) { return pair; } +/* generateKeyPairSync(type, options). */ +ScrDyn *scr_crypto_generate_key_pair_sync(ScrDyn *type, ScrDyn *options) { + ScrDyn *plain = sp_plain(options); + if (!plain) return NULL; + bool job_error; + ScrDyn *pair = sp_key_pair(type, plain, &job_error); + scr_dyn_release(plain); + return pair; +} + /* ── sign and verify ──────────────────────────────────────────────────── */ typedef struct { @@ -1931,12 +2278,44 @@ static void sp_key_pair_deliver(ScrClosure *self) { scr_dyn_release(callback); } -/* generateKeyPair(type, options, callback): argument errors throw, and the - * callback receives (null, publicKey, privateKey) in a later macrotask. */ +/* Calls the callback with the error alone, as Node does for a key + * generation that fails. */ +static void sp_key_pair_fail(ScrClosure *self) { + ScrDyn *callback = sp_cap(self, 0); + ScrDyn *error = sp_cap(self, 1); + ScrDyn *result = scr_dyn_call(callback, &error, 1, "callback"); + if (result) scr_dyn_release(result); + scr_dyn_release(error); + scr_dyn_release(callback); +} + +/* generateKeyPair(type, options, callback): a function in the options + * position is the callback. Argument errors throw; the callback receives + * (null, publicKey, privateKey), or the error of a key generation that + * fails, in a later macrotask. */ void scr_crypto_generate_key_pair_async(ScrDyn *type, ScrDyn *options, ScrDyn *callback) { - ScrDyn *pair = scr_crypto_generate_key_pair_sync(type, options); - if (!pair) return; - ScrClosure *c = scr_closure_new((void *)sp_key_pair_deliver, 2); + if (scr_dyn_is_callable(options)) { + callback = options; + options = scr_dyn_undefined(); + } + if (!scr_dyn_is_callable(callback)) { + scr_dyn_arg_type_fail("callback", "of type function", callback); + return; + } + ScrDyn *plain = sp_plain(options); + if (!plain) return; + bool job_error; + ScrDyn *pair = sp_key_pair(type, plain, &job_error); + scr_dyn_release(plain); + if (!pair && !job_error) return; + void *deliver = (void *)sp_key_pair_deliver; + if (!pair) { + ScrCaught *caught = scr_exc_take(); + pair = scr_caught_to_dyn(caught); + scr_caught_release(caught); + deliver = (void *)sp_key_pair_fail; + } + ScrClosure *c = scr_closure_new(deliver, 2); c->caps[0] = scr_box_new_obj(scr_dyn_retain_v, scr_dyn_release_v, NULL); scr_box_set_ref(c->caps[0], scr_dyn_retain(callback)); c->caps[1] = scr_box_new_obj(scr_dyn_retain_v, scr_dyn_release_v, NULL); diff --git a/packages/runtime/src/scr_publish.c b/packages/runtime/src/scr_publish.c index 4767122dd..afbe4d6af 100644 --- a/packages/runtime/src/scr_publish.c +++ b/packages/runtime/src/scr_publish.c @@ -527,62 +527,12 @@ void scr_throw_published_field(const char *prop, const char *owner) { scr_pub_throw("Cannot assign to read only property '%s' of object '#<%s>'", prop, owner); } -static void scr_pub_index_text(double index, char *out, size_t size) { - char buf[64]; - size_t n = scr_f64_to_str(index, buf); - if (n > size - 1) n = size - 1; - memcpy(out, buf, n); - out[n] = 0; +bool scr_throw_published_array(int op, const ScrArr *a, double index, double n) { + return scr_array_frozen_error(op, a->len, index, n, 0); } -/* op: 0 index write, 1 push (n = argument count), 2 pop, 3 length write, - * 4 unshift (n = argument count), 5 delete, 6 shift, 7 other in-place - * rewrites (splice, reverse, fill, copyWithin, sort). Returns false when - * Node's operation would not write the frozen array at all (it then is not - * an error: e.g. reversing an empty array). */ -bool scr_throw_published_array(int op, const ScrArr *a, double index, double n) { - static const char length[] = "Cannot assign to read only property 'length' of object '[object Array]'"; - char text[64]; - size_t len = a->len; - switch (op) { - case 0: - scr_pub_index_text(index, text, sizeof text); - if (index >= 0 && index < (double)len && index == (double)(size_t)index) - scr_pub_throw("Cannot assign to read only property '%s' of object '[object Array]'%s", text, ""); - else scr_pub_throw("Cannot add property %s, object is not extensible%s", text, ""); - return true; - case 1: - if (n == 0) break; - scr_pub_index_text((double)len, text, sizeof text); - scr_pub_throw("Cannot add property %s, object is not extensible%s", text, ""); - return true; - case 2: - case 6: - if (len == 0) break; - if (op == 6 && len > 1) { - scr_pub_throw("Cannot assign to read only property '0' of object '[object Array]'%s%s", "", ""); - return true; - } - scr_pub_index_text((double)(len - 1), text, sizeof text); - scr_pub_throw("Cannot delete property '%s' of [object Array]%s", text, ""); - return true; - case 3: break; - case 4: - if (n == 0) break; - scr_pub_index_text(len ? (double)len + n - 1 : 0, text, sizeof text); - scr_pub_throw("Cannot add property %s, object is not extensible%s", text, ""); - return true; - case 5: - scr_pub_index_text(index, text, sizeof text); - scr_pub_throw("Cannot delete property '%s' of [object Array]%s", text, ""); - return true; - default: - if (len == 0) return false; - scr_pub_throw("Cannot assign to read only property '0' of object '[object Array]'%s%s", "", ""); - return true; - } - scr_throw_error_msg(SCR_ERR_TYPE, length, sizeof length - 1); - return true; +bool scr_throw_published_array_range(int op, const ScrArr *a, double x, double y, double z) { + return scr_array_frozen_error(op, a->len, x, y, z); } void scr_throw_published_collection(bool set) { diff --git a/packages/runtime/src/scr_rc.h b/packages/runtime/src/scr_rc.h index 18df3a237..4d8e059a9 100644 --- a/packages/runtime/src/scr_rc.h +++ b/packages/runtime/src/scr_rc.h @@ -45,14 +45,32 @@ typedef size_t ScrRcCount; /* Whether `obj`, an object of any kind, is published (immortal). */ #define SCR_RC_PUBLISHED(obj) (SCR_RC_COUNT(obj) >= SCR_RC_PUBLISHED_MIN) +/* Frozen records (Object.freeze). The rc word of a record a program may + * freeze (llvm/shapes.ts) carries SCR_RC_FROZEN beside its count, and that + * record's emitted release ignores the bit. On 64-bit targets the bit is in + * the high half, outside the count; on 32-bit targets it is the count's bit + * 30, which SCR_RC_REFS masks. An immortal word has the bit set too; it + * does not read as frozen. */ +#if SIZE_MAX > UINT32_MAX +#define SCR_RC_FROZEN ((size_t)1 << 62) +#define SCR_RC_REFS(obj) SCR_RC_COUNT(obj) +#else +#define SCR_RC_FROZEN ((size_t)1 << 30) +#define SCR_RC_REFS(obj) (SCR_RC_COUNT(obj) == SCR_RC_IMMORTAL ? SCR_RC_IMMORTAL : SCR_RC_COUNT(obj) & ~SCR_RC_FROZEN) +#endif +static inline bool scr_rc_frozen(const void *obj) { + size_t word = *(const size_t *)obj; + return word != SIZE_MAX && (word & SCR_RC_FROZEN) != 0; +} + /* The class id in the high half of `obj`'s rc word, or 0 for an object - * without one: the high half of any other object's word is 0, or 0xffffffff - * (immortal) or 0x80000000 (program-lifetime), and a class id is never one of - * these. */ + * without one: the high half of any other object's word is 0, 0xffffffff + * (immortal), 0x80000000 (program-lifetime) or 0x40000000 (a frozen record), + * and a class id is never one of these. */ static inline uint32_t scr_rc_class_id(const void *obj) { #if SIZE_MAX > UINT32_MAX uint32_t high = (uint32_t)(*(const size_t *)obj >> 32); - return high == UINT32_MAX || high == UINT32_C(0x80000000) ? 0 : high; + return high == UINT32_MAX || high == UINT32_C(0x80000000) || high == UINT32_C(0x40000000) ? 0 : high; #else (void)obj; return 0; @@ -552,6 +570,14 @@ _Static_assert(sizeof(((ScrCycHdr *)0)->gen) == 1 && SCR_CYC_OLD == 2, * depth; deeper objects wait until the outer disposal drains the worklist. * The callback owns the object and must release its children and free it. */ void scr_rc_destroy(void *obj, void (*destroy)(void *)); +/* scr_rc_destroy for an emitted teardown that keeps the blocks it frees in + * a segment, linked through their first words, instead of freeing them onto + * the free list (llvm/shapes.ts): destroy_seg tears obj down, appending its + * blocks after the segment's last link `tail`, and returns the new last + * link, which this returns. A disposal past the depth budget is queued with + * `destroy`, which frees its own blocks, and leaves `tail` as it was. */ +void *scr_rc_destroy_seg(void *obj, void *(*destroy_seg)(void *, void *), + void (*destroy)(void *), void *tail); /* RC hooks for cycle-headered types. on_release: rc was decremented and * stayed above zero — buffer the object as a possible cycle root (may run @@ -725,12 +751,14 @@ void scr_pub_dyn_fn(void *v, ScrPub *ctx); void scr_pub_arr(void *v, ScrPub *ctx, ScrPubFn elem); void scr_pub_map(void *v, ScrPub *ctx, ScrPubFn key, ScrPubFn val); void scr_publish_refuse(const char *message); -/* Writes into published objects (emitted guards): Node's frozen-object - * TypeErrors. scr_throw_published_array answers false (and throws nothing) - * when Node's operation would not write the array. */ +/* Writes into published or frozen objects (emitted guards): Node's + * frozen-object TypeErrors. The array guards answer false (and throw + * nothing) when Node's operation would not write the array; their ops are + * scr_array_frozen_error's (scr_array.h). */ struct ScrArr; SCR_COLD void scr_throw_published_field(const char *prop, const char *owner); bool scr_throw_published_array(int op, const struct ScrArr *a, double index, double n); +bool scr_throw_published_array_range(int op, const struct ScrArr *a, double x, double y, double z); SCR_COLD void scr_throw_published_collection(bool set); /* Young-in nursery (scr_cycle.c built with SCR_YIN, linked by programs diff --git a/packages/runtime/src/scr_stream.c b/packages/runtime/src/scr_stream.c index 275c03039..cd95e12cd 100644 --- a/packages/runtime/src/scr_stream.c +++ b/packages/runtime/src/scr_stream.c @@ -269,6 +269,7 @@ struct ScrStreamState { * destroyed and runs its _destroy once construction completes. */ bool constructing, destroy_deferred; bool construct_finished; /* onConstruct ran: a second callback is ERR_MULTIPLE_CALLBACK */ + bool construct_called; /* the construct tick called the hook */ ScrError *destroy_err; /* owned: the deferred destroy's error */ ScrClosure *construct_cb; ScrStreamPlainInv construct_inv; @@ -2157,6 +2158,24 @@ static ScrDyn *scr_stream_wentry_encoding(const ScrStreamState *st, ScrDyn *enc) return scr_dyn_retain(enc ? enc : scr_dyn_undefined()); } +/* A compiled writev whose entries declare `encoding` a string: an entry + * without one (an object-mode write passes undefined or null) gets "". + * The batch array is the one this call receives. */ +void scr_stream_writev_fill_encodings(ScrDyn *chunks) { + if (!chunks || chunks->kind != SCR_DYN_ARR) return; + double n = scr_dyn_arr_len(chunks); + for (double i = 0; i < n; i++) { + ScrDyn *entry = scr_dyn_arr_at(chunks, i); /* +1 */ + ScrDyn *enc = entry->kind == SCR_DYN_OBJ ? scr_dyn_obj_get(entry, "encoding", 8) : NULL; + if (entry->kind == SCR_DYN_OBJ && (!enc || enc->kind == SCR_DYN_UNDEF || enc->kind == SCR_DYN_NULL)) { + ScrStr *empty = scr_str_new("", 0); + scr_dyn_obj_set(entry, "encoding", 8, scr_dyn_new_str(empty)); + scr_str_release(empty); + } + scr_dyn_release(entry); + } +} + /* A batch entry's callback: the write's callback, or Node's nop. */ static ScrDyn *scr_stream_writev_entry_cb_call(ScrClosure *clo, ScrDyn *const *args, size_t argc) { (void)args; @@ -2675,9 +2694,8 @@ void *scr_stream_native(ScrStream *s, void (*release)(void *)) { * way the compiler-emitted dyn thunks do (chunk as Buffer-flavored bytes, * encoding 'buffer', the error via the boundary encoding, the completion * callback as a callable dyn over the *_done glue). Options that Node's - * base WOULD consume but have no lowering here (objectMode truthy, - * decodeStrings:false, non-utf8 defaultEncoding, signal, construct, - * writev, and read/write/final on the transformish bases) throw the + * base WOULD consume but have no lowering here (signal, and + * read/write/final on the transformish bases) throw the * "not supported yet" Error — the compile-time fence's runtime twin. */ /* The boxed dyn callable (+1 — scr_box_get_ref retains; callers release @@ -2903,11 +2921,8 @@ static bool scr_stream_parse_dyn_opts(const ScrDyn *opts, char base, ScrStreamDy } } out->construct = scr_stream_dynopt_fn(opts, "construct", 9); + /* a Transform's batches reach writev, not the transform, as in Node */ if (base != 'r') out->writev = scr_stream_dynopt_fn(opts, "writev", 6); - if (transformish && out->writev != NULL) { - static const char m[] = "the stream option 'writev' on a Transform is not supported yet (batches would bypass the transform)"; - return scr_stream_dynopt_unsupported(m, sizeof m - 1); - } { /* a known encoding or ERR_UNKNOWN_ENCODING (the states' constructors) */ ScrDyn *v = scr_dyn_obj_get((ScrDyn *)opts, "defaultEncoding", 15); @@ -6296,6 +6311,55 @@ ScrStream *scr_stream_set_default_encoding(ScrStream *s, ScrDyn *enc) { return scr_stream_retain(s); } +/* Whether a defaultEncoding option value is the default: null, undefined, + * 'utf8' or 'utf-8' (the state constructors' kDefaultUTF8Encoding). */ +static bool scr_stream_default_encoding_value(const ScrDyn *v) { + if (!v || v->kind == SCR_DYN_UNDEF || v->kind == SCR_DYN_NULL) return true; + return v->kind == SCR_DYN_STR && ((v->v.str->len == 4 && memcmp(v->v.str->data, "utf8", 4) == 0) || + (v->v.str->len == 5 && memcmp(v->v.str->data, "utf-8", 5) == 0)); +} + +/* A runtime defaultEncoding option value, checked before construction as + * the state constructors check it: the default, a name Buffer.isEncoding + * knows, or ERR_UNKNOWN_ENCODING (thrown). Borrows v. */ +void scr_stream_check_encoding_option(ScrDyn *v) { + if (scr_stream_default_encoding_value(v)) return; + if (v->kind == SCR_DYN_STR) { + ScrStr *canonical = scr_bytes_normalize_encoding(v->v.str); + if (canonical) scr_str_release(canonical); + return; + } + ScrStr *shown = scr_dyn_string_coerce_js(v); + if (!shown) return; + char msg[128]; + int len = snprintf(msg, sizeof msg, "Unknown encoding: %.*s", (int)(shown->len < 64 ? shown->len : 64), + shown->data); + scr_str_release(shown); + scr_throw_error_msg_code(SCR_ERR_TYPE, msg, (size_t)(len < 0 ? 0 : len), "ERR_UNKNOWN_ENCODING"); +} + +/* Runtime decodeStrings and defaultEncoding option values over a fresh + * stream (after its objectMode): decodeStrings === false keeps string + * writes, and a known defaultEncoding is the write side's default as + * given and how a byte-mode readable side decodes push(string). Borrows + * both; answers the receiver (+1). */ +ScrStream *scr_stream_option_values(ScrStream *s, ScrDyn *decode_strings, ScrDyn *default_encoding) { + ScrStreamState *st = s->st; + if (st->has_w && decode_strings && decode_strings->kind == SCR_DYN_BOOL && !decode_strings->v.b) + scr_stream_set_decode_strings(s, false); + if (!scr_stream_default_encoding_value(default_encoding) && default_encoding->kind == SCR_DYN_STR) { + ScrStr *canonical = scr_bytes_normalize_encoding(default_encoding->v.str); + if (canonical) { + scr_stream_set_write_encoding(s, default_encoding->v.str); + if (st->has_r && !st->r.object_entries) scr_stream_release(scr_stream_set_push_encoding(s, canonical)); + scr_str_release(canonical); + } else if (scr_exc_pending()) { + scr_exc_clear(); /* checked before construction */ + } + } + return scr_stream_retain(s); +} + /* stream.objectMode: scr_stream_set_object_mode, answering the receiver * (+1). */ ScrStream *scr_stream_object_mode(ScrStream *s, bool readable, bool writable) { @@ -7460,6 +7524,17 @@ void scr_stream_set_construct(ScrStream *s, ScrClosure *cb /*moves*/, ScrStreamP scr_stream_construct(s, cb, inv); } +void scr_stream_replace_construct(ScrStream *s, ScrClosure *cb /*moves*/, ScrStreamPlainInv inv) { + ScrStreamState *st = s->st; + if (!st->constructing || st->construct_called || !st->construct_cb) { + scr_closure_release(cb); + return; + } + scr_closure_release(st->construct_cb); + st->construct_cb = cb; + st->construct_inv = inv; +} + /* A class's _writev under options that may carry their own writev (the * option, an instance property in Node, wins). */ void scr_stream_set_writev_default(ScrStream *s, ScrClosure *cb /*moves*/, ScrStreamWritevInv inv) { @@ -7728,6 +7803,7 @@ static void scr_stream_run_tick(ScrStreamTick *t) { scr_stream_error_buffer(s); break; case SCR_ST_CONSTRUCT: { + st->construct_called = true; if (st->construct_cb) { st->construct_inv(st->construct_cb, s); /* a throw inside _construct reaches onConstruct on a tick */ diff --git a/packages/runtime/src/scr_stream.h b/packages/runtime/src/scr_stream.h index d26077b5c..a4798c629 100644 --- a/packages/runtime/src/scr_stream.h +++ b/packages/runtime/src/scr_stream.h @@ -561,8 +561,15 @@ void scr_stream_set_writev_concat(ScrStream *s); /* The user construct option / _construct method (scheduled now; cb * moves), and the writev option / _writev method (cb moves). */ void scr_stream_set_construct(ScrStream *s, ScrClosure *cb, ScrStreamPlainInv inv); +/* `s._construct = fn` after construction: replaces a construct hook whose + * tick has not run yet (Node's constructNT reads _construct then), and has + * no effect otherwise (cb moves). */ +void scr_stream_replace_construct(ScrStream *s, ScrClosure *cb, ScrStreamPlainInv inv); void scr_stream_set_writev(ScrStream *s, ScrClosure *cb, ScrStreamWritevInv inv); void scr_stream_set_writev_default(ScrStream *s, ScrClosure *cb, ScrStreamWritevInv inv); +/* A compiled writev whose entries declare `encoding` a string: entries + * without an encoding (object mode) get "" before the conversion. */ +void scr_stream_writev_fill_encodings(struct ScrDyn *chunks); /* Duplex.from (duplexify.js): over an async generator function (the * compiled stage takes the source generator that the compiled source * makes from next() and ack(); all borrowed, helpers too), and over a @@ -605,6 +612,11 @@ ScrStream *scr_stream_duplex_from_promise(struct ScrPromise *p, ScrClosure *box) * forms answer the stream (+1). */ void scr_stream_set_decode_strings(ScrStream *s, bool decode); ScrStream *scr_stream_decode_strings(ScrStream *s, bool decode); +/* Runtime decodeStrings/defaultEncoding option values: checked before + * construction (ERR_UNKNOWN_ENCODING thrown), then applied to the fresh + * stream, answering it (+1). Values borrowed. */ +void scr_stream_check_encoding_option(struct ScrDyn *v); +ScrStream *scr_stream_option_values(ScrStream *s, struct ScrDyn *decode_strings, struct ScrDyn *default_encoding); void scr_stream_set_write_encoding(ScrStream *s, ScrStr *enc); ScrStream *scr_stream_write_encoding(ScrStream *s, ScrStr *enc); ScrStream *scr_stream_set_default_encoding(ScrStream *s, struct ScrDyn *enc); diff --git a/packages/runtime/src/scr_string.c b/packages/runtime/src/scr_string.c index f94554642..37b92e48c 100644 --- a/packages/runtime/src/scr_string.c +++ b/packages/runtime/src/scr_string.c @@ -1928,6 +1928,22 @@ static __attribute__((noinline)) double scr_str_unit_search(const ScrStr *s, con /* Search backwards from a clamped UTF-16 position. A low-surrogate position * maps to its scalar's first byte, so that scalar remains searchable. */ double scr_str_last_index_of_from(ScrStr *s, ScrStr *needle, double position) { + /* A proven-ASCII receiver: byte offsets are UTF-16 indexes, and a needle + * with other bytes (including lone surrogate halves) is never found. */ + if (s->ascii) { + size_t start = isnan(position) || position >= (double)s->len ? s->len + : position <= 0 ? 0 : (size_t)trunc(position); + if (needle->len == 0) return (double)start; + if (needle->len > s->len) return -1.0; + size_t max_start = s->len - needle->len; + if (start < max_start) max_start = start; + char first = needle->data[0]; + for (size_t i = max_start + 1; i-- > 0;) { + if (s->data[i] == first && memcmp(s->data + i, needle->data, needle->len) == 0) + return (double)i; + } + return -1.0; + } ScrSidx *e = scr_sidx_for(s); size_t len16 = scr_sidx_len(s, e); size_t start16 = isnan(position) || position >= (double)len16 ? len16 diff --git a/packages/runtime/src/scr_text_codec_dyn.c b/packages/runtime/src/scr_text_codec_dyn.c new file mode 100644 index 000000000..5102bc558 --- /dev/null +++ b/packages/runtime/src/scr_text_codec_dyn.c @@ -0,0 +1,215 @@ +/* WHATWG TextEncoder and TextDecoder instances as checked-dynamic values + * (SCR_DYNH_TEXT_ENCODER, SCR_DYNH_TEXT_DECODER). A codec is a compiled + * record (type-mapper.ts): the handle holds that record by reference, so + * a codec stored in an untyped slot and read back keeps its identity and + * a decoder keeps its streaming state. The compiler passes the record's + * reference-count entry points and, for decoders, the byte offsets of the + * encoding, fatal, ignoreBOM and state fields when it boxes one. A unit + * of its own so programs that never box a codec link none of it + * (native-toolchain gating: the textCodecDyn runtime feature). */ +#include "scr_platform.h" +#include "scr_runtime.h" + +#include + +typedef struct { + void *(*retain)(void *); + void (*release)(void *); + size_t encoding, fatal, ignore_bom, state; +} ScrTextCodecLayout; + +/* A program has one record shape per codec class, so one layout each. */ +static SCR_TL ScrTextCodecLayout scr_tcd_encoder_layout, scr_tcd_decoder_layout; + +static void *scr_tcd_encoder_retain(void *rec) { return scr_tcd_encoder_layout.retain(rec); } +static void scr_tcd_encoder_release(void *rec) { scr_tcd_encoder_layout.release(rec); } +static void *scr_tcd_decoder_retain(void *rec) { return scr_tcd_decoder_layout.retain(rec); } +static void scr_tcd_decoder_release(void *rec) { scr_tcd_decoder_layout.release(rec); } + +static void scr_tcd_codec_not_a_function(const char *what) { + ScrJsonBuf b; + scr_jb_init(&b); + scr_jb_puts(&b, what); + scr_jb_puts(&b, " is not a function"); + scr_throw_error(SCR_ERR_TYPE, scr_jb_finish(&b)); +} + +/* Boxes an owned string (+1 box; the string's reference moves in). */ +static ScrDyn *scr_tcd_str(ScrStr *text) { + ScrDyn *boxed = scr_dyn_new_str(text); + scr_str_release(text); + return boxed; +} + +static ScrDyn *scr_tcd_codec_arg(ScrDyn *const *args, size_t argc, size_t index) { + return index < argc ? args[index] : scr_dyn_undefined(); +} + +static double scr_tcd_decoder_encoding(const void *rec) { + double encoding; + memcpy(&encoding, (const char *)rec + scr_tcd_decoder_layout.encoding, sizeof encoding); + return encoding; +} + +static bool scr_tcd_decoder_flag(const void *rec, size_t offset) { + return ((const unsigned char *)rec)[offset] != 0; +} + +static ScrDyn *scr_tcd_encoder_invoke(void *rec, ScrDyn *self, const char *method, + ScrDyn *const *args, size_t argc, const char *what) { + (void)rec; + (void)self; + if (strcmp(method, "encode") == 0) { + /* encode(input = ""): any other input converts with ToString. */ + ScrDyn *input = scr_tcd_codec_arg(args, argc, 0); + ScrStr *text = + input->kind == SCR_DYN_UNDEF ? scr_str_new("", 0) : scr_dyn_string_coerce_js(input); + if (!text) return NULL; + ScrStr *utf8 = scr_str_new("utf8", 4); + ScrBytes *bytes = scr_bytes_from_str(text, utf8); + scr_str_release(utf8); + scr_str_release(text); + /* A fresh Uint8Array, not a Buffer. */ + bytes->is_buffer = false; + ScrDyn *result = scr_dyn_new_bytes(bytes); + scr_bytes_release(bytes); + return result; + } + if (strcmp(method, "encodeInto") == 0) + return scr_text_encode_into(scr_tcd_codec_arg(args, argc, 0), + scr_tcd_codec_arg(args, argc, 1)); + scr_tcd_codec_not_a_function(what); + return NULL; +} + +static ScrDyn *scr_tcd_encoder_get(void *rec, const char *key, size_t len) { + (void)rec; + if (len == 8 && memcmp(key, "encoding", 8) == 0) return scr_tcd_str(scr_str_new("utf-8", 5)); + return NULL; +} + +/* decode(input, options): the options object's `stream` member, read + * with JS truthiness; undefined and null mean no options. */ +static bool scr_tcd_decoder_stream_option(const ScrDyn *options, bool *stream) { + *stream = false; + if (options->kind == SCR_DYN_UNDEF || options->kind == SCR_DYN_NULL) return true; + if (options->kind != SCR_DYN_OBJ && options->kind != SCR_DYN_ARR && + options->kind != SCR_DYN_FUNC && options->kind != SCR_DYN_TYPED_REF && + options->kind != SCR_DYN_HANDLE) { + scr_dyn_arg_type_fail("options", "of type object", options); + return false; + } + ScrDyn *value = scr_dyn_obj_read(options, "stream", 6); + if (!value) return false; + *stream = scr_dyn_truthy(value); + scr_dyn_release(value); + return !scr_exc_pending(); +} + +static ScrDyn *scr_tcd_decoder_invoke(void *rec, ScrDyn *self, const char *method, + ScrDyn *const *args, size_t argc, const char *what) { + (void)self; + if (strcmp(method, "decode") != 0) { + scr_tcd_codec_not_a_function(what); + return NULL; + } + ScrBytes *input = scr_bytes_buffer_source(scr_tcd_codec_arg(args, argc, 0)); + if (!input) return NULL; + bool stream; + if (!scr_tcd_decoder_stream_option(scr_tcd_codec_arg(args, argc, 1), &stream)) { + scr_bytes_release(input); + return NULL; + } + const ScrTextCodecLayout *layout = &scr_tcd_decoder_layout; + ScrDyn *state; + memcpy(&state, (const char *)rec + layout->state, sizeof state); + ScrStr *text = scr_text_decode_stream(state, input, scr_tcd_decoder_encoding(rec), + scr_tcd_decoder_flag(rec, layout->fatal), + scr_tcd_decoder_flag(rec, layout->ignore_bom), stream); + scr_bytes_release(input); + return text ? scr_tcd_str(text) : NULL; +} + +static ScrDyn *scr_tcd_decoder_get(void *rec, const char *key, size_t len) { + if (len == 8 && memcmp(key, "encoding", 8) == 0) + return scr_tcd_str(scr_text_decoder_name(scr_tcd_decoder_encoding(rec))); + if (len == 5 && memcmp(key, "fatal", 5) == 0) + return scr_dyn_new_bool(scr_tcd_decoder_flag(rec, scr_tcd_decoder_layout.fatal)); + if (len == 9 && memcmp(key, "ignoreBOM", 9) == 0) + return scr_dyn_new_bool(scr_tcd_decoder_flag(rec, scr_tcd_decoder_layout.ignore_bom)); + return NULL; +} + +static bool scr_tcd_codec_set(void *rec, const char *key, size_t len, const ScrDyn *value) { + (void)rec; + (void)key; + (void)len; + (void)value; + return false; +} + +static const ScrDynHandleMethod scr_tcd_encoder_methods[] = { + {.name = "encode", .length = 0}, + {.name = "encodeInto", .length = 2}, + {.name = NULL}, +}; + +static const ScrDynHandleMethod scr_tcd_decoder_methods[] = { + {.name = "decode", .length = 0}, + {.name = NULL}, +}; + +static const ScrDynHandleOps scr_tcd_encoder_ops = { + .cls = "TextEncoder", + .to_string_tag = "TextEncoder", + .retain = scr_tcd_encoder_retain, + .release = scr_tcd_encoder_release, + .invoke = scr_tcd_encoder_invoke, + .get = scr_tcd_encoder_get, + .set = scr_tcd_codec_set, + .json_empty_object = true, + .methods = scr_tcd_encoder_methods, + .invalid_this = "Value of \"this\" must be of type TextEncoder", +}; + +static const ScrDynHandleOps scr_tcd_decoder_ops = { + .cls = "TextDecoder", + .to_string_tag = "TextDecoder", + .retain = scr_tcd_decoder_retain, + .release = scr_tcd_decoder_release, + .invoke = scr_tcd_decoder_invoke, + .get = scr_tcd_decoder_get, + .set = scr_tcd_codec_set, + .json_empty_object = true, + .methods = scr_tcd_decoder_methods, + .invalid_this = "Value of \"this\" must be of type TextDecoder", +}; + +ScrDyn *scr_text_encoder_to_dyn(void *rec, void *(*retain)(void *), void (*release)(void *)) { + if (!scr_tcd_encoder_layout.retain) { + scr_tcd_encoder_layout.retain = retain; + scr_tcd_encoder_layout.release = release; + scr_dyn_handle_install(SCR_DYNH_TEXT_ENCODER, &scr_tcd_encoder_ops); + } + return scr_dyn_new_handle(rec, SCR_DYNH_TEXT_ENCODER); +} + +ScrDyn *scr_text_decoder_to_dyn(void *rec, void *(*retain)(void *), void (*release)(void *), + size_t encoding, size_t fatal, size_t ignore_bom, size_t state) { + if (!scr_tcd_decoder_layout.retain) { + scr_tcd_decoder_layout = (ScrTextCodecLayout){retain, release, encoding, fatal, ignore_bom, state}; + scr_dyn_handle_install(SCR_DYNH_TEXT_DECODER, &scr_tcd_decoder_ops); + } + return scr_dyn_new_handle(rec, SCR_DYNH_TEXT_DECODER); +} + +void *scr_text_codec_from_dyn(const ScrDyn *d, bool decoder, const ScrDynPath *path, + const char *want) { + return scr_dyn_handle_unbox(d, decoder ? SCR_DYNH_TEXT_DECODER : SCR_DYNH_TEXT_ENCODER, path, + want); +} + +bool scr_text_codec_is(const ScrDyn *d, bool decoder) { + return d->kind == SCR_DYN_HANDLE && + d->v.handle.tag == (decoder ? SCR_DYNH_TEXT_DECODER : SCR_DYNH_TEXT_ENCODER); +} diff --git a/packages/runtime/test/test_json.c b/packages/runtime/test/test_json.c index 2c072006f..865f1c757 100644 --- a/packages/runtime/test/test_json.c +++ b/packages/runtime/test/test_json.c @@ -551,6 +551,91 @@ static void test_class_values(void) { scr_str_release(name); } +/* The class-view copy (scr_dyn_copy_property_descriptors) gives each + * property of the bag the attributes, value or accessors, and own-key + * position that defining it from its own descriptor would. */ +static ScrDyn *define_from(ScrDyn *target, const char *key, ScrDyn *value, bool writable, + bool enumerable, bool configurable) { + ScrDyn *descriptor = scr_dyn_new_obj(); + scr_dyn_obj_set(descriptor, "value", 5, value); + scr_dyn_obj_set(descriptor, "writable", 8, scr_dyn_new_bool(writable)); + scr_dyn_obj_set(descriptor, "enumerable", 10, scr_dyn_new_bool(enumerable)); + scr_dyn_obj_set(descriptor, "configurable", 12, scr_dyn_new_bool(configurable)); + ScrStr *name = S(key); + ScrDyn *k = scr_dyn_new_str(name); + ScrDyn *result = scr_dyn_define_property(target, k, descriptor); + scr_dyn_release(k); + scr_str_release(name); + scr_dyn_release(descriptor); + return result; +} + +static const ScrDynEntry *entry_of(const ScrDyn *obj, const char *key) { + for (size_t i = 0; i < obj->v.obj.len; i++) + if (obj->v.obj.entries[i].key_len == strlen(key) && + memcmp(obj->v.obj.entries[i].key, key, strlen(key)) == 0) + return &obj->v.obj.entries[i]; + return NULL; +} + +static void test_copy_property_descriptors(void) { + ScrDyn *bag = scr_dyn_new_obj(); + ScrDyn *getter = scr_dyn_new_obj(); + scr_dyn_release(define_from(bag, "label", scr_dyn_new_num(1), true, true, true)); + scr_dyn_release(define_from(bag, "hidden", scr_dyn_new_num(2), false, false, true)); + scr_dyn_obj_set(bag, "7", 1, scr_dyn_new_num(7)); + scr_dyn_obj_set(bag, "3", 1, scr_dyn_new_num(3)); + /* An accessor entry (the getter object stands in for its function). */ + scr_dyn_obj_set(bag, "computed", 8, scr_dyn_retain(scr_dyn_undefined())); + ScrDynEntry *accessor = (ScrDynEntry *)entry_of(bag, "computed"); + accessor->accessor = true; + accessor->getter = scr_dyn_retain(getter); + accessor->writable = false; + accessor->enumerable = false; + accessor->configurable = false; + + ScrDyn *view = scr_dyn_new_obj(); + scr_dyn_release(scr_dyn_copy_property_descriptors(view, bag)); + check(!scr_exc_pending() && view->v.obj.len == 5, "copy keeps every bag property"); + ScrDyn *keys = scr_dyn_obj_own_keys(view); + const char *order[] = {"3", "7", "label", "hidden", "computed"}; + bool ordered = keys->v.arr.len == 5; + for (size_t i = 0; ordered && i < 5; i++) ordered = str_is(keys->v.arr.items[i]->v.str, order[i]); + check(ordered, "copy keeps the bag's own-key order"); + scr_dyn_release(keys); + const ScrDynEntry *hidden = entry_of(view, "hidden"); + check(hidden && !hidden->accessor && hidden->value->v.num == 2 && !hidden->writable && + !hidden->enumerable && hidden->configurable, + "copy keeps data attributes"); + const ScrDynEntry *copied = entry_of(view, "computed"); + check(copied && copied->accessor && copied->getter == getter && !copied->setter && + !copied->writable && !copied->enumerable && !copied->configurable && + copied->value->kind == SCR_DYN_UNDEF, + "copy keeps an accessor without calling it"); + + /* A property the target already has is redefined like defineProperty. */ + ScrDyn *existing = scr_dyn_new_obj(); + scr_dyn_obj_set(existing, "label", 5, scr_dyn_new_num(9)); + scr_dyn_release(scr_dyn_copy_property_descriptors(existing, bag)); + const ScrDynEntry *label = entry_of(existing, "label"); + check(!scr_exc_pending() && existing->v.obj.len == 5 && existing->v.obj.entries[0].key_len == 5 && + label->value->v.num == 1, + "copy redefines an existing property in place"); + + /* A target that takes no new properties throws as defineProperty does. */ + ScrDyn *closed = scr_dyn_new_obj(); + scr_dyn_release(scr_dyn_prevent_extensions(closed)); + ScrDyn *refused = scr_dyn_copy_property_descriptors(closed, bag); + check(!refused && scr_exc_pending() && closed->v.obj.len == 0, "copy onto a non-extensible object throws"); + scr_exc_clear(); + + scr_dyn_release(closed); + scr_dyn_release(existing); + scr_dyn_release(view); + scr_dyn_release(bag); + scr_dyn_release(getter); +} + static void test_primitive_owners(void) { ScrDyn *values[] = {scr_dyn_new_null(), scr_dyn_new_bool(false), scr_dyn_new_bool(true)}; const char *texts[] = {"null", "false", "true"}; @@ -579,6 +664,7 @@ int main(void) { test_class_values(); scr_init(); test_primitive_owners(); + test_copy_property_descriptors(); /* ── primitives ─────────────────────────────────────────────────── */ ScrDyn *d = parse_ok("null", "parse null"); diff --git a/packages/threads/package.json b/packages/threads/package.json index 11b8ef624..a37a3ba82 100644 --- a/packages/threads/package.json +++ b/packages/threads/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/threads", - "version": "0.4.2", + "version": "0.4.3", "description": "Zero-copy publication of immutable object graphs to worker threads (scriptc), with a deep-freezing Node.js implementation", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/packages/types/package.json b/packages/types/package.json index 689c7b72f..906c68dec 100644 --- a/packages/types/package.json +++ b/packages/types/package.json @@ -1,6 +1,6 @@ { "name": "@scriptc/types", - "version": "0.4.2", + "version": "0.4.3", "description": "Contract types for scriptc: type-only annotations (Int32, NoThrow, Immutable) that scriptc checks and optimizes, with no runtime semantics", "license": "Apache-2.0", "homepage": "https://scriptc.dev", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 525994ba1..5fc32620a 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -110,8 +110,8 @@ importers: specifier: workspace:* version: link:../runtime tsc-ts: - specifier: 0.0.8 - version: 0.0.8 + specifier: 0.0.9 + version: 0.0.9 typescript: specifier: 7.0.2 version: 7.0.2 @@ -1377,42 +1377,42 @@ packages: '@ts-morph/common@0.11.1': resolution: {integrity: sha512-7hWZS0NRpEsNV8vWJzg7FEz6V8MaLNeJOmwmghqUXTpzk16V1LLZhdo+4QvE/+zv4cVci0OviuJFnqhEfoV3+g==} - '@tsc-ts/darwin-arm64@0.0.8': - resolution: {integrity: sha512-+hB9yWk58lwZHGrVEzqEC4D7MQW7Vd2z7blV3TULIKZPTMwdf8aT7XFAmfP48PoGff0H9yuQKyRXJl9I+sQv9w==} + '@tsc-ts/darwin-arm64@0.0.9': + resolution: {integrity: sha512-aIjSulef3DIEX7F3RRWLXyHWFgjXiVffBkrfE83ay/tpbsQaniu16cFxYGYNaqqVGft/oivecpdTDBbpPlsb8Q==} cpu: [arm64] os: [darwin] - '@tsc-ts/darwin-x64@0.0.8': - resolution: {integrity: sha512-yz4UddicqMXe+T7nV7ftpf155lkiK6gIPNwSB8rH9dlwiV6a17CO/vpSk627QLMqnfGNicB6YwLMk/SJ9moPdw==} + '@tsc-ts/darwin-x64@0.0.9': + resolution: {integrity: sha512-LHE74tzIEDtrEN8ENvCH/ejnnZjT0D2Vaz2HmggcvJudgr0W300CcwGf2Kdfw8MVBBk6JN0HPF9+a5AUUtFZGg==} cpu: [x64] os: [darwin] - '@tsc-ts/linux-arm64-gnu@0.0.8': - resolution: {integrity: sha512-LHqfT9mQ4tusPdWmnanCvC8/YBOF5bHM5h5863a3t2HTgFVdps1Qwb0yzrW5DDbojIhQ0Z+GupxtXVUQcV0Q/Q==} + '@tsc-ts/linux-arm64-gnu@0.0.9': + resolution: {integrity: sha512-dNiQIKApLcKIJPzb3pk2gfGrrgmOWa+q3/qjrx5ia/SzIWz0+lENlVvHiIn9ukqDacDE30FW8xHzby8Kyd+10A==} cpu: [arm64] os: [linux] libc: [glibc] - '@tsc-ts/linux-arm64-musl@0.0.8': - resolution: {integrity: sha512-I0CC3xZMwoQM6OiVsubRHb0VI/RrJpneb33kDQhadYHq3taPos71KwP+CbCXNu88rNsDsyn7D1oCVfJb73g4Mw==} + '@tsc-ts/linux-arm64-musl@0.0.9': + resolution: {integrity: sha512-DolAqHtr1GUyG03Ch8+sJzpX1G+xzazAmj18kABzuOPre/6njrSWnmqJmtXQWvhwLBBEjajIkUJ4V+t5qigsrg==} cpu: [arm64] os: [linux] libc: [musl] - '@tsc-ts/linux-x64-gnu@0.0.8': - resolution: {integrity: sha512-PLERuwUuxT92x0BZAMQubiQP5jhlqoLmswaZWqcvfKAZyBp0TbWiVOmuNnDbJozvBaNUrfdGPb9suiQ+Me7M6A==} + '@tsc-ts/linux-x64-gnu@0.0.9': + resolution: {integrity: sha512-leANVZ4tKz3myBSVogb173ttbAurPOOGjy1WeBlRhkVJ7zOPMiyzJi3cPOAK/4O8RcxXOsCiC2AfKPKDYM8bEw==} cpu: [x64] os: [linux] libc: [glibc] - '@tsc-ts/linux-x64-musl@0.0.8': - resolution: {integrity: sha512-zDl36l1uyWc0mQniOAMZmkVD9TrlFExHlDT0fL8JGD4XLgx+bRe+To38yP0KSBehXBzjDNn8xrEoJ+3RP6mYwQ==} + '@tsc-ts/linux-x64-musl@0.0.9': + resolution: {integrity: sha512-a4FW16QXkrwNHAr4Qo/qDA5w9LxCJNrYbpqiz3TJ1sePTqvW/JTJc14tjx6avhW4mIaQp8AB52m7TqNAuDMm4Q==} cpu: [x64] os: [linux] libc: [musl] - '@tsc-ts/win32-x64-msvc@0.0.8': - resolution: {integrity: sha512-9gO+pFoLiIxITlAE2P6JL8ihZwns7z/2Raw5+38RqLzHmnSsRN47+H4p/4EqQsLW5/TTsAo9uSeSkf3EuRg+XA==} + '@tsc-ts/win32-x64-msvc@0.0.9': + resolution: {integrity: sha512-phThYnmOinlvY4dS4DEl1L9t/mFLiHR1zL5plSUfMB5NizA/jESfgMGRu+HW3uwOpCETSxgsuTNAXO7UuEIHIQ==} cpu: [x64] os: [win32] @@ -2577,8 +2577,8 @@ packages: ts-toolbelt@6.15.5: resolution: {integrity: sha512-FZIXf1ksVyLcfr7M317jbB67XFJhOO1YqdTcuGaq9q5jLUoTikukZ+98TPjKiP2jC5CgmYdWWYs0s2nLSU0/1A==} - tsc-ts@0.0.8: - resolution: {integrity: sha512-EiyDziYOryBg/YEvev/mKCn+hD4M6VFjH7VP9nv2WadDKYaPrdb+a3hP+wYr/UsDWNE7AdL7YGnjZppeXlXy6g==} + tsc-ts@0.0.9: + resolution: {integrity: sha512-7fDfC/o1YoITG6sSXj4ZznijQYg8AfoCagmYvwG1BFLR/hz4Lwnq7jqDwcZfoDuPYTkrs8jHUTAVG1turniJHA==} engines: {node: '>=20'} hasBin: true @@ -3454,25 +3454,25 @@ snapshots: mkdirp: 1.0.4 path-browserify: 1.0.1 - '@tsc-ts/darwin-arm64@0.0.8': + '@tsc-ts/darwin-arm64@0.0.9': optional: true - '@tsc-ts/darwin-x64@0.0.8': + '@tsc-ts/darwin-x64@0.0.9': optional: true - '@tsc-ts/linux-arm64-gnu@0.0.8': + '@tsc-ts/linux-arm64-gnu@0.0.9': optional: true - '@tsc-ts/linux-arm64-musl@0.0.8': + '@tsc-ts/linux-arm64-musl@0.0.9': optional: true - '@tsc-ts/linux-x64-gnu@0.0.8': + '@tsc-ts/linux-x64-gnu@0.0.9': optional: true - '@tsc-ts/linux-x64-musl@0.0.8': + '@tsc-ts/linux-x64-musl@0.0.9': optional: true - '@tsc-ts/win32-x64-msvc@0.0.8': + '@tsc-ts/win32-x64-msvc@0.0.9': optional: true '@tybys/wasm-util@0.10.3': @@ -4813,17 +4813,17 @@ snapshots: ts-toolbelt@6.15.5: {} - tsc-ts@0.0.8: + tsc-ts@0.0.9: dependencies: '@scriptc/threads': 0.4.2 optionalDependencies: - '@tsc-ts/darwin-arm64': 0.0.8 - '@tsc-ts/darwin-x64': 0.0.8 - '@tsc-ts/linux-arm64-gnu': 0.0.8 - '@tsc-ts/linux-arm64-musl': 0.0.8 - '@tsc-ts/linux-x64-gnu': 0.0.8 - '@tsc-ts/linux-x64-musl': 0.0.8 - '@tsc-ts/win32-x64-msvc': 0.0.8 + '@tsc-ts/darwin-arm64': 0.0.9 + '@tsc-ts/darwin-x64': 0.0.9 + '@tsc-ts/linux-arm64-gnu': 0.0.9 + '@tsc-ts/linux-arm64-musl': 0.0.9 + '@tsc-ts/linux-x64-gnu': 0.0.9 + '@tsc-ts/linux-x64-musl': 0.0.9 + '@tsc-ts/win32-x64-msvc': 0.0.9 tslib@2.8.1: optional: true diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index e74d40442..c9f873c4e 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -7,14 +7,14 @@ minimumReleaseAgeExclude: - "effect@4.0.0" - "@effect/platform-node@4.0.0" - "@effect/platform-node-shared@4.0.0" - - "@tsc-ts/darwin-arm64@0.0.8" - - "@tsc-ts/darwin-x64@0.0.8" - - "@tsc-ts/linux-arm64-gnu@0.0.8" - - "@tsc-ts/linux-arm64-musl@0.0.8" - - "@tsc-ts/linux-x64-gnu@0.0.8" - - "@tsc-ts/linux-x64-musl@0.0.8" - - "@tsc-ts/win32-x64-msvc@0.0.8" - - "tsc-ts@0.0.8" + - "@tsc-ts/darwin-arm64@0.0.9" + - "@tsc-ts/darwin-x64@0.0.9" + - "@tsc-ts/linux-arm64-gnu@0.0.9" + - "@tsc-ts/linux-arm64-musl@0.0.9" + - "@tsc-ts/linux-x64-gnu@0.0.9" + - "@tsc-ts/linux-x64-musl@0.0.9" + - "@tsc-ts/win32-x64-msvc@0.0.9" + - "tsc-ts@0.0.9" - "@scriptc/threads@0.4.2" allowBuilds: esbuild: true diff --git a/scripts/ci-test-costs.json b/scripts/ci-test-costs.json index 4c4964579..99d47e972 100644 --- a/scripts/ci-test-costs.json +++ b/scripts/ci-test-costs.json @@ -409,6 +409,7 @@ "tests/harness/stdlib-surface.test.ts": 48, "tests/harness/strip-vt-linear.test.ts": 1795, "tests/harness/surface-manifest.test.ts": 5309, + "tests/harness/teardown-segments.test.ts": 2600, "tests/harness/terminal-geometry.test.ts": 2730, "tests/harness/test262-runner.test.ts": 207, "tests/harness/three-library.test.ts": 251122, @@ -828,6 +829,7 @@ "tests/harness/stdlib-surface.test.ts": 46, "tests/harness/strip-vt-linear.test.ts": 2546, "tests/harness/surface-manifest.test.ts": 5951, + "tests/harness/teardown-segments.test.ts": 5, "tests/harness/terminal-geometry.test.ts": 3270, "tests/harness/test262-runner.test.ts": 230, "tests/harness/three-library.test.ts": 361470, diff --git a/scripts/gen-cipher-info.mjs b/scripts/gen-cipher-info.mjs new file mode 100644 index 000000000..06bd8bce3 --- /dev/null +++ b/scripts/gen-cipher-info.mjs @@ -0,0 +1,125 @@ +// Writes packages/runtime/src/scr_cipher_info.h from the running Node's +// crypto.getCipherInfo: one record per cipher, the names (lowercase) and nids +// that find it, and the keyLength and ivLength options each accepts. The +// candidate names are getCiphers() and `openssl list -cipher-algorithms` +// (OpenSSL's aliases and its legacy-provider ciphers, which getCiphers() +// leaves out). Run with the pinned Node: node scripts/gen-cipher-info.mjs +import crypto from "node:crypto"; +import { execFileSync } from "node:child_process"; +import fs from "node:fs"; + +const listed = execFileSync("openssl", ["list", "-cipher-algorithms"], { encoding: "utf8" }) + .match(/[A-Za-z0-9][A-Za-z0-9._-]*/g) + .filter((word) => !["Legacy", "Provided", "default", "legacy", "base", "fips"].includes(word)); +const candidates = new Set([...crypto.getCiphers(), ...listed]); +for (let nid = 1; nid < 1400; nid++) { + const info = crypto.getCipherInfo(nid); + if (info) candidates.add(info.name); +} + +function ivRule(name, info) { + const accepted = []; + for (let v = 0; v <= 70; v++) if (crypto.getCipherInfo(name, { ivLength: v })) accepted.push(v); + const large = !!crypto.getCipherInfo(name, { ivLength: 100000 }); + const own = info.ivLength ?? 0; + if (accepted.length === 0) return "SC_INFO_NONE"; + if (accepted.length === 1 && accepted[0] === own && !large) return "SC_INFO_EXACT"; + if (accepted.join() === "7,8,9,10,11,12,13" && !large) return "SC_INFO_CCM"; + if (accepted.join() === [...Array(15)].map((_, i) => i + 1).join() && !large) + return "SC_INFO_OCB"; + if (accepted.length === 71 && large) return "SC_INFO_ANY"; + throw new Error(`unclassified ivLength rule for ${name}: ${accepted}`); +} +function keyRule(name, info) { + const accepted = []; + for (let k = 0; k <= 80; k++) if (crypto.getCipherInfo(name, { keyLength: k })) accepted.push(k); + if (accepted.length === 0) return "SC_INFO_NONE"; + if (accepted.length === 1 && accepted[0] === info.keyLength) return "SC_INFO_EXACT"; + throw new Error(`unclassified keyLength rule for ${name}: ${accepted}`); +} + +const records = []; +const recordIndex = new Map(); +function recordOf(name, info) { + const key = JSON.stringify(info); + let index = recordIndex.get(key); + const rules = [keyRule(name, info), ivRule(name, info)]; + if (index === undefined) { + index = records.length; + records.push({ info, rules }); + recordIndex.set(key, index); + } else if (records[index].rules.join() !== rules.join()) { + throw new Error(`rules differ for ${name}`); + } + return index; +} + +const names = []; +for (const name of [...candidates].map((n) => n.toLowerCase()).sort()) { + if (names.length && names.at(-1)[0] === name) continue; + const info = crypto.getCipherInfo(name); + if (info) names.push([name, recordOf(name, info)]); +} +const nids = []; +for (let nid = 1; nid < 1400; nid++) { + const info = crypto.getCipherInfo(nid); + if (info) nids.push([nid, recordOf(info.name, info)]); +} + +const str = (s) => (s === undefined ? "NULL" : JSON.stringify(s)); +const num = (n) => (n === undefined ? -1 : n); +let out = `/* Generated by scripts/gen-cipher-info.mjs from Node ${process.version}'s + * crypto.getCipherInfo: every cipher OpenSSL names, with the fields Node + * reports (-1 or NULL for a field it leaves out) and the keyLength and + * ivLength options each accepts. Do not edit. */ +#ifndef SCR_CIPHER_INFO_H +#define SCR_CIPHER_INFO_H + +enum { SC_INFO_NONE, SC_INFO_EXACT, SC_INFO_CCM, SC_INFO_OCB, SC_INFO_ANY }; + +typedef struct { + const char *mode; + const char *name; + int nid; + int block_size; + int iv_length; + int key_length; + unsigned char key_rule; + unsigned char iv_rule; +} ScCipherInfo; + +static const ScCipherInfo sc_cipher_infos[] = { +`; +for (const { info, rules } of records) + out += ` {${str(info.mode)}, ${str(info.name)}, ${num(info.nid)}, ${num(info.blockSize)}, ${num(info.ivLength)}, ${info.keyLength}, ${rules[0]}, ${rules[1]}},\n`; +out += `}; + +/* Lowercase names, sorted, with their record. */ +static const struct { + const char *name; + unsigned short info; +} sc_cipher_info_names[] = { +`; +for (const [name, index] of names) out += ` {${JSON.stringify(name)}, ${index}},\n`; +out += `}; + +/* Nids, ascending, with their record. */ +static const struct { + unsigned short nid; + unsigned short info; +} sc_cipher_info_nids[] = { +`; +for (const [nid, index] of nids) out += ` {${nid}, ${index}},\n`; +out += `}; + +#endif +`; +for (const { info } of records) { + const keys = Object.keys(info).join(); + const expected = ["mode", "name", "nid", "blockSize", "ivLength", "keyLength"] + .filter((k) => k in info) + .join(); + if (keys !== expected) throw new Error(`unexpected key order ${keys}`); +} +fs.writeFileSync(new URL("../packages/runtime/src/scr_cipher_info.h", import.meta.url), out); +console.log(records.length, "records,", names.length, "names,", nids.length, "nids"); diff --git a/scripts/zero-cost-compare.mjs b/scripts/zero-cost-compare.mjs index 160e33438..f202c2ab7 100644 --- a/scripts/zero-cost-compare.mjs +++ b/scripts/zero-cost-compare.mjs @@ -394,8 +394,8 @@ export function dynamicTierSymbols(names) { * not yet traced to its typed root. bench/thread-slots, * bench/channel-messages and bench/worker-messages, added later, link them * through worker messages: - * node:worker_threads values are dyn objects. bench/dynamic-methods is - * untyped JavaScript. Every other bench workload is fully typed and must link none. The gate and + * node:worker_threads values are dyn objects. bench/dynamic-methods and + * bench/dynamic-properties are untyped JavaScript. Every other bench workload is fully typed and must link none. The gate and * tests/harness/bench-dynamic-tier.test.ts enforce that, and the test also * fails when a listed workload stops linking them, so remove a workload here * when a leak fix makes its binary clean. */ @@ -410,6 +410,7 @@ export const DYNAMIC_WORKLOADS = new Set([ "bench/thread-slots", "bench/channel-messages", "bench/dynamic-methods", + "bench/dynamic-properties", "bench/worker-messages", "bench/typed-array-scan-workers", "bench/log-summary", diff --git a/tests/corpus/1006-json-fs-config.ts b/tests/corpus/1006-json-fs-config.ts index f2b1d48c0..1783202c9 100644 --- a/tests/corpus/1006-json-fs-config.ts +++ b/tests/corpus/1006-json-fs-config.ts @@ -11,7 +11,7 @@ function tail(path: string): string { } return path.slice(i + 1); } -const path = "tmp-1006-" + tail(process.argv[1]) + ".json"; +const path = "tmp-1006-" + tail(process.argv[1]) + "-" + process.pid + ".json"; type Config = { name: string; diff --git a/tests/corpus/1354-builtin-bare-specifiers.ts b/tests/corpus/1354-builtin-bare-specifiers.ts index cd0db6984..826c3e7a0 100644 --- a/tests/corpus/1354-builtin-bare-specifiers.ts +++ b/tests/corpus/1354-builtin-bare-specifiers.ts @@ -15,7 +15,7 @@ function tail(path: string): string { return path.slice(i + 1); } -const scratch = "tmp-1354-" + tail(process.argv[1]); +const scratch = "tmp-1354-" + tail(process.argv[1]) + "-" + process.pid; if (existsSync(scratch)) { if (existsSync(j(scratch, "f.txt"))) { rmSync(j(scratch, "f.txt")); diff --git a/tests/corpus/1357-fsp-roundtrip.ts b/tests/corpus/1357-fsp-roundtrip.ts index 4ccae7f3a..7368da360 100644 --- a/tests/corpus/1357-fsp-roundtrip.ts +++ b/tests/corpus/1357-fsp-roundtrip.ts @@ -15,7 +15,7 @@ function tail(path: string): string { } async function main(): Promise { - const scratch = "tmp-1357-" + tail(process.argv[1]); + const scratch = "tmp-1357-" + tail(process.argv[1]) + "-" + process.pid; if (existsSync(scratch)) { if (existsSync(scratch + "/a.txt")) { rmSync(scratch + "/a.txt"); diff --git a/tests/corpus/2844-node-disposable-resources.ts b/tests/corpus/2844-node-disposable-resources.ts index 042f617ce..ebeda87a2 100644 --- a/tests/corpus/2844-node-disposable-resources.ts +++ b/tests/corpus/2844-node-disposable-resources.ts @@ -14,7 +14,7 @@ async function main(): Promise { console.log(timeout.hasRef(), immediate.hasRef()); } - const path = "/tmp/scriptc-disposable-file.txt"; + const path = `/tmp/scriptc-disposable-file-${process.pid}.txt`; await writeFile(path, "resource"); { await using handle = await open(path, "r"); diff --git a/tests/corpus/5566-fs-dirent-dyn-boundaries/main.mjs b/tests/corpus/5566-fs-dirent-dyn-boundaries/main.mjs index 95bcbecee..77bdf8905 100644 --- a/tests/corpus/5566-fs-dirent-dyn-boundaries/main.mjs +++ b/tests/corpus/5566-fs-dirent-dyn-boundaries/main.mjs @@ -8,7 +8,7 @@ function tail(path) { while (i >= 0 && path.charAt(i) !== "/" && path.charAt(i) !== "\\") i = i - 1; return path.slice(i + 1); } -const dir = `/tmp/scr-dirent-dyn-${tail(process.argv[1] === undefined ? "x" : process.argv[1])}`; +const dir = `/tmp/scr-dirent-dyn-${tail(process.argv[1] === undefined ? "x" : process.argv[1])}-${process.pid}`; if (fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true }); fs.mkdirSync(`${dir}/sub`, { recursive: true }); fs.writeFileSync(`${dir}/a.txt`, "x"); diff --git a/tests/corpus/7154-crypto-cipher-info.cjs b/tests/corpus/7154-crypto-cipher-info.cjs new file mode 100644 index 000000000..1498f7183 --- /dev/null +++ b/tests/corpus/7154-crypto-cipher-info.cjs @@ -0,0 +1,88 @@ +// crypto.getCipherInfo: every getCiphers() name, OpenSSL's aliases in any +// case and its legacy-provider ciphers, lookups by nid, the keyLength and +// ivLength options (exact lengths, CCM's 7 to 13 bytes, OCB's 1 to 15 and +// any length for GCM; the legacy ciphers answer undefined for either), +// and Node's argument errors. +const crypto = require("crypto"); + +for (const name of crypto.getCiphers()) console.log(name, JSON.stringify(crypto.getCipherInfo(name))); +for (const name of ["AES-128-CBC", "Aes256", "id-aes128-CCM", "chacha20-POLY1305", "des", "bf", "bf-cbc", "rc4", "seed-cbc", "cast5-cbc", "rc2-cbc", "idea-cbc", "des-ecb", "desx", "nope", ""]) + console.log(JSON.stringify(name), JSON.stringify(crypto.getCipherInfo(name))); +for (const nid of [0, -1, 1, 5, 31, 91, 419, 427, 895, 896, 901, 1018, 1198, 1248, 99999]) + console.log("nid", nid, JSON.stringify(crypto.getCipherInfo(nid))); + +const options = [ + { keyLength: 16 }, + { keyLength: 32 }, + { ivLength: 16 }, + { ivLength: 12 }, + { ivLength: 7 }, + { ivLength: 13 }, + { ivLength: 14 }, + { ivLength: 0 }, + { ivLength: 1 }, + { ivLength: 15 }, + { ivLength: 1000 }, + { keyLength: 16, ivLength: 12 }, + { keyLength: undefined, ivLength: undefined }, + {}, +]; +for (const name of ["aes-128-cbc", "aes-128-gcm", "aes-128-ccm", "aes-128-ocb", "aes-128-ecb", "chacha20-poly1305", "aes-128-wrap-pad", "bf-cbc", "rc4"]) + for (const option of options) console.log(name, JSON.stringify(option), JSON.stringify(crypto.getCipherInfo(name, option))); + +function attempt(label, run) { + try { + console.log(label, "ok", JSON.stringify(run())); + } catch (e) { + console.log(label, e.name, e.code, e.message); + } +} +attempt("no argument", () => crypto.getCipherInfo()); +attempt("null", () => crypto.getCipherInfo(null)); +attempt("boolean", () => crypto.getCipherInfo(true)); +attempt("object", () => crypto.getCipherInfo({})); +attempt("fraction", () => crypto.getCipherInfo(1.5)); +attempt("NaN", () => crypto.getCipherInfo(NaN)); +attempt("2^31", () => crypto.getCipherInfo(2 ** 31)); +attempt("options null", () => crypto.getCipherInfo("aes-128-cbc", null)); +attempt("options number", () => crypto.getCipherInfo("aes-128-cbc", 5)); +attempt("options string", () => crypto.getCipherInfo("aes-128-cbc", "x")); +attempt("options array", () => crypto.getCipherInfo("aes-128-cbc", [])); +attempt("keyLength string", () => crypto.getCipherInfo("aes-128-cbc", { keyLength: "16" })); +attempt("keyLength fraction", () => crypto.getCipherInfo("aes-128-cbc", { keyLength: 16.5 })); +attempt("ivLength 2^31", () => crypto.getCipherInfo("aes-128-cbc", { ivLength: 2 ** 31 })); +attempt("ivLength null", () => crypto.getCipherInfo("aes-128-cbc", { ivLength: null })); +attempt("unknown name with bad options", () => crypto.getCipherInfo("nope", { keyLength: "x" })); +const info = crypto.getCipherInfo("aes-256-gcm", { ivLength: 7 }); +console.log(Object.keys(info).join(), Object.getPrototypeOf(info) === Object.prototype); + +// The options are read as destructuring reads them: inherited properties +// and getters count, and both are read before either is validated. +attempt("getter keyLength", () => crypto.getCipherInfo("aes-128-cbc", { get keyLength() { return 17; } })); +attempt("getter ivLength", () => crypto.getCipherInfo("aes-128-cbc", { get ivLength() { return 5; } })); +attempt("getter valid", () => crypto.getCipherInfo("aes-128-cbc", { get keyLength() { return 16; } })); +attempt("inherited keyLength", () => crypto.getCipherInfo("aes-128-cbc", Object.create({ keyLength: 17 }))); +attempt("inherited valid", () => crypto.getCipherInfo("aes-128-cbc", Object.create({ keyLength: 16, ivLength: 16 }))); +class LengthOptions { + constructor(ivLength) { + this.ivLength = ivLength; + } +} +attempt("class field", () => crypto.getCipherInfo("aes-128-cbc", new LengthOptions(5))); +attempt("class field valid", () => crypto.getCipherInfo("aes-128-cbc", new LengthOptions(16))); +attempt("getter wrong type", () => crypto.getCipherInfo("aes-128-cbc", { get keyLength() { return "x"; } })); +attempt("getter throws", () => crypto.getCipherInfo("aes-128-cbc", { get keyLength() { throw new Error("getter"); } })); +const reads = []; +attempt("read order", () => + crypto.getCipherInfo("aes-128-cbc", { + get ivLength() { + reads.push("ivLength"); + return "x"; + }, + get keyLength() { + reads.push("keyLength"); + return "y"; + }, + }), +); +console.log("reads", reads.join()); diff --git a/tests/corpus/7155-cipher-aes-ccm.cjs b/tests/corpus/7155-cipher-aes-ccm.cjs new file mode 100644 index 000000000..5e6407e52 --- /dev/null +++ b/tests/corpus/7155-cipher-aes-ccm.cjs @@ -0,0 +1,199 @@ +// AES-CCM through createCipheriv and createDecipheriv: RFC 3610 and NIST +// SP 800-38C vectors, every tag and IV length, AAD with plaintextLength, +// the message-length caps of 12- and 13-byte IVs, a decipher's failed +// authentication reported by final(), the one-update rule, the AAD and +// tag calls in each state, and as a stream. +const crypto = require("crypto"); + +function attempt(label, run) { + try { + const v = run(); + console.log(label, "ok", Buffer.isBuffer(v) ? v.toString("hex") : v); + } catch (e) { + console.log(label, e.name, e.code, JSON.stringify(e.message), JSON.stringify(Object.keys(e))); + } +} +const hex = (s) => Buffer.from(s, "hex"); + +// NIST SP 800-38C examples C.1 to C.3. +const nistKey = hex("404142434445464748494a4b4c4d4e4f"); +for (const [iv, aad, pt, tagLen] of [ + ["10111213141516", "0001020304050607", "20212223", 4], + ["1011121314151617", "000102030405060708090a0b0c0d0e0f", "202122232425262728292a2b2c2d2e2f", 6], + ["101112131415161718191a1b", "000102030405060708090a0b0c0d0e0f10111213", "202122232425262728292a2b2c2d2e2f3031323334353637", 8], +]) { + attempt(`nist iv ${iv.length / 2}`, () => { + const c = crypto.createCipheriv("aes-128-ccm", nistKey, hex(iv), { authTagLength: tagLen }); + c.setAAD(hex(aad), { plaintextLength: pt.length / 2 }); + return Buffer.concat([c.update(hex(pt)), c.final(), c.getAuthTag()]); + }); +} +// RFC 3610 packet vector 1. +attempt("rfc3610 #1", () => { + const c = crypto.createCipheriv("aes-128-ccm", hex("c0c1c2c3c4c5c6c7c8c9cacbcccdcecf"), hex("00000003020100a0a1a2a3a4a5"), { authTagLength: 8 }); + c.setAAD(hex("0001020304050607"), { plaintextLength: 23 }); + return Buffer.concat([c.update(hex("08090a0b0c0d0e0f101112131415161718191a1b1c1d1e")), c.final(), c.getAuthTag()]); +}); + +const key = Buffer.alloc(16, 1); +const pt = Buffer.from("hello world, ccm mode, three blocks of it!"); +for (const size of [16, 24, 32]) + for (const name of [`aes-${size * 8}-ccm`, `id-aes${size * 8}-ccm`, `AES-${size * 8}-CCM`]) + attempt(name, () => { + const c = crypto.createCipheriv(name, Buffer.alloc(size, 3), Buffer.alloc(12, 2), { authTagLength: 10 }); + return Buffer.concat([c.update(pt), c.final(), c.getAuthTag()]); + }); +for (const tagLen of [2, 3, 4, 5, 6, 8, 10, 12, 14, 15, 16, 18]) + attempt(`tag ${tagLen}`, () => { + const c = crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(12), { authTagLength: tagLen }); + return Buffer.concat([c.update(pt), c.final(), c.getAuthTag()]); + }); +for (const ivLen of [0, 6, 7, 8, 11, 12, 13, 14]) + attempt(`iv ${ivLen}`, () => { + const c = crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(ivLen, 9), { authTagLength: 16 }); + return Buffer.concat([c.update(pt), c.final(), c.getAuthTag()]); + }); +attempt("no tag length", () => crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(12))); +attempt("no tag length, mixed case", () => crypto.createDecipheriv("Aes-128-Ccm", key, Buffer.alloc(12))); +attempt("tag length string", () => crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(12), { authTagLength: "16" })); +attempt("bad iv and no tag length", () => crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(5))); +attempt("bad key", () => crypto.createCipheriv("aes-128-ccm", Buffer.alloc(15), Buffer.alloc(12), { authTagLength: 16 })); + +const iv = Buffer.alloc(12, 2), aad = Buffer.from("header"); +function seal(steps) { + const c = crypto.createCipheriv("aes-128-ccm", key, iv, { authTagLength: 16 }); + const parts = []; + for (const step of steps) { + try { + const out = step(c); + if (Buffer.isBuffer(out)) parts.push(out.toString("hex")); + } catch (e) { + parts.push(`<${e.code ?? e.name}: ${e.message}>`); + } + } + try { + parts.push("tag " + c.getAuthTag().toString("hex")); + } catch (e) { + parts.push(`<${e.code}: ${e.message}>`); + } + return parts.join(" "); +} +const update = (data) => (c) => c.update(data); +const final = () => (c) => c.final(); +const setAAD = (data, options) => (c) => (c.setAAD(data, options), Buffer.alloc(0)); +console.log("plain", seal([update(pt), final()])); +console.log("aad", seal([setAAD(aad, { plaintextLength: pt.length }), update(pt), final()])); +console.log("aad without length", seal([setAAD(aad), update(pt), final()])); +console.log("aad bad length", seal([setAAD(aad, { plaintextLength: -1 }), update(pt), final()])); +console.log("aad length mismatch", seal([setAAD(aad, { plaintextLength: 5 }), update(pt), final()])); +console.log("aad twice", seal([setAAD(aad, { plaintextLength: pt.length }), setAAD(aad, { plaintextLength: pt.length }), update(pt), final()])); +console.log("empty aad", seal([setAAD(Buffer.alloc(0), { plaintextLength: pt.length }), update(pt), final()])); +console.log("aad after update", seal([update(pt), setAAD(aad, { plaintextLength: 3 }), final()])); +console.log("update twice", seal([update(pt), update(pt), final()])); +console.log("update empty twice", seal([update(Buffer.alloc(0)), update(Buffer.alloc(0)), final()])); +console.log("no update", seal([final()])); +console.log("aad, no update", seal([setAAD(aad, { plaintextLength: pt.length }), final()])); +console.log("aad length 0, no update", seal([setAAD(aad, { plaintextLength: 0 }), final()])); +console.log("tag before final", seal([update(pt)])); +console.log("final twice", seal([update(pt), final(), final()])); +console.log("update after final", seal([update(pt), final(), update(pt)])); +console.log("auto padding", seal([(c) => (c.setAutoPadding(false), Buffer.alloc(0)), update(pt), final()])); +console.log("strings", seal([(c) => Buffer.from(c.update("abcd", "utf8", "hex"), "hex"), (c) => Buffer.from(c.final("hex"), "hex")])); + +// Message-length caps: 2^16 - 1 bytes with a 13-byte IV, 2^24 - 1 with 12. +for (const [ivLen, size] of [[13, 65535], [13, 65536], [12, 65536]]) + attempt(`length ${size}, iv ${ivLen}`, () => { + const c = crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(ivLen), { authTagLength: 4 }); + return crypto.createHash("sha256").update(c.update(Buffer.alloc(size, 1))).digest("hex").slice(0, 16); + }); +attempt("aad length 65536, iv 13", () => crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(13), { authTagLength: 4 }).setAAD(aad, { plaintextLength: 65536 })); +attempt("aad length 16777216, iv 12", () => crypto.createCipheriv("aes-128-ccm", key, Buffer.alloc(12), { authTagLength: 4 }).setAAD(aad, { plaintextLength: 16777216 })); + +const sealer = crypto.createCipheriv("aes-128-ccm", key, iv, { authTagLength: 16 }); +sealer.setAAD(aad, { plaintextLength: pt.length }); +const ct = Buffer.concat([sealer.update(pt), sealer.final()]); +const tag = sealer.getAuthTag(); +function open(steps) { + const d = crypto.createDecipheriv("aes-128-ccm", key, iv, { authTagLength: 16 }); + const parts = []; + for (const step of steps) { + try { + const out = step(d); + if (Buffer.isBuffer(out)) parts.push(JSON.stringify(out.toString())); + } catch (e) { + parts.push(`<${e.code ?? e.name}: ${e.message}>`); + } + } + return parts.join(" "); +} +const setAuthTag = (t) => (d) => (d.setAuthTag(t), Buffer.alloc(0)); +console.log("open", open([setAuthTag(tag), setAAD(aad, { plaintextLength: ct.length }), update(ct), final()])); +console.log("open, tag after aad", open([setAAD(aad, { plaintextLength: ct.length }), setAuthTag(tag), update(ct), final()])); +console.log("open without tag", open([setAAD(aad, { plaintextLength: ct.length }), update(ct), final()])); +console.log("open bad tag", open([setAuthTag(Buffer.alloc(16)), setAAD(aad, { plaintextLength: ct.length }), update(ct), final()])); +console.log("open without aad", open([setAuthTag(tag), update(ct), final()])); +console.log("open tampered", open([setAuthTag(tag), setAAD(aad, { plaintextLength: ct.length }), update(Buffer.concat([ct.subarray(0, 5), Buffer.from([ct[5] ^ 1]), ct.subarray(6)])), final()])); +console.log("open tag after update", open([setAAD(aad, { plaintextLength: ct.length }), update(ct), setAuthTag(tag), final()])); +console.log("open tag twice", open([setAuthTag(tag), setAuthTag(tag), setAAD(aad, { plaintextLength: ct.length }), update(ct), final()])); +console.log("open tag wrong length", open([setAuthTag(tag.subarray(0, 8))])); +console.log("open update twice", open([setAuthTag(tag), setAAD(aad, { plaintextLength: ct.length }), update(ct), update(ct), final()])); +console.log("open final only", open([setAuthTag(tag), final()])); +console.log("open length mismatch", open([setAuthTag(tag), setAAD(aad, { plaintextLength: 3 }), update(ct), final()])); +console.log("open getAuthTag", open([(d) => d.getAuthTag()])); +console.log("open empty message", (() => { + const c = crypto.createCipheriv("aes-128-ccm", key, iv, { authTagLength: 8 }); + c.update(Buffer.alloc(0)); + c.final(); + const t = c.getAuthTag(); + const d = crypto.createDecipheriv("aes-128-ccm", key, iv, { authTagLength: 8 }); + d.setAuthTag(t); + return JSON.stringify(Buffer.concat([d.update(Buffer.alloc(0)), d.final()]).toString()); +})()); + +// An empty message is still authenticated: its tag covers the nonce, the +// AAD and the declared length, so a wrong tag, other AAD or another +// plaintextLength fails in final(), as does a decipher without an update. +const empty = Buffer.alloc(0); +const sealEmpty = (aadBuf) => { + const c = crypto.createCipheriv("aes-128-ccm", key, iv, { authTagLength: 16 }); + if (aadBuf) c.setAAD(aadBuf, { plaintextLength: 0 }); + c.update(empty); + c.final(); + return c.getAuthTag(); +}; +const emptyTag = sealEmpty(aad); +const emptyTagNoAad = sealEmpty(null); +const flipped = Buffer.from(emptyTag); +flipped[0] ^= 1; +const openEmpty = (steps) => { + const d = crypto.createDecipheriv("aes-128-ccm", key, iv, { authTagLength: 16 }); + const out = []; + try { + for (const step of steps) { + const v = step(d); + if (Buffer.isBuffer(v)) out.push(v); + } + return `ok ${JSON.stringify(Buffer.concat(out).toString())}`; + } catch (e) { + return `${e.name} ${e.code} ${JSON.stringify(e.message)} after ${out.length} outputs`; + } +}; +console.log("empty tag", emptyTag.toString("hex"), emptyTagNoAad.toString("hex")); +console.log("empty, right tag", openEmpty([setAuthTag(emptyTag), setAAD(aad, { plaintextLength: 0 }), update(empty), final()])); +console.log("empty, wrong tag", openEmpty([setAuthTag(Buffer.alloc(16, 9)), setAAD(aad, { plaintextLength: 0 }), update(empty), final()])); +console.log("empty, flipped tag", openEmpty([setAuthTag(flipped), setAAD(aad, { plaintextLength: 0 }), update(empty), final()])); +console.log("empty, other aad", openEmpty([setAuthTag(emptyTag), setAAD(Buffer.from("other"), { plaintextLength: 0 }), update(empty), final()])); +console.log("empty, plaintextLength 5", openEmpty([setAuthTag(emptyTag), setAAD(aad, { plaintextLength: 5 }), update(empty), final()])); +console.log("empty, no update", openEmpty([setAuthTag(emptyTag), setAAD(aad, { plaintextLength: 0 }), final()])); +console.log("empty, no tag", openEmpty([setAAD(aad, { plaintextLength: 0 }), update(empty), final()])); +console.log("empty, no aad, right tag", openEmpty([setAuthTag(emptyTagNoAad), update(empty), final()])); +console.log("empty, no aad, wrong tag", openEmpty([setAuthTag(Buffer.alloc(16, 9)), update(empty), final()])); +console.log("empty, aad dropped", openEmpty([setAuthTag(emptyTag), update(empty), final()])); +console.log("empty, a string update", openEmpty([setAuthTag(Buffer.alloc(16, 9)), (d) => d.update(""), final()])); + +// As a stream: one write and the end. +const stream = crypto.createCipheriv("aes-128-ccm", key, iv, { authTagLength: 12 }); +const chunks = []; +stream.on("data", (chunk) => chunks.push(chunk)); +stream.on("end", () => console.log("stream", Buffer.concat(chunks).toString("hex"), stream.getAuthTag().toString("hex"))); +stream.end(pt); diff --git a/tests/corpus/7156-cipher-key-wrap.cjs b/tests/corpus/7156-cipher-key-wrap.cjs new file mode 100644 index 000000000..a958d16c5 --- /dev/null +++ b/tests/corpus/7156-cipher-key-wrap.cjs @@ -0,0 +1,103 @@ +// The AES key-wrap ciphers through createCipheriv and createDecipheriv: +// RFC 3394 and RFC 5649 vectors under every name (aes-N-wrap, aesN-wrap, +// id-aesN-wrap, the -pad forms and the -inv ciphers), the input and IV +// lengths each takes, integrity failures, and the one-update rule. +const crypto = require("crypto"); + +function attempt(label, run) { + try { + const v = run(); + console.log(label, "ok", Buffer.isBuffer(v) ? v.toString("hex") : v); + } catch (e) { + console.log(label, e.name, e.code, JSON.stringify(e.message), JSON.stringify(Object.keys(e))); + } +} +const hex = (s) => Buffer.from(s, "hex"); +const wrap = (name, key, iv, input) => { + const c = crypto.createCipheriv(name, key, iv); + return Buffer.concat([c.update(input), c.final()]); +}; +const unwrap = (name, key, iv, input) => { + const d = crypto.createDecipheriv(name, key, iv); + return Buffer.concat([d.update(input), d.final()]); +}; +const iv8 = hex("a6a6a6a6a6a6a6a6"), iv4 = hex("a65959a6"); + +// RFC 3394 section 4. +const kek = hex("000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"); +const keyData = hex("00112233445566778899aabbccddeeff000102030405060708090a0b0c0d0e0f"); +for (const [kekBits, dataBits] of [[128, 128], [192, 128], [256, 128], [192, 192], [256, 192], [256, 256]]) { + const k = kek.subarray(0, kekBits / 8), d = keyData.subarray(0, dataBits / 8); + for (const name of [`aes-${kekBits}-wrap`, `aes${kekBits}-wrap`, `id-aes${kekBits}-wrap`, `ID-AES${kekBits}-WRAP`]) { + attempt(`${name} ${dataBits}`, () => wrap(name, k, iv8, d)); + attempt(`${name} ${dataBits} unwrap`, () => unwrap(name, k, iv8, wrap(name, k, iv8, d))); + } +} +// RFC 5649 section 6. +const kek5649 = hex("5840df6e29b02af1ab493b705bf16ea1ae8338f4dcc176a8"); +for (const data of ["c37b7e6492584340bed12207808941155068f738", "466f7250617369"]) + for (const name of ["aes-192-wrap-pad", "aes192-wrap-pad", "id-aes192-wrap-pad"]) { + attempt(`${name} ${data.length / 2}`, () => wrap(name, kek5649, iv4, hex(data))); + attempt(`${name} ${data.length / 2} unwrap`, () => unwrap(name, kek5649, iv4, wrap(name, kek5649, iv4, hex(data)))); + } + +const k128 = kek.subarray(0, 16); +for (const size of [0, 1, 7, 8, 9, 15, 16, 17, 24, 40]) { + attempt(`wrap ${size} bytes`, () => wrap("id-aes128-wrap", k128, iv8, Buffer.alloc(size, 5))); + attempt(`wrap-pad ${size} bytes`, () => { + const w = wrap("id-aes128-wrap-pad", k128, iv4, Buffer.alloc(size, 5)); + return w.toString("hex") + " -> " + unwrap("id-aes128-wrap-pad", k128, iv4, w).toString("hex"); + }); +} +for (const size of [8, 16, 20, 24, 32]) + for (const name of ["id-aes128-wrap", "id-aes128-wrap-pad"]) { + const iv = name.endsWith("pad") ? iv4 : iv8; + attempt(`${name} unwrap ${size} bytes`, () => unwrap(name, k128, iv, Buffer.alloc(size, 1))); + } +attempt("other iv", () => wrap("id-aes128-wrap", k128, Buffer.alloc(8, 1), keyData.subarray(0, 16))); +attempt("unwrap with another iv", () => unwrap("id-aes128-wrap", k128, Buffer.alloc(8, 1), wrap("id-aes128-wrap", k128, iv8, keyData.subarray(0, 16)))); +attempt("pad other iv", () => wrap("id-aes128-wrap-pad", k128, Buffer.alloc(4, 7), Buffer.from("key"))); +attempt("pad unwrap with another iv", () => unwrap("id-aes128-wrap-pad", k128, Buffer.alloc(4, 7), wrap("id-aes128-wrap-pad", k128, iv4, Buffer.from("key")))); +attempt("pad tampered length", () => { + const w = wrap("id-aes128-wrap-pad", k128, iv4, Buffer.alloc(20, 9)); + w[3] ^= 1; + return unwrap("id-aes128-wrap-pad", k128, iv4, w); +}); +for (const name of ["aes-128-wrap-inv", "aes128-wrap-inv", "aes-256-wrap-inv", "aes-128-wrap-pad-inv", "aes256-wrap-pad-inv"]) { + const bits = Number(name.match(/\d+/)[0]); + const iv = name.includes("pad") ? iv4 : iv8; + attempt(name, () => { + const w = wrap(name, kek.subarray(0, bits / 8), iv, keyData.subarray(0, 24)); + return w.toString("hex") + " -> " + unwrap(name, kek.subarray(0, bits / 8), iv, w).toString("hex"); + }); +} +attempt("null iv", () => crypto.createCipheriv("id-aes128-wrap", k128, null)); +attempt("iv 16", () => crypto.createCipheriv("id-aes128-wrap", k128, Buffer.alloc(16))); +attempt("pad iv 8", () => crypto.createCipheriv("id-aes128-wrap-pad", k128, iv8)); +attempt("key 24", () => crypto.createCipheriv("id-aes128-wrap", Buffer.alloc(24), iv8)); + +function steps(list) { + const c = crypto.createCipheriv("id-aes128-wrap", k128, iv8); + return list + .map((step) => { + try { + const out = step(c); + return Buffer.isBuffer(out) ? out.toString("hex") || '""' : String(out); + } catch (e) { + return `<${e.code ?? e.name}: ${e.message}>`; + } + }) + .join(" "); +} +const data = keyData.subarray(0, 16); +console.log("update twice", steps([(c) => c.update(data), (c) => c.update(data), (c) => c.final()])); +console.log("final only", steps([(c) => c.final()])); +console.log("final twice", steps([(c) => c.update(data), (c) => c.final(), (c) => c.final()])); +console.log("update after final", steps([(c) => c.update(data), (c) => c.final(), (c) => c.update(data)])); +console.log("bad then good", steps([(c) => c.update(Buffer.alloc(12)), (c) => c.update(data), (c) => c.final()])); +console.log("bad then final", steps([(c) => c.update(Buffer.alloc(12)), (c) => c.final()])); +console.log("empty then data", steps([(c) => c.update(Buffer.alloc(0)), (c) => c.update(data), (c) => c.final()])); +console.log("hex", steps([(c) => c.update(data, undefined, "hex"), (c) => c.final("hex")])); +console.log("auto padding", steps([(c) => c.setAutoPadding(false) === c, (c) => c.update(data)])); +console.log("setAAD", steps([(c) => c.setAAD(Buffer.alloc(1))])); +console.log("getAuthTag", steps([(c) => c.update(data), (c) => c.final(), (c) => c.getAuthTag()])); diff --git a/tests/corpus/7157-crypto-cipher-info-typed.ts b/tests/corpus/7157-crypto-cipher-info-typed.ts new file mode 100644 index 000000000..2de0f0069 --- /dev/null +++ b/tests/corpus/7157-crypto-cipher-info-typed.ts @@ -0,0 +1,42 @@ +// getCipherInfo, AES-CCM and AES key wrap through their declared types: the +// CipherInfo record's fields, absent fields and key order, the keyLength and +// ivLength options, and the CipherCCM and DecipherCCM members. +import { createCipheriv, createDecipheriv, getCipherInfo, type CipherInfo } from "node:crypto"; + +const info: CipherInfo | undefined = getCipherInfo("aes-128-ccm"); +if (info) console.log(info.name, info.nid, info.mode, info.keyLength, info.ivLength, info.blockSize); +const byNid = getCipherInfo(901); +console.log(byNid?.name, byNid?.mode); +const sized = getCipherInfo("aes-128-ccm", { ivLength: 7 }); +console.log(sized?.ivLength, getCipherInfo("aes-128-ccm", { ivLength: 6 })); +console.log(getCipherInfo("nope")); +console.log(JSON.stringify(getCipherInfo("AES-256-WRAP-PAD")!)); +console.log(Object.keys(getCipherInfo("chacha20")!), JSON.stringify(getCipherInfo("aes-128-ecb")!)); +for (const name of ["rc4", "des-ede3-cbc", "aes-256-xts", "chacha20-poly1305"]) { + const found = getCipherInfo(name, { keyLength: 32 }); + console.log(name, found ? Object.keys(found).join() : "none", found?.blockSize, found?.ivLength); +} + +const key = Buffer.alloc(16, 1); +const iv = Buffer.alloc(12, 2); +const aad = Buffer.from("header"); +const plain = Buffer.from("typed ccm message"); +const cipher = createCipheriv("aes-128-ccm", key, iv, { authTagLength: 8 }); +cipher.setAAD(aad, { plaintextLength: plain.length }); +const ciphertext = Buffer.concat([cipher.update(plain), cipher.final()]); +const tag: Buffer = cipher.getAuthTag(); +console.log(ciphertext.toString("hex"), tag.toString("hex")); +const decipher = createDecipheriv("aes-128-ccm", key, iv, { authTagLength: 8 }); +decipher.setAuthTag(tag); +decipher.setAAD(aad, { plaintextLength: ciphertext.length }); +const opened: Buffer = decipher.update(ciphertext); +decipher.final(); +console.log(opened.toString()); + +const kek = Buffer.alloc(24, 3); +const icv = Buffer.from("A6A6A6A6A6A6A6A6", "hex"); +const wrap = createCipheriv("aes192-wrap", kek, icv); +const wrapped = Buffer.concat([wrap.update(Buffer.alloc(16, 9)), wrap.final()]); +console.log(wrapped.toString("base64")); +const unwrap = createDecipheriv("id-aes192-wrap", kek, icv); +console.log(Buffer.concat([unwrap.update(wrapped), unwrap.final()]).toString("hex")); diff --git a/tests/corpus/7260-teardown-segments.ts b/tests/corpus/7260-teardown-segments.ts new file mode 100644 index 000000000..cbd2b144d --- /dev/null +++ b/tests/corpus/7260-teardown-segments.ts @@ -0,0 +1,179 @@ +// Teardowns that hand their blocks back in order. +// +// A typed teardown of a type with children of its own type keeps the blocks +// it frees in one segment and frees the segment when the chain's first +// teardown returns, so the next structure built of that type reuses them in +// order. This program rebuilds structures of such types for many +// generations and reads every value of each generation before dropping it: +// a block handed out twice, or one freed while still in use, shows up as a +// wrong checksum. The shapes cover the ways a segment is put together: +// record literals (allocated before their fields) and constructors whose +// arguments build the children first, lists built by prepending and by +// appending, trees far deeper than one direct teardown chain and lists +// longer than the runtime's whole depth budget (whose tails wait in its +// queue), subtrees still owned elsewhere when their parent goes, records of +// one type holding records of another, and nodes of a cycle-capable class +// that are still candidates of the cycle collector when the last reference +// goes, and rings of them that only the collector frees. + +interface Node { + left: Node | null; + right: Node | null; + value: number; +} + +function literalTree(depth: number, value: number): Node { + return depth === 0 + ? { left: null, right: null, value } + : { left: literalTree(depth - 1, value * 2), right: literalTree(depth - 1, value * 2 + 1), value }; +} + +function sum(node: Node | null): number { + return node === null ? 0 : node.value + sum(node.left) + sum(node.right); +} + +class Branch { + readonly left: Branch | null; + readonly right: Branch | null; + readonly label: string; + constructor(left: Branch | null, right: Branch | null, label: string) { + this.left = left; + this.right = right; + this.label = label; + } +} + +function classTree(depth: number, label: string): Branch { + return depth === 0 + ? new Branch(null, null, label) + : new Branch(classTree(depth - 1, label + "l"), classTree(depth - 1, label + "r"), label); +} + +function labels(branch: Branch | null): number { + return branch === null ? 0 : branch.label.length + labels(branch.left) + labels(branch.right); +} + +interface Cell { + next: Cell | null; + n: number; +} + +function prepended(count: number, base: number): Cell | null { + let head: Cell | null = null; + for (let i = 0; i < count; i++) head = { next: head, n: base + i }; + return head; +} + +function appended(count: number, base: number): Cell | null { + if (count === 0) return null; + const head: Cell = { next: null, n: base }; + let tail = head; + for (let i = 1; i < count; i++) { + const cell: Cell = { next: null, n: base + i }; + tail.next = cell; + tail = cell; + } + return head; +} + +function cells(head: Cell | null): [number, number] { + let count = 0; + let total = 0; + for (let cell = head; cell !== null; cell = cell.next) { + count++; + total += cell.n; + } + return [count, total]; +} + +// Generations of literal trees: shallow, one chain deep, and several chains +// deep. Each generation is checked against the sum the shape must have. +for (const depth of [0, 1, 5, 16, 17, 18]) { + let ok = true; + for (let generation = 0; generation < 6; generation++) { + const tree = literalTree(depth, 1); + const expected = ((2 ** (depth + 1)) * (2 ** (depth + 1) - 1)) / 2; + if (sum(tree) !== expected) ok = false; + } + console.log(`literal trees of depth ${depth}: ${ok}`); +} + +// Generations of class trees built children first. +for (const depth of [3, 12, 17]) { + let total = 0; + for (let generation = 0; generation < 4; generation++) total += labels(classTree(depth, "t")); + console.log(`class trees of depth ${depth}: ${total}`); +} + +// Lists both ways, alternating, so each reuses the other's blocks; the long +// ones outlast the runtime's depth budget. +for (const count of [1, 15, 16, 17, 1000, 3000, 40000]) { + const seen: string[] = []; + for (let generation = 0; generation < 4; generation++) { + const list = generation % 2 === 0 ? prepended(count, generation) : appended(count, generation); + const [length, total] = cells(list); + seen.push(`${length}/${total}`); + } + console.log(`lists of ${count}: ${seen.join(" ")}`); +} + +// Subtrees owned elsewhere survive their parent's teardown, and the blocks +// freed around them are not handed out while they live. +const kept: Node[] = []; +for (let generation = 0; generation < 5; generation++) { + const tree = literalTree(10, generation + 1); + if (tree.left !== null && tree.left.right !== null) kept.push(tree.left.right); + if (tree.right !== null) kept.push(tree.right); + // Building the next tree reuses the dropped blocks. + literalTree(10, 7); +} +console.log(`kept subtrees: ${kept.length} ${kept.map(sum).join(",")}`); + +// A record of one type holding trees of another: the outer teardown frees +// its own blocks as before and tears each inner tree down as a segment. +interface Forest { + first: Node; + second: Node | null; + rest: Forest | null; +} + +function forest(count: number): Forest | null { + let out: Forest | null = null; + for (let i = 0; i < count; i++) out = { first: literalTree(4, i), second: i % 3 === 0 ? null : literalTree(2, i), rest: out }; + return out; +} + +let forestTotal = 0; +for (let generation = 0; generation < 3; generation++) { + for (let f = forest(500); f !== null; f = f.rest) forestTotal += sum(f.first) + sum(f.second); +} +console.log(`forests: ${forestTotal}`); + +// A cycle-capable class: releases of shared links make nodes candidates of +// the cycle collector, which are then torn down while still buffered, and +// the peer links close some rings, which the collector frees. +class Ring { + next: Ring | null = null; + peer: Ring | null = null; + id: number; + constructor(id: number) { + this.id = id; + } +} + +let ringTotal = 0; +for (let generation = 0; generation < 3; generation++) { + let head: Ring | null = null; + const peers: Ring[] = []; + for (let i = 0; i < 2000; i++) { + const ring: Ring = new Ring(i); + ring.next = head; + if (i % 7 === 0) peers.push(ring); + head = ring; + } + for (let r = head; r !== null; r = r.next) if (r.id % 5 === 0) r.peer = peers[r.id % peers.length] ?? null; + for (const peer of peers) ringTotal += peer.id; + peers.length = 0; + for (let r = head; r !== null; r = r.next) ringTotal += r.peer === null ? 0 : r.peer.id; +} +console.log(`rings: ${ringTotal}`); diff --git a/tests/corpus/7264-array-dense-paired-loops.ts b/tests/corpus/7264-array-dense-paired-loops.ts new file mode 100644 index 000000000..59cedf60b --- /dev/null +++ b/tests/corpus/7264-array-dense-paired-loops.ts @@ -0,0 +1,120 @@ +// Two-counter loops that index number arrays at both counters +// (`for (let i = lo, j = hi; i < j; i++, j--)`), and element copies through +// `a[i]!` reads, both inside and outside the ranges a single entry test can +// cover: filled prefixes, holes, present undefined, reads past the end, +// fractional and negative starts, int32 arrays and long checked arrays. + +function reversePrefix(a: number[], end: number): void { + for (let i = 0, j = end; i < j; i++, j--) { + const t = a[i]!; + a[i] = a[j]!; + a[j] = t; + } +} + +function show(label: string, a: number[]): void { + console.log(label, a.length, JSON.stringify(a), a); +} + +// The whole prefix is present: the entry test passes. +const plain: number[] = []; +for (let k = 0; k < 8; k++) plain[k] = k * 3; +reversePrefix(plain, 5); +show("plain", plain); +reversePrefix(plain, 7); +show("whole", plain); +reversePrefix(plain, 0); +show("empty range", plain); + +// The high start is past the end: the reads there miss, the stores extend. +const short = [1, 2, 3]; +reversePrefix(short, 5); +show("past the end", short); + +// A hole inside the range. +const holey: number[] = [10, 20]; +holey[3] = 40; +holey[4] = 50; +reversePrefix(holey, 4); +show("hole", holey); + +// A present undefined inside the range, copied by a mirror loop. +const loose: number[] = [7, 8, 9, 10]; +(loose as (number | undefined)[])[2] = undefined; +for (let i = 0, j = 3; i < j; i++, j--) loose[i] = loose[j]!; +show("undefined", loose); + +// Fractional and negative starts index non-element properties. +const fractional = [1, 2, 3, 4]; +reversePrefix(fractional, 2.5); +show("fractional", fractional); +const negative = [5, 6, 7]; +for (let i = -1, j = 2; i < j; i++, j--) negative[i] = negative[j]!; +show("negative", negative); + +// A local int32 array (every stored element is a small integer). +function int32Flips(start: number): void { + const counts = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]; + let flips = 0; + for (let first = counts[start]!; first !== 0; first = counts[0]!) { + for (let i = 0, j = first; i < j; i++, j--) { + const t = counts[i]!; + counts[i] = counts[j]!; + counts[j] = t; + } + flips++; + if (flips > 20) break; + } + console.log("int32", flips, counts.join(",")); +} +int32Flips(3); +int32Flips(9); + +// A local long array with checked int32 stores, widened by a fraction. +function longChecked(n: number, fraction: number): void { + const long = new Array(n).fill(0); + for (let k = 0; k < long.length; k++) long[k] = k; + for (let i = 0, j = n - 1; i < j; i++, j--) { + const t = long[i]!; + long[i] = long[j]!; + long[j] = t; + } + console.log("long", long[0], long[1], long[n - 2], long[n - 1]); + long[10] = fraction; + for (let i = 0, j = 20; i < j; i++, j--) { + const t = long[i]!; + long[i] = long[j]!; + long[j] = t; + } + console.log("widened", long.slice(0, 12).join(","), long[20]); +} +longChecked(5000, 0.5); + +// Element copies through `a[i]!`: the source is shorter than the range. +function copy(to: number[], from: number[], n: number): void { + for (let i = 0; i < n; i++) to[i] = from[i]!; +} +const into = [0, 0, 0, 0, 0, 0]; +copy(into, [1, 2, 3, 4, 5, 6], 6); +show("copy", into); +copy(into, [9, 8], 4); +show("short source", into); +const gappy: number[] = [1]; +gappy[2] = 3; +gappy[3] = 4; +copy(into, gappy, 4); +show("gappy source", into); + +// Two names for one array. +function aliased(n: number): void { + const same: number[] = []; + for (let k = 0; k < n; k++) same[k] = k + 1; + const alias = same; + for (let i = 0, j = n - 1; i < j; i++, j--) { + const t = alias[i]!; + same[i] = alias[j]!; + same[j] = t; + } + show("alias", same); +} +aliased(5); diff --git a/tests/corpus/7281-http-server-connect-tunnel.ts b/tests/corpus/7281-http-server-connect-tunnel.ts new file mode 100644 index 000000000..d8b396c07 --- /dev/null +++ b/tests/corpus/7281-http-server-connect-tunnel.ts @@ -0,0 +1,81 @@ +// An HTTP CONNECT proxy on node:http: the server's 'connect' listener gets +// the request, the raw client socket and the bytes read past the request +// head, opens a TCP connection to the requested host and pipes both ways. +// 'once' and 'addListener' register the same event, and 'upgrade' hands +// over the socket for an Upgrade request. +import * as http from "node:http"; +import * as net from "node:net"; + +const target = net.createServer((socket) => { + socket.on("data", (d) => socket.write(`echo:${d.toString()}`)); +}); + +const proxy = http.createServer((_req, res) => res.end("plain")); +const seen: string[] = []; +proxy.on("connect", (req, clientSocket, head) => { + seen.push(`connect ${req.method} ${req.url} head=${head.length}`); + const [host, port] = (req.url ?? "").split(":"); + const upstream = net.connect(Number(port), host, () => { + clientSocket.write("HTTP/1.1 200 Connection Established\r\n\r\n"); + if (head.length > 0) upstream.write(head); + upstream.pipe(clientSocket); + clientSocket.pipe(upstream); + }); +}); +proxy.once("connect", (req) => seen.push(`once ${req.url}`)); +proxy.addListener("upgrade", (req, socket, head) => { + seen.push(`upgrade ${req.headers.upgrade} head=${head.length}`); + socket.end("HTTP/1.1 101 Switching Protocols\r\nUpgrade: echo\r\nConnection: Upgrade\r\n\r\n"); +}); + +function tunnel( + proxyPort: number, + targetPort: number, + payload: string, + withHead: boolean, +): Promise { + return new Promise((resolve) => { + const c = net.connect(proxyPort, "127.0.0.1", () => { + const request = `CONNECT 127.0.0.1:${targetPort} HTTP/1.1\r\nHost: 127.0.0.1:${targetPort}\r\n\r\n`; + c.write(withHead ? request + payload : request); + }); + let text = ""; + let sent = withHead; + c.on("data", (d) => { + text += d.toString(); + if (!sent && text.includes("\r\n\r\n")) { + sent = true; + c.write(payload); + } + if (sent && text.includes(`echo:${payload}`)) { + c.end(); + resolve(text); + } + }); + }); +} + +function upgrade(proxyPort: number): Promise { + return new Promise((resolve) => { + const u = net.connect(proxyPort, "127.0.0.1", () => { + u.write("GET /ws HTTP/1.1\r\nHost: x\r\nConnection: Upgrade\r\nUpgrade: echo\r\n\r\n"); + }); + let text = ""; + u.on("data", (d) => (text += d.toString())); + u.on("end", () => resolve(text.split("\r\n")[0] ?? "")); + }); +} + +target.listen(0, "127.0.0.1", () => { + const targetPort = (target.address() as net.AddressInfo).port; + proxy.listen(0, "127.0.0.1", async () => { + const proxyPort = (proxy.address() as net.AddressInfo).port; + console.log(JSON.stringify(await tunnel(proxyPort, targetPort, "first", false))); + console.log(JSON.stringify(await tunnel(proxyPort, targetPort, "second", false))); + console.log(JSON.stringify(await tunnel(proxyPort, targetPort, "with-head", true))); + console.log(JSON.stringify(await upgrade(proxyPort))); + console.log(seen.join("\n").replaceAll(String(targetPort), "PORT")); + proxy.close(); + target.close(); + }); +}); diff --git a/tests/corpus/7282-http2-once-add-listener.ts b/tests/corpus/7282-http2-once-add-listener.ts new file mode 100644 index 000000000..ab1e24a9f --- /dev/null +++ b/tests/corpus/7282-http2-once-add-listener.ts @@ -0,0 +1,54 @@ +// node:http2 listeners registered with once and addListener on the server, +// the client session and both stream kinds. A once listener runs for the +// first event only; addListener is on. +import * as http2 from "node:http2"; +import * as net from "node:net"; + +const log: string[] = []; +const accepted: string[] = []; +const server = http2.createServer(); +server.once("connection", (socket: net.Socket) => accepted.push(`connection ${typeof socket.remotePort}`)); +server.addListener("session", () => accepted.push("session")); +server.once("stream", () => log.push("server first stream")); +server.addListener("stream", (stream, headers) => { + const header = headers[":path"]; + const path = typeof header === "string" ? header : ""; + log.push(`server stream ${path}`); + let body = ""; + stream.addListener("data", (chunk) => (body += chunk.toString())); + stream.once("end", () => { + stream.respond({ ":status": 200, "x-path": path }); + stream.end(`${path}:${body}`); + }); +}); +server.once("close", () => log.push("server close")); + +function request(client: http2.ClientHttp2Session, path: string, body: string): Promise { + return new Promise((resolve) => { + const req = client.request({ ":path": path, ":method": "POST" }); + req.once("response", (headers) => { + const status = headers[":status"]; + const echoed = headers["x-path"]; + log.push(`client response ${typeof status === "number" ? status : "?"} ${typeof echoed === "string" ? echoed : "?"}`); + }); + let text = ""; + req.addListener("data", (chunk) => (text += chunk.toString())); + req.once("end", () => resolve(text)); + req.end(body); + }); +} + +server.listen(0, () => { + const port = (server.address() as net.AddressInfo).port; + const client = http2.connect(`http://localhost:${port}`); + client.once("connect", () => log.push("client connect")); + client.addListener("close", () => { + log.push("client close"); + server.close(() => console.log([accepted.sort().join(", "), ...log].join("\n"))); + }); + void (async () => { + console.log(await request(client, "/a", "one")); + console.log(await request(client, "/b", "two")); + client.close(); + })(); +}); diff --git a/tests/corpus/7287-http-held-request-backpressure.ts b/tests/corpus/7287-http-held-request-backpressure.ts new file mode 100644 index 000000000..1953b8ec6 --- /dev/null +++ b/tests/corpus/7287-http-held-request-backpressure.ts @@ -0,0 +1,106 @@ +// A server request that holds its body (nothing reads it yet, or its +// reader paused it) stops reading its socket once its buffer is full, so +// a client that keeps uploading is held back by TCP. Each client writes +// 64 KiB chunks, one after the previous write's callback, and stops when +// the response arrives. An unread body followed by a pipelined request is +// discarded once the response finishes, and the second request is served. +import * as http from "node:http"; +import * as net from "node:net"; +import type { AddressInfo } from "node:net"; + +const MiB = 1024 * 1024; + +function upload(port: number, path: string): Promise { + return new Promise((resolve) => { + const socket = net.connect(port, "127.0.0.1"); + const frame = Buffer.concat([Buffer.from("10000\r\n"), Buffer.alloc(65536, 98), Buffer.from("\r\n")]); + let sent = 0; + let sentBeforeResponse = -1; + let response = ""; + const writeNext = (): void => { + if (sentBeforeResponse >= 0 || socket.destroyed || sent >= 256 * MiB) return; + sent += 65536; + socket.write(frame, () => { + writeNext(); + }); + }; + socket.on("connect", () => { + socket.write(`POST ${path} HTTP/1.1\r\nHost: localhost\r\nTransfer-Encoding: chunked\r\n\r\n`); + writeNext(); + }); + socket.on("data", (chunk: Buffer) => { + response += chunk.toString("latin1"); + if (sentBeforeResponse < 0 && response.includes("\r\n\r\n")) { + sentBeforeResponse = sent; + socket.destroy(); + resolve(sentBeforeResponse); + } + }); + socket.on("error", () => {}); + }); +} + +function pipelined(port: number): Promise { + return new Promise((resolve) => { + const socket = net.connect(port, "127.0.0.1"); + const body = "c".repeat(300000); + let response = ""; + socket.on("connect", () => { + socket.write( + `POST /unread HTTP/1.1\r\nHost: localhost\r\nContent-Length: ${body.length}\r\n\r\n${body}` + + "GET /second HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n", + ); + }); + socket.on("data", (chunk: Buffer) => { + response += chunk.toString("latin1"); + }); + socket.on("end", () => { + const lines = response.split("\r\n").filter((line) => line.startsWith("HTTP/") || line.startsWith("served")); + resolve(lines); + }); + }); +} + +let pausedRequest: http.IncomingMessage | undefined; + +const server = http.createServer((req, res) => { + if (req.url === "/unconsumed") { + setTimeout(() => { + res.writeHead(413, { "content-type": "text/plain" }); + res.end("unconsumed"); + }, 300); + return; + } + if (req.url === "/paused") { + let seen = 0; + req.on("data", (chunk: Buffer) => { + seen += chunk.length; + if (pausedRequest === undefined) { + pausedRequest = req; + req.pause(); + setTimeout(() => { + res.writeHead(413, { "content-type": "text/plain" }); + res.end(`paused after ${seen > 0}`); + req.resume(); + }, 300); + } + }); + return; + } + res.writeHead(200, { "content-type": "text/plain" }); + res.end(`served ${req.url}`); +}); + +server.listen(0, "127.0.0.1", () => { + const port = (server.address() as AddressInfo).port; + void (async () => { + const before = process.memoryUsage().rss; + const unconsumed = await upload(port, "/unconsumed"); + console.log("unconsumed: client stopped before 32 MiB:", unconsumed < 32 * MiB); + const paused = await upload(port, "/paused"); + console.log("paused: client stopped before 32 MiB:", paused < 32 * MiB); + console.log("memory grew under 64 MiB:", process.memoryUsage().rss - before < 64 * MiB); + for (const line of await pipelined(port)) console.log(line); + server.close(() => console.log("closed")); + })(); +}); diff --git a/tests/corpus/7380-cycle-deferred-folded-key/core.ts b/tests/corpus/7380-cycle-deferred-folded-key/core.ts new file mode 100644 index 000000000..7f4f4ab14 --- /dev/null +++ b/tests/corpus/7380-cycle-deferred-folded-key/core.ts @@ -0,0 +1,5 @@ +import { describe, kind, mark } from "./util.ts"; + +export const version = 3; +export const marked = mark({}); +console.log("core body", kind(), describe()); diff --git a/tests/corpus/7380-cycle-deferred-folded-key/main.ts b/tests/corpus/7380-cycle-deferred-folded-key/main.ts new file mode 100644 index 000000000..92617eba3 --- /dev/null +++ b/tests/corpus/7380-cycle-deferred-folded-key/main.ts @@ -0,0 +1,10 @@ +// A cycle member runs user code before it declares a constant that only +// functions read, one of them as a computed key and one under `typeof` +// (both resolved from the constant's literal type). Nothing calls those +// functions before the declaration ran, so they read the constant +// normally once the other member's top level calls them. +import { marked } from "./core.ts"; +import { describe, kind, mark } from "./util.ts"; + +console.log(JSON.stringify(marked), kind()); +console.log(JSON.stringify(mark({ seen: false })), describe()); diff --git a/tests/corpus/7380-cycle-deferred-folded-key/util.ts b/tests/corpus/7380-cycle-deferred-folded-key/util.ts new file mode 100644 index 000000000..a038d9e63 --- /dev/null +++ b/tests/corpus/7380-cycle-deferred-folded-key/util.ts @@ -0,0 +1,23 @@ +import { version } from "./core.ts"; + +export function makeCounter(): () => number { + let n = 0; + return () => ++n; +} +const next = makeCounter(); +console.log("util body", next(), next()); + +export const TAG = "~tagged"; + +export function mark(box: Record): Record { + box[TAG] = true; + return box; +} + +export function kind(): string { + return typeof TAG; +} + +export function describe(): string { + return `core v${version}`; +} diff --git a/tests/corpus/7381-cycle-deferred-reached-tdz/alpha.ts b/tests/corpus/7381-cycle-deferred-reached-tdz/alpha.ts new file mode 100644 index 000000000..8dc002c27 --- /dev/null +++ b/tests/corpus/7381-cycle-deferred-reached-tdz/alpha.ts @@ -0,0 +1,13 @@ +import { probe } from "./beta.ts"; + +export function report(): string { + return `alpha ${NAME}`; +} + +export function late(): string { + return `${OTHER} after ${probe}`; +} + +export const NAME = "ready"; +export const OTHER = "other"; +console.log("alpha body", report()); diff --git a/tests/corpus/7381-cycle-deferred-reached-tdz/beta.ts b/tests/corpus/7381-cycle-deferred-reached-tdz/beta.ts new file mode 100644 index 000000000..c3d1551ff --- /dev/null +++ b/tests/corpus/7381-cycle-deferred-reached-tdz/beta.ts @@ -0,0 +1,12 @@ +import { report } from "./alpha.ts"; + +function attempt(): string { + try { + return report(); + } catch (error) { + return error instanceof ReferenceError ? `ReferenceError: ${error.message}` : "other error"; + } +} + +export const probe = attempt(); +console.log("beta body", probe); diff --git a/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts b/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts new file mode 100644 index 000000000..6a0b7e3ca --- /dev/null +++ b/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts @@ -0,0 +1,7 @@ +// The first cycle member to run calls a function of the other member +// before that member declared the constant the function reads: Node's +// temporal-dead-zone ReferenceError. A second function of the same member +// reads another constant but runs only after the cycle finished. +import { late, report } from "./alpha.ts"; + +console.log(report(), late()); diff --git a/tests/corpus/7382-cycle-static-this-tdz/alpha.ts b/tests/corpus/7382-cycle-static-this-tdz/alpha.ts new file mode 100644 index 000000000..ff8eb4a87 --- /dev/null +++ b/tests/corpus/7382-cycle-static-this-tdz/alpha.ts @@ -0,0 +1,11 @@ +import { Probe, Lazy } from "./beta.ts"; + +export { Probe }; +export const label = "ready"; +export const other = "other"; + +export function later(): string { + return `${Lazy.describe()} ${other}`; +} + +console.log("alpha body", Probe.result, Lazy.value); diff --git a/tests/corpus/7382-cycle-static-this-tdz/beta.ts b/tests/corpus/7382-cycle-static-this-tdz/beta.ts new file mode 100644 index 000000000..428907967 --- /dev/null +++ b/tests/corpus/7382-cycle-static-this-tdz/beta.ts @@ -0,0 +1,26 @@ +import { label, other } from "./alpha.ts"; + +function guard(read: () => string): string { + try { + return read(); + } catch (error) { + return error instanceof ReferenceError ? `ReferenceError: ${error.message}` : "other error"; + } +} + +export class Probe { + static helper(): string { + return guard(() => `label ${label}`); + } + static result = this.helper(); +} + +export class Lazy { + static read(): string { + return guard(() => `other ${other}`); + } + static describe(): string { + return `lazy ${other}`; + } + static value = [0].map(() => this.read()).join(""); +} diff --git a/tests/corpus/7382-cycle-static-this-tdz/main.ts b/tests/corpus/7382-cycle-static-this-tdz/main.ts new file mode 100644 index 000000000..e3bbf1581 --- /dev/null +++ b/tests/corpus/7382-cycle-static-this-tdz/main.ts @@ -0,0 +1,8 @@ +// The first cycle member to run builds classes whose static initializers +// call a method through `this`, directly and from an arrow function; both +// methods read constants of the other member before it declared them: +// Node's temporal-dead-zone ReferenceError. A third method reads a constant +// only after the cycle finished. +import { label, Probe, later } from "./alpha.ts"; + +console.log(label, Probe.result, later()); diff --git a/tests/corpus/7383-cycle-class-expression-name-tdz/alpha.ts b/tests/corpus/7383-cycle-class-expression-name-tdz/alpha.ts new file mode 100644 index 000000000..ff8eb4a87 --- /dev/null +++ b/tests/corpus/7383-cycle-class-expression-name-tdz/alpha.ts @@ -0,0 +1,11 @@ +import { Probe, Lazy } from "./beta.ts"; + +export { Probe }; +export const label = "ready"; +export const other = "other"; + +export function later(): string { + return `${Lazy.describe()} ${other}`; +} + +console.log("alpha body", Probe.result, Lazy.value); diff --git a/tests/corpus/7383-cycle-class-expression-name-tdz/beta.ts b/tests/corpus/7383-cycle-class-expression-name-tdz/beta.ts new file mode 100644 index 000000000..d1d89ad1a --- /dev/null +++ b/tests/corpus/7383-cycle-class-expression-name-tdz/beta.ts @@ -0,0 +1,26 @@ +import { label, other } from "./alpha.ts"; + +function guard(read: () => string): string { + try { + return read(); + } catch (error) { + return error instanceof ReferenceError ? `ReferenceError: ${error.message}` : "other error"; + } +} + +export const Probe = class ProbeImpl { + static helper(): string { + return guard(() => `label ${label}`); + } + static result = ProbeImpl.helper(); +}; + +export const Lazy = class LazyImpl { + static read(): string { + return guard(() => `other ${other}`); + } + static describe(): string { + return `lazy ${other}`; + } + static value = [0].map(() => LazyImpl.read()).join(""); +}; diff --git a/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts b/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts new file mode 100644 index 000000000..93b7a47ee --- /dev/null +++ b/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts @@ -0,0 +1,9 @@ +// The first cycle member to run builds classes from named class +// expressions whose static initializers call a method through the inner +// name, directly and from an arrow function; both methods read constants +// of the other member before it declared them: Node's temporal-dead-zone +// ReferenceError. A third method reads a constant only after the cycle +// finished. +import { label, Probe, later } from "./alpha.ts"; + +console.log(label, Probe.result, later()); diff --git a/tests/corpus/7390-record-key-read-presence.ts b/tests/corpus/7390-record-key-read-presence.ts new file mode 100644 index 000000000..76bb355ca --- /dev/null +++ b/tests/corpus/7390-record-key-read-presence.ts @@ -0,0 +1,49 @@ +// Keyed reads of string-indexed records whose values are numbers or +// booleans, consumed by `??`, `!`, comparisons with undefined and typeof: +// declared fields, an optional declared field, index-signature entries and +// missing keys. A stored NaN is a value, so `??` keeps it; -0 survives the +// read; a missing key is undefined. +interface Scores { + base: number; + bonus?: number; + [name: string]: number | undefined; +} + +const scores: Scores = { base: 10 }; +scores["alice"] = 3; +scores["nan"] = NaN; +scores["negzero"] = -0; + +const keys = ["base", "bonus", "alice", "nan", "negzero", "missing"]; +for (const key of keys) { + const read = scores[key]; + console.log( + key, + scores[key] ?? "default", + scores[key] === undefined, + typeof scores[key], + Object.is(scores[key] ?? 1, -0), + (scores[key] ?? 0) + 1, + read, + ); +} +scores.bonus = 5; +console.log("bonus", scores["bonus"] ?? "default", scores["bonus"]! * 2); + +const counts: Record = {}; +for (const word of "a b a c b a d".split(" ")) counts[word] = (counts[word] ?? 0) + 1; +let total = 0; +for (const word in counts) total += counts[word]!; +console.log(JSON.stringify(counts), total, counts["zzz"] ?? -1); + +const flags: Record = { on: true, off: false }; +for (const key of ["on", "off", "unset"]) { + console.log(key, flags[key] ?? "unset", flags[key] === undefined, typeof flags[key]); +} +let seen = 0; +for (let i = 0; i < 1000; i++) { + const key = "k" + (i % 7); + flags[key] = !(flags[key] ?? false); + if (flags[key]!) seen++; +} +console.log(seen, Object.keys(flags).length); diff --git a/tests/corpus/7404-frozen-records.ts b/tests/corpus/7404-frozen-records.ts new file mode 100644 index 000000000..acf9b393b --- /dev/null +++ b/tests/corpus/7404-frozen-records.ts @@ -0,0 +1,124 @@ +// Object.freeze of records that other references still hold: writes +// through any reference then throw Node's TypeError (this module is strict +// mode code; 7405 covers sloppy mode). Covers static freezes, freezes +// through untyped values, Object.isFrozen, keyed writes and deletes, +// optional-field deletes, Object.assign, tuples, and frozen records that are +// released or collected as cycles. +export {}; + +type Counter = { count: number; label: string }; +type Link = { name: string; next?: Link }; + +function report(e: unknown): void { + console.log((e as Error).name + ": " + (e as Error).message); +} + +function freezeUnknown(value: unknown): unknown { + return Object.freeze(value); +} + +function isFrozenUnknown(value: unknown): boolean { + return Object.isFrozen(value); +} + +function deepFreeze(value: unknown): unknown { + if (typeof value === "object" && value !== null) { + Object.freeze(value); + for (const key of Object.keys(value)) deepFreeze((value as Record)[key]); + } + return value; +} + +type Config = { name: string; limits: { max: number; min: number } }; + +function bump(c: Counter): void { + c.count += 1; +} + +function writes(): void { + const a: Counter = { count: 1, label: "a" }; + const alias = a; + console.log(Object.isFrozen(a), isFrozenUnknown(a)); + const frozen = Object.freeze(alias); + console.log(frozen === a, Object.isFrozen(a), isFrozenUnknown(alias)); + try { bump(a); } catch (e) { report(e); } + try { a.label = "b"; } catch (e) { report(e); } + console.log(a.count, a.label, JSON.stringify(a)); + + const viaUnknown: Counter = { count: 10, label: "u" }; + const back = freezeUnknown(viaUnknown); + console.log(back === viaUnknown, Object.isFrozen(viaUnknown)); + try { viaUnknown.count = 11; } catch (e) { report(e); } + console.log(viaUnknown.count); + + const dict: Record = { one: 1 }; + const dictAlias = dict; + Object.freeze(dictAlias); + try { dict["one"] = 2; } catch (e) { report(e); } + try { dict["two"] = 2; } catch (e) { report(e); } + try { delete dict["one"]; } catch (e) { report(e); } + delete dict["three"]; + console.log(JSON.stringify(dict), Object.isFrozen(dict)); + + const optional: { id: number; note?: string } = { id: 1, note: "n" }; + const optionalAlias = optional; + Object.freeze(optionalAlias); + try { delete optional.note; } catch (e) { report(e); } + console.log(JSON.stringify(optional)); + const bare: { id: number; note?: string } = { id: 2 }; + Object.freeze(bare); + delete bare.note; + console.log(JSON.stringify(bare)); + + const target: Counter = { count: 0, label: "t" }; + Object.freeze(target); + try { Object.assign(target, { count: 5, label: "x" }); } catch (e) { report(e); } + console.log(JSON.stringify(target)); + + const pair: [string, number] = ["p", 1]; + Object.freeze(pair); + try { pair[1] = 2; } catch (e) { report(e); } + console.log(pair[0], pair[1]); +} + +function deepFreezes(): void { + const config: Config = JSON.parse('{"name":"c","limits":{"max":2,"min":1}}'); + const limits = config.limits; + console.log(Object.isFrozen(config), Object.isFrozen(limits)); + deepFreeze(config); + console.log(Object.isFrozen(config), Object.isFrozen(limits), isFrozenUnknown(config.limits)); + try { config.limits.max = 3; } catch (e) { report(e); } + try { limits.min = 0; } catch (e) { report(e); } + try { config.name = "d"; } catch (e) { report(e); } + console.log(JSON.stringify(config)); +} + +function lifetimes(): void { + const list: Counter[] = []; + for (let i = 0; i < 4; i++) { + const item: Counter = { count: i, label: "x" + i }; + list.push(item); + if (i % 2 === 1) Object.freeze(item); + } + for (const item of list) { + try { item.count = item.count * 10; } catch (e) { console.log(item.label, (e as Error).message); } + } + console.log(JSON.stringify(list)); + const kept = list[1]!; + list.length = 0; + console.log(kept.count, Object.isFrozen(kept)); + + for (let round = 0; round < 3; round++) { + const first: Link = { name: "first" + round }; + const second: Link = { name: "second" + round, next: first }; + first.next = second; + Object.freeze(first); + Object.freeze(second); + try { first.name = "changed"; } catch (e) { report(e); } + console.log(first.next?.name, second.next?.name); + } +} + +writes(); +deepFreezes(); +lifetimes(); diff --git a/tests/corpus/7405-frozen-records-sloppy.cjs b/tests/corpus/7405-frozen-records-sloppy.cjs new file mode 100644 index 000000000..2b4e29213 --- /dev/null +++ b/tests/corpus/7405-frozen-records-sloppy.cjs @@ -0,0 +1,53 @@ +// Writes to frozen records in sloppy-mode code leave them unchanged, where +// strict mode code throws (7404). The records are typed through JSDoc, so +// the writes are native record writes. + +/** @typedef {{ count: number, label: string }} Counter */ +/** @typedef {{ id: number, note?: string }} Note */ +/** @typedef {{ kind: "a", name: string }} Small */ +/** @typedef {{ kind: "b", name: string, size: number }} Large */ + +/** @param {Counter} c */ +function bump(c) { + c.count += 1; + c.label = c.label + "!"; +} + +/** @param {Note} n */ +function dropNote(n) { + delete n.note; +} + +/** @param {number} count @returns {Counter} */ +function counter(count) { + return { count: count, label: "c" + count }; +} + +const open = counter(1); +const frozen = Object.freeze(counter(2)); +bump(open); +bump(frozen); +console.log(JSON.stringify(open), JSON.stringify(frozen), Object.isFrozen(frozen)); + +/** @type {Note} */ +const note = { id: 1, note: "n" }; +Object.freeze(note); +dropNote(note); +console.log(JSON.stringify(note)); + +/** @param {Small | Large} shape */ +function rename(shape) { + shape.name = "renamed"; +} + +/** @param {boolean} large @returns {Small | Large} */ +function shapeOf(large) { + return large ? { kind: "b", name: "b", size: 1 } : { kind: "a", name: "a" }; +} + +const openShape = shapeOf(true); +const frozenShape = shapeOf(false); +Object.freeze(frozenShape); +rename(openShape); +rename(frozenShape); +console.log(openShape.name, frozenShape.name); diff --git a/tests/corpus/7406-frozen-containers.ts b/tests/corpus/7406-frozen-containers.ts new file mode 100644 index 000000000..a8ba8e35c --- /dev/null +++ b/tests/corpus/7406-frozen-containers.ts @@ -0,0 +1,142 @@ +// Object.freeze of arrays, Maps and Sets that other references still hold. +// A frozen array refuses every write through any reference with Node's +// TypeError for the operation's first write (this module is strict mode +// code). A frozen Map or Set only stops being extensible: its entries stay +// writable, and Object.isFrozen, isSealed and isExtensible report it. +export {}; + +function report(e: unknown): void { + console.log((e as Error).name + ": " + (e as Error).message); +} + +function freezeUnknown(value: unknown): unknown { + return Object.freeze(value); +} + +function writeAny(value: any): void { + value[0] = 7; +} + +function pushAny(value: any): void { + value.push(8); +} + +function integrity(value: unknown): string { + return [Object.isFrozen(value), Object.isSealed(value), Object.isExtensible(value)].join(" "); +} + +function arrays(): void { + const a: number[] = [1, 2, 3]; + const alias = a; + console.log(Object.isFrozen(a), integrity(a)); + Object.freeze(alias); + console.log(Object.isFrozen(a), integrity(a)); + try { a[0] = 9; } catch (e) { report(e); } + try { a[5] = 9; } catch (e) { report(e); } + try { a.length = 0; } catch (e) { report(e); } + try { a.push(4); } catch (e) { report(e); } + try { a.pop(); } catch (e) { report(e); } + try { a.shift(); } catch (e) { report(e); } + try { a.unshift(0); } catch (e) { report(e); } + try { a.splice(0, 1); } catch (e) { report(e); } + try { a.splice(1, 0, 7); } catch (e) { report(e); } + try { a.reverse(); } catch (e) { report(e); } + try { a.sort(); } catch (e) { report(e); } + try { a.fill(0, 1); } catch (e) { report(e); } + try { a.copyWithin(1, 0); } catch (e) { report(e); } + console.log(a.join(), a.length); + + const one: string[] = ["x"]; + Object.freeze(one); + console.log(one.reverse().join(), one.sort().join(), one.copyWithin(0, 1).join()); + try { one.splice(0, 1); } catch (e) { report(e); } + try { one.shift(); } catch (e) { report(e); } + const none: string[] = []; + Object.freeze(none); + console.log(none.reverse().length, none.fill("y").length); + try { none.pop(); } catch (e) { report(e); } + try { none.splice(0, 0, "z"); } catch (e) { report(e); } + + let calls = 0; + const byValue = (x: number, y: number): number => { + calls++; + return x - y; + }; + const unsorted: number[] = [3, 1, 2]; + Object.freeze(unsorted); + try { unsorted.sort(byValue); } catch (e) { report(e); } + const single: number[] = [5]; + Object.freeze(single); + console.log(unsorted.join(), calls > 0, single.sort(byValue).join()); + + // A comparator that freezes the array stops the sort's write-back. + const freezing: number[] = [3, 1, 2]; + try { + freezing.sort((x, y) => { + Object.freeze(freezing); + return x - y; + }); + } catch (e) { report(e); } + console.log(freezing.join(), Object.isFrozen(freezing)); + + const open: number[] = [4, 5]; + open.push(6); + open[0] = 1; + console.log(open.join(), Object.isFrozen(open)); +} + +function throughUnknown(): void { + const a: number[] = [1, 2]; + console.log(integrity(a)); + const back = freezeUnknown(a); + console.log(back === a, Object.isFrozen(a), integrity(a)); + try { a.push(3); } catch (e) { report(e); } + try { a[1] = 5; } catch (e) { report(e); } + try { writeAny(a); } catch (e) { report(e); } + try { pushAny(a); } catch (e) { report(e); } + console.log(a.join()); + + const rows: number[][] = [[1], [2]]; + freezeUnknown(rows[1]); + rows[0]!.push(10); + try { rows[1]!.push(20); } catch (e) { report(e); } + console.log(JSON.stringify(rows), Object.isFrozen(rows), Object.isFrozen(rows[1])); +} + +function collections(): void { + const m = new Map([["x", 1]]); + console.log(integrity(m)); + const frozen = Object.freeze(m); + m.set("y", 2); + m.delete("x"); + console.log(frozen === m, Object.isFrozen(m), Object.isSealed(m), Object.isExtensible(m), m.size); + const s = new Set([1]); + freezeUnknown(s); + s.add(2); + console.log(integrity(s), s.size); + const sealed = new Set(); + Object.seal(sealed); + const fresh = new Map(); + Object.preventExtensions(fresh); + console.log(Object.isFrozen(sealed), Object.isExtensible(fresh), integrity(new Map())); +} + +function lifetimes(): void { + const kept: string[][] = []; + for (let round = 0; round < 3; round++) { + const row = ["r" + round, "s" + round]; + if (round !== 1) Object.freeze(row); + kept.push(row); + } + for (const row of kept) { + try { row[0] = "changed"; } catch (e) { console.log(row[1], (e as Error).message); } + } + console.log(JSON.stringify(kept)); + kept.length = 0; + console.log(kept.length); +} + +arrays(); +throughUnknown(); +collections(); +lifetimes(); diff --git a/tests/corpus/7445-crypto-key-pair-options.mjs b/tests/corpus/7445-crypto-key-pair-options.mjs new file mode 100644 index 000000000..2e8f7e8b3 --- /dev/null +++ b/tests/corpus/7445-crypto-key-pair-options.mjs @@ -0,0 +1,117 @@ +// generateKeyPairSync and generateKeyPair check their arguments in Node's +// order: the callback, the type, the options object, the public and then +// the private key encoding, and then the options of the key type. The +// encodings follow Node's parseKeyEncoding rules for formats, types, +// ciphers and passphrases. +import * as crypto from "node:crypto"; + +function attempt(label, run) { + try { + const result = run(); + console.log(label, result === undefined ? "returned" : Object.keys(result)); + } catch (e) { + console.log(label, "throws", e.name, e.code, e.message); + } +} + +const ec = (options) => () => crypto.generateKeyPairSync("ec", { namedCurve: "P-256", ...options }); +const rsa = (options) => () => crypto.generateKeyPairSync("rsa", { modulusLength: 1024, ...options }); + +// The type, then the options object, then the encodings, then the rest. +attempt("number type", () => crypto.generateKeyPairSync(5, 7)); +attempt("unknown type, bad encoding", () => crypto.generateKeyPairSync("foo", { publicKeyEncoding: 5 })); +attempt("unknown type, number options", () => crypto.generateKeyPairSync("foo", 5)); +attempt("unknown type, null options", () => crypto.generateKeyPairSync("foo", null)); +attempt("unknown type, array options", () => crypto.generateKeyPairSync("foo", [])); +attempt("unknown type", () => crypto.generateKeyPairSync("foo")); +attempt("upper-case types", () => crypto.generateKeyPairSync("RSA", {})); +attempt("upper-case Ed25519", () => crypto.generateKeyPairSync("Ed25519")); +attempt("upper-case ML-DSA", () => crypto.generateKeyPairSync("ML-DSA-44")); +attempt("unknown type, pkcs1", () => crypto.generateKeyPairSync("foo", { publicKeyEncoding: { format: "pem", type: "pkcs1" } })); +attempt("rsa, bad modulusLength and encoding", () => crypto.generateKeyPairSync("rsa", { modulusLength: "x", publicKeyEncoding: 5 })); +attempt("rsa, bad modulusLength and private encoding", () => crypto.generateKeyPairSync("rsa", { modulusLength: "x", privateKeyEncoding: 5 })); +attempt("rsa, negative modulusLength and format", () => crypto.generateKeyPairSync("rsa", { modulusLength: -1, publicKeyEncoding: { format: "x" } })); +attempt("rsa, bad exponent and encoding", () => crypto.generateKeyPairSync("rsa", { modulusLength: 1024, publicExponent: "x", publicKeyEncoding: 5 })); +attempt("rsa-pss, bad encoding", () => crypto.generateKeyPairSync("rsa-pss", { publicKeyEncoding: 5 })); +attempt("rsa-pss, bad modulusLength", () => crypto.generateKeyPairSync("rsa-pss", { modulusLength: "x" })); +attempt("dsa, bad encoding", () => crypto.generateKeyPairSync("dsa", { publicKeyEncoding: 5 })); +attempt("dsa, no options", () => crypto.generateKeyPairSync("dsa")); +attempt("dh, number options", () => crypto.generateKeyPairSync("dh", 5)); +attempt("ec, no curve and bad encoding", () => crypto.generateKeyPairSync("ec", { publicKeyEncoding: 5 })); +attempt("ec, unknown curve and bad encoding", () => crypto.generateKeyPairSync("ec", { namedCurve: "nope", publicKeyEncoding: 5 })); +attempt("ec, number curve", () => crypto.generateKeyPairSync("ec", { namedCurve: 5 })); +attempt("ec, lower-case NIST name", () => crypto.generateKeyPairSync("ec", { namedCurve: "p-256" })); +attempt("ec, unknown curve and paramEncoding", () => crypto.generateKeyPairSync("ec", { namedCurve: "nope", paramEncoding: "bogus" })); +attempt("ec, paramEncoding bogus", ec({ paramEncoding: "bogus" })); +attempt("ec, paramEncoding named", () => crypto.generateKeyPairSync("ec", { namedCurve: "P-256", paramEncoding: "named" })); +attempt("ec, paramEncoding null", () => crypto.generateKeyPairSync("ec", { namedCurve: "P-256", paramEncoding: null })); +attempt("ed25519, no options", () => crypto.generateKeyPairSync("ed25519")); +attempt("ed25519, null options", () => crypto.generateKeyPairSync("ed25519", null)); +attempt("ed25519, number options", () => crypto.generateKeyPairSync("ed25519", 5)); +attempt("ed25519, bad encoding", () => crypto.generateKeyPairSync("ed25519", { publicKeyEncoding: 5 })); + +// Encodings. +attempt("both encodings invalid", rsa({ publicKeyEncoding: { format: "x" }, privateKeyEncoding: 5 })); +attempt("array encoding", rsa({ publicKeyEncoding: [] })); +attempt("null encodings", () => crypto.generateKeyPairSync("ec", { namedCurve: "P-256", publicKeyEncoding: null, privateKeyEncoding: null })); +attempt("der without type", rsa({ publicKeyEncoding: { format: "der" } })); +attempt("pem without type", rsa({ privateKeyEncoding: { format: "pem" } })); +attempt("jwk without type", () => crypto.generateKeyPairSync("ec", { namedCurve: "P-256", publicKeyEncoding: { format: "jwk" }, privateKeyEncoding: { format: "jwk" } })); +attempt("jwk with bad type", rsa({ publicKeyEncoding: { format: "jwk", type: "bogus" } })); +attempt("jwk with pkcs1 for ec", ec({ publicKeyEncoding: { format: "jwk", type: "pkcs1" } })); +attempt("public sec1", rsa({ publicKeyEncoding: { format: "pem", type: "sec1" } })); +attempt("public pkcs8", rsa({ publicKeyEncoding: { format: "pem", type: "pkcs8" } })); +attempt("private spki", rsa({ privateKeyEncoding: { format: "pem", type: "spki" } })); +attempt("private sec1 for rsa", rsa({ privateKeyEncoding: { format: "pem", type: "sec1" } })); +attempt("public pkcs1 for ec", ec({ publicKeyEncoding: { format: "pem", type: "pkcs1" } })); +attempt("private pkcs1 for ec", ec({ privateKeyEncoding: { format: "der", type: "pkcs1" } })); +attempt("public pkcs1 for ed25519", () => crypto.generateKeyPairSync("ed25519", { publicKeyEncoding: { format: "pem", type: "pkcs1" } })); +attempt("private sec1 for ed25519", () => crypto.generateKeyPairSync("ed25519", { privateKeyEncoding: { format: "pem", type: "sec1" } })); +attempt("raw-public as private", ec({ privateKeyEncoding: { format: "raw-public" } })); +attempt("raw-private as public", ec({ publicKeyEncoding: { format: "raw-private" } })); +attempt("raw-seed as public", ec({ publicKeyEncoding: { format: "raw-seed" } })); +attempt("raw-public with bad type", ec({ publicKeyEncoding: { format: "raw-public", type: "x" } })); +attempt("raw-private with a type", ec({ privateKeyEncoding: { format: "raw-private", type: "pkcs8" } })); +attempt("raw-private with a passphrase", ec({ privateKeyEncoding: { format: "raw-private", passphrase: "x" } })); +attempt("raw-seed with a cipher", ec({ privateKeyEncoding: { format: "raw-seed", cipher: "aes-128-cbc" } })); +attempt("number cipher", rsa({ privateKeyEncoding: { format: "pem", type: "pkcs8", cipher: 5 } })); +attempt("passphrase without cipher", rsa({ privateKeyEncoding: { format: "pem", type: "pkcs8", passphrase: "x" } })); +attempt("passphrase with null cipher", rsa({ privateKeyEncoding: { format: "pem", type: "pkcs8", cipher: null, passphrase: "x" } })); +attempt("der pkcs1 with a cipher", rsa({ privateKeyEncoding: { format: "der", type: "pkcs1", cipher: "aes-128-cbc", passphrase: "x" } })); +attempt("der sec1 with a cipher", ec({ privateKeyEncoding: { format: "der", type: "sec1", cipher: "aes-128-cbc", passphrase: "x" } })); +attempt("number passphrase", rsa({ privateKeyEncoding: { format: "pem", type: "pkcs8", cipher: "aes-128-cbc", passphrase: 5 } })); +attempt("missing passphrase", rsa({ privateKeyEncoding: { format: "pem", type: "pkcs8", cipher: "aes-128-cbc" } })); +attempt("encoding errors before the modulusLength", () => crypto.generateKeyPairSync("rsa", { modulusLength: 1.5, privateKeyEncoding: { format: "pem", type: "pkcs8", passphrase: "x" } })); + +// generateKeyPair checks its callback first, reads a function in the +// options position as the callback, and throws argument errors. +attempt("async, options as the callback", () => crypto.generateKeyPair("rsa", { modulusLength: 1024 })); +attempt("async, undefined callback", () => crypto.generateKeyPair("rsa", { modulusLength: 1024 }, undefined)); +attempt("async, number callback", () => crypto.generateKeyPair("rsa", { modulusLength: 1024 }, 5)); +attempt("async, number type without callback", () => crypto.generateKeyPair(5, {})); +attempt("async, unknown type", () => crypto.generateKeyPair("foo", undefined, () => console.log("never"))); +attempt("async, ec without options", () => crypto.generateKeyPair("ec", () => console.log("never"))); +attempt("async, bad encoding", () => crypto.generateKeyPair("ec", { namedCurve: "P-256", publicKeyEncoding: { format: "pem", type: "pkcs1" } }, () => console.log("never"))); +attempt("async, shifted callback", () => + crypto.generateKeyPair("ed25519", (err, publicKey, privateKey) => { + console.log("shifted callback", err, publicKey.asymmetricKeyType, privateKey.type); + crypto.generateKeyPair( + "ed25519", + (err2, publicKey2) => { + console.log("options callback", err2, publicKey2.type); + // A callable Proxy is a function, in either position. + const proxied = new Proxy( + (err3, publicKey3) => { + console.log("proxy callback", err3, publicKey3.type); + crypto.generateKeyPair("ed25519", proxiedOptions); + }, + {}, + ); + const proxiedOptions = new Proxy((err4, publicKey4) => console.log("proxy options callback", err4, publicKey4.type), {}); + crypto.generateKeyPair("ed25519", {}, proxied); + }, + () => console.log("never"), + ); + }), +); +console.log("returned"); diff --git a/tests/corpus/7446-crypto-rsa-key-pair-errors.ts b/tests/corpus/7446-crypto-rsa-key-pair-errors.ts new file mode 100644 index 000000000..ca5416bcd --- /dev/null +++ b/tests/corpus/7446-crypto-rsa-key-pair-errors.ts @@ -0,0 +1,72 @@ +// RSA key generation parameters: publicExponent's validateUint32 checks +// and its 65537 default, and the errors OpenSSL reports for modulus +// lengths and exponents it does not generate, in OpenSSL's order. +// generateKeyPairSync throws those errors and generateKeyPair passes them +// to its callback, alone, in a later macrotask. +import * as crypto from "node:crypto"; + +type RsaOptions = { modulusLength: number; publicExponent?: unknown }; + +function attempt(label: string, options: RsaOptions): void { + try { + const { publicKey } = crypto.generateKeyPairSync("rsa", options as crypto.RSAKeyPairKeyObjectOptions); + const details = publicKey.asymmetricKeyDetails!; + console.log(label, details.modulusLength, details.publicExponent); + } catch (e) { + const error = e as NodeJS.ErrnoException; + console.log(label, "throws", error.name, error.code, error.message); + } +} + +attempt("exponent 1.5", { modulusLength: 1024, publicExponent: 1.5 }); +attempt("exponent NaN", { modulusLength: 1024, publicExponent: NaN }); +attempt("exponent Infinity", { modulusLength: 1024, publicExponent: Infinity }); +attempt("exponent -1", { modulusLength: 1024, publicExponent: -1 }); +attempt("exponent 2 ** 32", { modulusLength: 1024, publicExponent: 2 ** 32 }); +attempt("exponent string", { modulusLength: 1024, publicExponent: "3" }); +attempt("exponent bigint", { modulusLength: 1024, publicExponent: 3n }); +attempt("exponent null", { modulusLength: 1024, publicExponent: null }); +attempt("exponent undefined", { modulusLength: 1024, publicExponent: undefined }); +attempt("exponent 3", { modulusLength: 1024, publicExponent: 3 }); +attempt("exponent 17", { modulusLength: 1024, publicExponent: 17 }); +attempt("exponent 0", { modulusLength: 1024, publicExponent: 0 }); +attempt("exponent -0", { modulusLength: 1024, publicExponent: -0 }); +attempt("exponent 1", { modulusLength: 1024, publicExponent: 1 }); +attempt("exponent 2", { modulusLength: 1024, publicExponent: 2 }); +attempt("exponent 4", { modulusLength: 1024, publicExponent: 4 }); +attempt("exponent 2 ** 31", { modulusLength: 1024, publicExponent: 2 ** 31 }); +attempt("modulusLength 0", { modulusLength: 0 }); +attempt("modulusLength -0", { modulusLength: -0 }); +attempt("modulusLength 100", { modulusLength: 100 }); +attempt("modulusLength 511", { modulusLength: 511 }); +attempt("modulusLength 2 ** 31", { modulusLength: 2 ** 31 }); +attempt("modulusLength 2 ** 32 - 1", { modulusLength: 2 ** 32 - 1 }); +attempt("modulusLength 100, exponent 4", { modulusLength: 100, publicExponent: 4 }); +attempt("modulusLength 100, exponent 0", { modulusLength: 100, publicExponent: 0 }); +attempt("modulusLength 2 ** 31, exponent 0", { modulusLength: 2 ** 31, publicExponent: 0 }); +attempt("modulusLength 2 ** 31, exponent 4", { modulusLength: 2 ** 31, publicExponent: 4 }); + +function later(label: string, options: RsaOptions, next: () => void): void { + try { + crypto.generateKeyPair("rsa", options as crypto.RSAKeyPairKeyObjectOptions, (...args: unknown[]) => { + const error = args[0] as Error | null; + console.log(label, "callback", args.length, error === null ? null : [error.name, error.message]); + next(); + }); + console.log(label, "returned"); + } catch (e) { + console.log(label, "throws", (e as Error).message); + next(); + } +} + +later("async exponent 4", { modulusLength: 1024, publicExponent: 4 }, () => + later("async modulusLength 100", { modulusLength: 100 }, () => + later("async exponent 0", { modulusLength: 1024, publicExponent: 0 }, () => + later("async exponent 1.5", { modulusLength: 1024, publicExponent: 1.5 }, () => + later("async exponent 3", { modulusLength: 1024, publicExponent: 3 }, () => console.log("done")), + ), + ), + ), +); +process.nextTick(() => console.log("tick")); diff --git a/tests/corpus/7447-union-record-arm-width.mjs b/tests/corpus/7447-union-record-arm-width.mjs new file mode 100644 index 000000000..dc12f8a59 --- /dev/null +++ b/tests/corpus/7447-union-record-arm-width.mjs @@ -0,0 +1,37 @@ +// Values read back from arrays whose elements mix object literals of +// different widths, an empty object among them, keep every field: the +// record arm with more fields is tried before `{}`, which matches every +// object. +const pairs = [ + ["a", { publicKeyEncoding: 5 }], + ["b", {}], + ["c", { publicKeyEncoding: 6, extra: true }], +]; +for (const [label, value] of pairs) console.log(label, Object.keys(value), JSON.stringify(value)); + +const options = [ + ["rsa", { modulusLength: 1024 }], + ["ec", { namedCurve: "P-256", paramEncoding: "named" }], + ["none", {}], +]; +for (const [type, value] of options) { + const { modulusLength, namedCurve, paramEncoding } = value; + console.log(type, modulusLength, namedCurve, paramEncoding); +} + +const mixed = [{ a: 1 }, {}, "s", { a: 2, b: "two" }]; +for (const value of mixed) { + console.log(typeof value, JSON.stringify(value), typeof value === "object" ? Object.keys(value).length : -1); +} + +function describe(value) { + return Object.entries(value) + .map(([key, field]) => `${key}=${field}`) + .join(";"); +} +const rows = [ + [1, { x: 1, y: 2 }], + [2, {}], + [3, { x: 3 }], +]; +for (const [id, value] of rows) console.log(id, describe(value), value.x, value.y); diff --git a/tests/corpus/7448-union-record-arm-width.ts b/tests/corpus/7448-union-record-arm-width.ts new file mode 100644 index 000000000..1518dd575 --- /dev/null +++ b/tests/corpus/7448-union-record-arm-width.ts @@ -0,0 +1,21 @@ +// A checked cast to a union of record types keeps every field of the +// value: arms with more required fields, then more fields, are tried +// before an arm whose fields are all optional, which matches every object. +type Full = { p: number; q: string }; +type Bare = { p?: number }; +type Other = { r: boolean }; + +const values = JSON.parse('[{"p":5,"q":"x"},{},{"r":true},{"p":7}]') as (Full | Bare | Other)[]; +for (const value of values) { + const q = "q" in value ? value.q : "-"; + const r = "r" in value ? value.r : "-"; + console.log(JSON.stringify(value), "q" in value, "r" in value, q, r); +} + +const one = JSON.parse('{"p":1,"q":"y"}') as Bare | Full; +console.log(JSON.stringify(one), "q" in one ? one.q : "no q"); + +type Point = { x: number; y?: number }; +type Labeled = { x: number; label: string }; +const points = JSON.parse('[{"x":1,"label":"a"},{"x":2,"y":3},{"x":4}]') as (Point | Labeled)[]; +for (const point of points) console.log(point.x, "label" in point ? point.label : "none", "y" in point ? point.y : "none"); diff --git a/tests/corpus/7449-crypto-raw-keys.mjs b/tests/corpus/7449-crypto-raw-keys.mjs new file mode 100644 index 000000000..0795dbb0b --- /dev/null +++ b/tests/corpus/7449-crypto-raw-keys.mjs @@ -0,0 +1,120 @@ +// The raw-public, raw-private and raw-seed key formats: generateKeyPair +// encodings, KeyObject.export and createPublicKey/createPrivateKey with +// asymmetricKeyType and namedCurve, for EC keys (uncompressed and +// compressed points, scalars at the curve's size) and Ed25519 keys, and +// Node's errors for the other key types and formats. +import * as crypto from "node:crypto"; + +function describe(value) { + if (Buffer.isBuffer(value)) return `Buffer(${value.length})`; + if (value instanceof crypto.KeyObject) return `${value.type} ${value.asymmetricKeyType} ${JSON.stringify(value.asymmetricKeyDetails)}`; + if (value && typeof value === "object") return Object.entries(value).map(([key, field]) => `${key}=${describe(field)}`).join(" "); + return String(value); +} + +function attempt(label, run) { + try { + console.log(label, describe(run())); + } catch (e) { + console.log(label, "throws", e.name, e.code, e.message); + } +} + +// generateKeyPair encodings +for (const namedCurve of ["P-256", "P-384", "P-521", "secp256k1"]) { + attempt(`ec ${namedCurve} raw-public`, () => crypto.generateKeyPairSync("ec", { namedCurve, publicKeyEncoding: { format: "raw-public" } })); + attempt(`ec ${namedCurve} compressed`, () => crypto.generateKeyPairSync("ec", { namedCurve, publicKeyEncoding: { format: "raw-public", type: "compressed" } })); + attempt(`ec ${namedCurve} raw-private`, () => crypto.generateKeyPairSync("ec", { namedCurve, privateKeyEncoding: { format: "raw-private" } })); +} +attempt("ec raw-seed", () => crypto.generateKeyPairSync("ec", { namedCurve: "P-256", privateKeyEncoding: { format: "raw-seed" } })); +attempt("ed25519 raw", () => + crypto.generateKeyPairSync("ed25519", { publicKeyEncoding: { format: "raw-public" }, privateKeyEncoding: { format: "raw-private" } }), +); +attempt("ed25519 raw-seed", () => crypto.generateKeyPairSync("ed25519", { privateKeyEncoding: { format: "raw-seed" } })); +attempt("rsa raw-public", () => crypto.generateKeyPairSync("rsa", { modulusLength: 1024, publicKeyEncoding: { format: "raw-public" } })); + +// export +const ec = crypto.generateKeyPairSync("ec", { namedCurve: "P-256" }); +const ed = crypto.generateKeyPairSync("ed25519"); +const rsa = crypto.generateKeyPairSync("rsa", { modulusLength: 1024 }); +const point = Buffer.from(ec.publicKey.export({ format: "raw-public" })); +const compressed = Buffer.from(ec.publicKey.export({ format: "raw-public", type: "compressed" })); +const scalar = Buffer.from(ec.privateKey.export({ format: "raw-private" })); +const jwk = ec.privateKey.export({ format: "jwk" }); +console.log("ec point", point.length, point[0], point.subarray(1, 33).toString("base64url") === jwk.x, point.subarray(33).toString("base64url") === jwk.y); +console.log("ec compressed", compressed.length, compressed[0] === 2 + (point[64] & 1), compressed.subarray(1).equals(point.subarray(1, 33))); +console.log("ec scalar", scalar.length, scalar.toString("base64url") === jwk.d); +console.log("ec uncompressed", Buffer.from(ec.publicKey.export({ format: "raw-public", type: "uncompressed" })).equals(point)); +const edPublic = Buffer.from(ed.publicKey.export({ format: "raw-public" })); +const edSeed = Buffer.from(ed.privateKey.export({ format: "raw-private" })); +const edJwk = ed.privateKey.export({ format: "jwk" }); +console.log("ed25519", edPublic.toString("base64url") === edJwk.x, edSeed.toString("base64url") === edJwk.d); +attempt("ed25519 raw-public with a type", () => ed.publicKey.export({ format: "raw-public", type: "x" })); +attempt("ec bad point type", () => ec.publicKey.export({ format: "raw-public", type: "x" })); +attempt("ec null point type", () => ec.publicKey.export({ format: "raw-public", type: null })); +attempt("private raw-public", () => ec.privateKey.export({ format: "raw-public" })); +attempt("public raw-private", () => ec.publicKey.export({ format: "raw-private" })); +attempt("ec raw-seed", () => ec.privateKey.export({ format: "raw-seed" })); +attempt("ed25519 raw-seed", () => ed.privateKey.export({ format: "raw-seed" })); +attempt("rsa raw-public", () => rsa.publicKey.export({ format: "raw-public" })); +attempt("rsa raw-private", () => rsa.privateKey.export({ format: "raw-private" })); +for (const format of [undefined, 5, "jwk", "raw-private", "raw-seed"]) + attempt(`passphrase with ${format}`, () => ec.privateKey.export({ format, passphrase: "x" })); + +// import +const pub = (key) => () => crypto.createPublicKey(key); +const priv = (key) => () => crypto.createPrivateKey(key); +attempt("ec point", pub({ key: point, format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("ec compressed point", pub({ key: compressed, format: "raw-public", asymmetricKeyType: "ec", namedCurve: "prime256v1" })); +attempt("ec scalar", priv({ key: scalar, format: "raw-private", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("ec scalar as public", pub({ key: scalar, format: "raw-private", asymmetricKeyType: "ec", namedCurve: "P-256" })); +const reread = crypto.createPublicKey({ key: compressed, format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-256" }); +console.log("ec reread", reread.equals(ec.publicKey), Buffer.from(reread.export({ format: "raw-public" })).equals(point)); +const rereadPrivate = crypto.createPrivateKey({ key: scalar, format: "raw-private", asymmetricKeyType: "ec", namedCurve: "P-256" }); +console.log("ec private reread", crypto.createPublicKey(rereadPrivate).equals(ec.publicKey)); +attempt("ed25519 public", pub({ key: edPublic, format: "raw-public", asymmetricKeyType: "ed25519" })); +attempt("ed25519 seed", priv({ key: edSeed, format: "raw-private", asymmetricKeyType: "ed25519" })); +console.log("ed25519 reread", crypto.createPublicKey({ key: edSeed, format: "raw-private", asymmetricKeyType: "ed25519" }).equals(ed.publicKey)); + +// fixed keys: the EC scalar 1 is the generator, and an Ed25519 seed of +// zeros has a known public key +const one = Buffer.concat([Buffer.alloc(31), Buffer.from([1])]); +const generator = crypto.createPrivateKey({ key: one, format: "raw-private", asymmetricKeyType: "ec", namedCurve: "P-256" }); +console.log("generator", Buffer.from(crypto.createPublicKey(generator).export({ format: "raw-public", type: "compressed" })).toString("hex")); +const zeros = crypto.createPrivateKey({ key: Buffer.alloc(32), format: "raw-private", asymmetricKeyType: "ed25519" }); +console.log("zero seed", Buffer.from(crypto.createPublicKey(zeros).export({ format: "raw-public" })).toString("hex")); +const message = Buffer.from("raw keys"); +console.log("zero seed signature", crypto.sign(null, message, zeros).toString("hex").slice(0, 32)); + +// import errors +attempt("raw-public to createPrivateKey", priv({ key: point, format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("string data", pub({ key: "abc", format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("no asymmetricKeyType", pub({ key: point, format: "raw-public" })); +attempt("number asymmetricKeyType", pub({ key: point, format: "raw-public", asymmetricKeyType: 5 })); +attempt("no namedCurve", pub({ key: point, format: "raw-public", asymmetricKeyType: "ec" })); +attempt("unknown asymmetricKeyType", pub({ key: point, format: "raw-public", asymmetricKeyType: "nope" })); +attempt("Ed25519 spelling", pub({ key: edPublic, format: "raw-public", asymmetricKeyType: "Ed25519" })); +attempt("unknown curve", pub({ key: point, format: "raw-public", asymmetricKeyType: "ec", namedCurve: "nope" })); +attempt("lower-case NIST curve", pub({ key: point, format: "raw-public", asymmetricKeyType: "ec", namedCurve: "p-256" })); +attempt("other curve", pub({ key: point, format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-384" })); +attempt("short point", pub({ key: point.subarray(0, 64), format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("off-curve point", pub({ key: Buffer.concat([Buffer.from([4]), Buffer.alloc(64, 1)]), format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("bad compressed x", pub({ key: Buffer.concat([Buffer.from([2]), Buffer.alloc(32, 0xff)]), format: "raw-public", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("short scalar", priv({ key: scalar.subarray(0, 31), format: "raw-private", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("long scalar", priv({ key: Buffer.concat([Buffer.alloc(1), scalar]), format: "raw-private", asymmetricKeyType: "ec", namedCurve: "P-256" })); +attempt("ec raw-seed", priv({ key: scalar, format: "raw-seed", asymmetricKeyType: "ec", namedCurve: "nope" })); +attempt("ed25519 raw-seed", priv({ key: edSeed, format: "raw-seed", asymmetricKeyType: "ed25519" })); +attempt("short ed25519 key", pub({ key: edPublic.subarray(0, 31), format: "raw-public", asymmetricKeyType: "ed25519" })); +attempt("short ed25519 seed", priv({ key: edSeed.subarray(0, 31), format: "raw-private", asymmetricKeyType: "ed25519" })); +for (const asymmetricKeyType of ["rsa", "rsa-pss", "dsa", "dh"]) + attempt(`${asymmetricKeyType} raw`, pub({ key: edPublic, format: "raw-public", asymmetricKeyType })); + +// generateKeyPair passes an encoding the key type has no form for to its +// callback +crypto.generateKeyPair("ed25519", { privateKeyEncoding: { format: "raw-seed" } }, (...args) => { + console.log("async raw-seed", args.length, args[0].code); + crypto.generateKeyPair("ec", { namedCurve: "P-384", publicKeyEncoding: { format: "raw-public", type: "compressed" } }, (err, publicKey, privateKey) => { + console.log("async raw-public", err, describe(publicKey), describe(privateKey)); + }); +}); +console.log("returned"); diff --git a/tests/corpus/7460-transform-writev.mjs b/tests/corpus/7460-transform-writev.mjs new file mode 100644 index 000000000..d78bdfc77 --- /dev/null +++ b/tests/corpus/7460-transform-writev.mjs @@ -0,0 +1,33 @@ +// writev on a Transform and a PassThrough (the writev option): a batch of +// buffered writes reaches writev instead of the transform, a lone write +// still reaches the transform, and the readable side gets what writev +// pushes. +import { Transform, PassThrough } from "node:stream"; +const t = new Transform({ + transform(chunk, enc, cb) { console.log("transform", String(chunk), enc); cb(null, chunk); }, + writev(chunks, cb) { + console.log("writev", chunks.map((c) => `${c.chunk}/${c.encoding}`).join(",")); + for (const { chunk } of chunks) this.push(String(chunk).toUpperCase()); + cb(); + }, +}); +t.on("data", (d) => console.log("data", String(d))); +t.on("finish", () => console.log("finish")); +t.on("end", () => console.log("end")); +t.write("a"); +t.cork(); +t.write("b"); +t.write("c", "latin1"); +process.nextTick(() => { t.uncork(); t.end("d"); }); + +const p = new PassThrough({ + objectMode: true, + writev(chunks, cb) { console.log("pt writev", JSON.stringify(chunks.map((c) => c.chunk))); cb(); }, +}); +p.on("data", (d) => console.log("pt data", JSON.stringify(d))); +p.cork(); +p.write({ x: 1 }); +p.write([2]); +p.uncork(); +p.write("solo"); +p.end(); diff --git a/tests/corpus/7461-transform-writev.cjs b/tests/corpus/7461-transform-writev.cjs new file mode 100644 index 000000000..dd45cf09d --- /dev/null +++ b/tests/corpus/7461-transform-writev.cjs @@ -0,0 +1,33 @@ +// Constructor-function Transform and PassThrough streams with _writev: +// the prototype's _writev takes the batches, and a writev option shadows +// it with the stream as this. +'use strict'; +const { Transform, PassThrough } = require('node:stream'); +function T(opts) { Transform.call(this, opts); } +Object.setPrototypeOf(T.prototype, Transform.prototype); +Object.setPrototypeOf(T, Transform); +T.prototype._transform = function (chunk, enc, cb) { cb(null, 'T(' + chunk + ')'); }; +T.prototype._writev = function (chunks, cb) { + this.push('V(' + chunks.map((c) => String(c.chunk)).join('+') + ')'); + cb(); +}; +const t = Object.create(T.prototype); +T.call(t, {}); +t.on('data', (d) => console.log('t data', String(d))); +t.on('end', () => console.log('t end')); +t.write('a'); +t.cork(); +t.write('b'); +t.write('c'); +process.nextTick(() => { t.uncork(); t.end(); }); +function P(opts) { PassThrough.call(this, opts); } +Object.setPrototypeOf(P.prototype, PassThrough.prototype); +Object.setPrototypeOf(P, PassThrough); +const p = Object.create(P.prototype); +P.call(p, { writev(chunks, cb) { console.log('p writev', chunks.length, this === p); cb(); } }); +p.on('data', (d) => console.log('p data', String(d))); +p.cork(); +p.write('x'); +p.write('y'); +p.uncork(); +p.write('z'); diff --git a/tests/corpus/7462-transform-writev.ts b/tests/corpus/7462-transform-writev.ts new file mode 100644 index 000000000..ca079b3c9 --- /dev/null +++ b/tests/corpus/7462-transform-writev.ts @@ -0,0 +1,109 @@ +// Typed writev on Transform and PassThrough: an options writev with +// decodeStrings: false and flush, a Transform subclass's _writev, a +// batch completing with an error, and object-mode entries whose +// encoding is declared as a string or as Node's string | null | +// undefined. +import { Transform, PassThrough, Writable } from "node:stream"; +type WritevChunks = Array<{ chunk: any; encoding: BufferEncoding }>; +type Cb = (error?: Error | null) => void; +type TransformCallback = (error?: Error | null, data?: any) => void; +const log: string[] = []; +const t = new Transform({ + decodeStrings: false, + transform(chunk: string, enc: BufferEncoding, cb: TransformCallback): void { + log.push(`transform ${chunk} ${enc}`); + cb(null, chunk); + }, + flush(cb: TransformCallback): void { + log.push("flush"); + cb(null, "!"); + }, + writev(chunks: WritevChunks, cb: Cb): void { + log.push(`writev ${chunks.map((c) => `${c.chunk}/${c.encoding}`).join(",")}`); + this.push(chunks.map((c) => String(c.chunk)).join("")); + cb(); + }, +}); +t.on("data", (d: Buffer) => log.push(`data ${d.toString()}`)); +t.on("end", () => log.push("end")); +t.on("finish", () => log.push("finish")); +t.on("close", () => { + console.log(log.join("\n")); + next(); +}); +t.write("one"); +t.cork(); +t.write("two", "latin1"); +t.write("three"); +t.uncork(); +t.end("four"); + +class Batcher extends Transform { + _transform(chunk: Buffer, _enc: BufferEncoding, cb: TransformCallback): void { + console.log("batcher transform", chunk.toString()); + cb(null, chunk); + } + _writev(chunks: Array<{ chunk: any; encoding: BufferEncoding }>, cb: Cb): void { + console.log("batcher writev", chunks.length); + this.push(Buffer.concat(chunks.map((c) => c.chunk as Buffer))); + cb(); + } +} + +function next(): void { + const b = new Batcher(); + b.on("data", (d: Buffer) => console.log("batcher data", d.toString())); + b.write("x"); + b.cork(); + b.write("y"); + b.write("z"); + process.nextTick(() => { + b.uncork(); + b.end(); + }); + b.on("finish", () => { + const p = new PassThrough({ + writev(chunks: WritevChunks, cb: Cb): void { + console.log("pt writev", chunks.length, String(chunks[0]!.chunk)); + cb(new Error("batch failed")); + }, + }); + p.on("error", (e: Error) => { + console.log("pt error", e.message); + objectEntries(); + }); + p.cork(); + p.write("1"); + p.write("2"); + p.uncork(); + }); +} + +function objectEntries(): void { + const declared = new Writable({ + objectMode: true, + write(_chunk: any, _enc: BufferEncoding, cb: Cb): void { cb(); }, + writev(chunks: WritevChunks, cb: Cb): void { + console.log("string encodings", JSON.stringify(chunks.map((c) => c.chunk))); + cb(); + }, + }); + declared.write({ n: 0 }); + declared.cork(); + declared.write({ n: 1 }); + declared.write({ n: 2 }, () => {}); + declared.uncork(); + const nodeTyped = new Writable({ + objectMode: true, + write(_chunk: any, _enc: BufferEncoding, cb: Cb): void { cb(); }, + writev(chunks: Array<{ chunk: any; encoding: string | null | undefined }>, cb: Cb): void { + console.log("node encodings", JSON.stringify(chunks.map((c) => [c.chunk, c.encoding === undefined ? "undefined" : c.encoding]))); + cb(); + }, + }); + nodeTyped.write({ n: 0 }); + nodeTyped.cork(); + nodeTyped.write({ n: 1 }); + nodeTyped.write({ n: 2 }, () => {}); + nodeTyped.uncork(); +} diff --git a/tests/corpus/7463-stream-option-values.ts b/tests/corpus/7463-stream-option-values.ts new file mode 100644 index 000000000..04f497b6b --- /dev/null +++ b/tests/corpus/7463-stream-option-values.ts @@ -0,0 +1,58 @@ +// Typed decodeStrings and defaultEncoding options whose values are known +// only at run time: decodeStrings === false keeps string writes, a known +// defaultEncoding is the write default as given and how push(string) +// decodes, null and utf8 are the default, an unknown name throws +// ERR_UNKNOWN_ENCODING before construction, and a subclass's super() +// options take them the same way. +import { Writable, Readable, Duplex } from "node:stream"; +type Cb = (error?: Error | null) => void; +const flags: boolean[] = JSON.parse("[false, true]"); +const encodings: Array = JSON.parse('["hex", "latin1", null, "UTF8"]'); +for (const decode of flags) { + const w = new Writable({ + decodeStrings: decode, + defaultEncoding: encodings[0] as BufferEncoding, + write(chunk: any, enc: string, cb: Cb): void { + console.log("write", decode, typeof chunk, enc, String(chunk)); + cb(); + }, + }); + w.write("6869"); +} +const r = new Readable({ defaultEncoding: encodings[1] as BufferEncoding, read(): void {} }); +r.push("é"); +r.push(null); +r.on("data", (d: Buffer) => console.log("readable", d.toString("hex"))); +const dflt = new Writable({ + defaultEncoding: encodings[2] as any, + write(chunk: Buffer, enc: string, cb: Cb): void { + console.log("null default", enc, chunk.toString()); + cb(); + }, +}); +dflt.write("ok"); +const upper = new Duplex({ + defaultEncoding: encodings[3] as BufferEncoding, + decodeStrings: flags[0], + read(): void {}, + write(chunk: any, enc: string, cb: Cb): void { + console.log("duplex", typeof chunk, enc); + cb(); + }, +}); +upper.write("x"); +try { + new Writable({ defaultEncoding: JSON.parse('"nope"') }); +} catch (e) { + console.log("bad", (e as Error & { code?: string }).code, (e as Error).message); +} +class Sink extends Writable { + constructor(decode: boolean, encoding: BufferEncoding) { + super({ decodeStrings: decode, defaultEncoding: encoding }); + } + _write(chunk: any, enc: string, cb: Cb): void { + console.log("sink", typeof chunk, enc); + cb(); + } +} +new Sink(flags[0]!, encodings[1] as BufferEncoding).write("y"); diff --git a/tests/corpus/7464-stream-option-values.mjs b/tests/corpus/7464-stream-option-values.mjs new file mode 100644 index 000000000..dfeb7a447 --- /dev/null +++ b/tests/corpus/7464-stream-option-values.mjs @@ -0,0 +1,28 @@ +// Option values known only at run time in JavaScript: decodeStrings and +// defaultEncoding read from an object, a write callback whose chunk is +// then a string, decodeStrings: 0 (only false turns decoding off), a +// defaultEncoding that is not a string, and a _construct assigned after +// construction, which Node's constructor never reads. +import { Writable, Readable } from "node:stream"; +const opts = { decode: false, enc: "base64" }; +const w = new Writable({ + decodeStrings: opts.decode, + defaultEncoding: opts.enc, + write(c, e, cb) { console.log("js", typeof c, e, String(c)); cb(); }, +}); +w.write("aGk="); +try { new Writable({ defaultEncoding: 5 }); } catch (e) { console.log("bad", e.code, e.message); } +const zero = new Writable({ decodeStrings: 0, write(c, e, cb) { console.log("zero", Buffer.isBuffer(c), e); cb(); } }); +zero.write("z"); +const late = new Writable({ write(chunk, enc, cb) { console.log("write", chunk.toString()); cb(); } }); +let ran = false; +late._construct = (cb) => { ran = true; cb(); }; +late.write("a"); +late.end(() => console.log("ended, construct ran:", ran)); +const r = new Readable({ read() {} }); +const order = []; +(order.push("receiver"), r)._construct = (order.push("value"), function (cb) { console.log("never"); cb(); }); +console.log(order.join(",")); +r.push("x"); +r.push(null); +r.on("data", (d) => console.log("data", d.toString())); diff --git a/tests/corpus/7465-transform-callback-values.ts b/tests/corpus/7465-transform-callback-values.ts new file mode 100644 index 000000000..e7003e894 --- /dev/null +++ b/tests/corpus/7465-transform-callback-values.ts @@ -0,0 +1,41 @@ +// A transform or flush callback passing a value that is not a Buffer or a +// string: the push reports Node's ERR_INVALID_ARG_TYPE through 'error' +// and destroys the stream, for option callbacks and _transform methods, +// while Buffer and string values take the typed path. +import { Transform } from "node:stream"; +type TransformCallback = (error?: Error | null, data?: any) => void; +const t = new Transform({ + transform(chunk: Buffer, _enc: BufferEncoding, cb: TransformCallback): void { + console.log("transform", chunk.toString()); + cb(null, 42); + }, +}); +t.on("error", (e: Error & { code?: string }) => console.log("error", e.name, e.code, e.message)); +t.on("data", (d: Buffer) => console.log("data", d.toString())); +t.on("close", () => console.log("close", t.destroyed)); +console.log("write", t.write("a")); +const f = new Transform({ + transform(chunk: Buffer, _enc: BufferEncoding, cb: TransformCallback): void { cb(null, chunk); }, + flush(cb: TransformCallback): void { cb(null, { tail: true }); }, +}); +f.on("error", (e: Error & { code?: string }) => console.log("flush error", e.code, e.message)); +f.on("data", (d: Buffer) => console.log("flush data", d.toString())); +f.on("close", () => console.log("flush close")); +f.end("z"); +class Bad extends Transform { + _transform(chunk: Buffer, _enc: BufferEncoding, cb: TransformCallback): void { + cb(null, chunk.length); + } +} +const bad = new Bad(); +bad.on("error", (e: Error & { code?: string }) => console.log("error", e.code, e.message)); +bad.on("close", () => console.log("close")); +bad.write("abc"); +class Good extends Transform { + _transform(chunk: Buffer, _enc: BufferEncoding, cb: TransformCallback): void { + cb(null, chunk.toString().toUpperCase()); + } +} +const good = new Good(); +good.on("data", (d: Buffer) => console.log("good", d.toString())); +good.end("xyz"); diff --git a/tests/corpus/7466-stream-construct-assign.mjs b/tests/corpus/7466-stream-construct-assign.mjs new file mode 100644 index 000000000..37684afcd --- /dev/null +++ b/tests/corpus/7466-stream-construct-assign.mjs @@ -0,0 +1,46 @@ +// Assigning _construct after construction: Node reads it on the +// construction tick, so an assignment before that tick replaces the +// scheduled hook, and one after it, or on a stream without a hook, has no +// effect. +import { Readable, Writable } from "node:stream"; + +const w = new Writable({ + construct(cb) { + console.log("original construct"); + cb(); + }, + write(chunk, _enc, cb) { + console.log("write", chunk.toString()); + cb(); + }, +}); +w._construct = function (cb) { + console.log("replacement construct"); + cb(); +}; +w.write("a"); +w.end(); +w.on("finish", () => { + w._construct = function (cb) { + console.log("late construct"); + cb(); + }; + console.log("finish"); + readable(); +}); + +// A stream without a construct hook: the assignment has no effect. +function readable() { + const r = new Readable({ + read() { + this.push("x"); + this.push(null); + }, + }); + r._construct = function (cb) { + console.log("never"); + cb(); + }; + r.on("data", (d) => console.log("data", d.toString())); + r.on("end", () => console.log("end")); +} diff --git a/tests/corpus/7500-pipeline-callback-undefined.ts b/tests/corpus/7500-pipeline-callback-undefined.ts new file mode 100644 index 000000000..e1503ff6b --- /dev/null +++ b/tests/corpus/7500-pipeline-callback-undefined.ts @@ -0,0 +1,120 @@ +// pipeline() with an async function or async generator destination: a +// typed callback whose error parameter admits undefined receives +// undefined when the pipeline succeeds, as in Node, and the error when it +// fails. +import { pipeline, Readable } from "node:stream"; + +function source(): Readable { + return Readable.from(["a", "b", "c"]); +} + +async function join(src: AsyncIterable): Promise { + let s = ""; + for await (const c of src) s += c; + return s; +} + +function optional(): Promise { + return new Promise((resolve) => { + pipeline(source(), join, (err?: Error | null, value?: string) => { + console.log("optional", err, err === undefined, value); + resolve(); + }); + }); +} + +function declared(): Promise { + return new Promise((resolve) => { + pipeline( + source(), + async function (src: AsyncIterable) { + let n = 0; + for await (const c of src) n += c.length; + return n; + }, + (err: Error | null | undefined, value?: number) => { + console.log("declared", err, typeof err, value); + resolve(); + }, + ); + }); +} + +function unknownParam(): Promise { + return new Promise((resolve) => { + pipeline(source(), join, (err: unknown, value: unknown) => { + console.log("unknown", err, value); + resolve(); + }); + }); +} + +function rest(): Promise { + return new Promise((resolve) => { + pipeline( + source(), + async function* (src: AsyncIterable) { + for await (const c of src) yield c.toUpperCase(); + }, + async function (src: AsyncIterable) { + for await (const c of src) void c; + }, + (...args: unknown[]) => { + console.log("rest", args.length, args); + resolve(); + }, + ); + }); +} + +function generatorDestination(): Promise { + return new Promise((resolve) => { + pipeline( + source(), + async function* (src: AsyncIterable) { + for await (const c of src) yield c; + }, + (err: Error | null | undefined, value: unknown) => { + console.log("generator", err, value); + resolve(); + }, + ); + }); +} + +function named(): Promise { + return new Promise((resolve) => { + function done(err: Error | null | undefined, value: string | undefined): void { + console.log("named", err, value); + resolve(); + } + pipeline(source(), join, done); + }); +} + +function failing(): Promise { + return new Promise((resolve) => { + pipeline( + source(), + async function (src: AsyncIterable) { + for await (const c of src) if (c === "b") throw new Error("boom at " + c); + return "never"; + }, + (err?: Error | null, value?: string) => { + console.log("failing", err?.message, value); + resolve(); + }, + ); + }); +} + +async function main(): Promise { + await optional(); + await declared(); + await unknownParam(); + await rest(); + await generatorDestination(); + await named(); + await failing(); +} +void main(); diff --git a/tests/corpus/7501-pipeline-callback-undefined.mjs b/tests/corpus/7501-pipeline-callback-undefined.mjs new file mode 100644 index 000000000..c51738e77 --- /dev/null +++ b/tests/corpus/7501-pipeline-callback-undefined.mjs @@ -0,0 +1,22 @@ +// pipeline() with an async function destination in JavaScript: the +// callback receives undefined and the destination's value on success. +import { pipeline, Readable } from "node:stream"; + +pipeline( + Readable.from(["x", "y"]), + async function (src) { + const out = []; + for await (const c of src) out.push(c); + return out.join("-"); + }, + (err, value) => { + console.log("callback", err, typeof err, value); + pipeline( + Readable.from(["z"]), + async function (src) { + for await (const c of src) void c; + }, + (...args) => console.log("no value", args), + ); + }, +); diff --git a/tests/corpus/7502-readline-cursor-undefined.ts b/tests/corpus/7502-readline-cursor-undefined.ts new file mode 100644 index 000000000..abac985cb --- /dev/null +++ b/tests/corpus/7502-readline-cursor-undefined.ts @@ -0,0 +1,28 @@ +// rl.cursor answers undefined on an interface that is not a terminal and +// the cursor position on a terminal interface. +import * as readline from "node:readline"; +import { PassThrough, Writable } from "node:stream"; + +const input = new PassThrough(); +const rl = readline.createInterface({ input }); +const c = rl.cursor; +console.log("cursor", c, typeof rl.cursor, rl.cursor === undefined); +console.log("arithmetic", rl.cursor + 1, Number.isNaN(rl.cursor * 2)); +console.log("fallback", rl.cursor ?? -1, `${rl.cursor}`); + +const interfaces = [rl]; +function first(): readline.Interface { + return interfaces[0]!; +} +console.log("call receiver", first().cursor); +rl.close(); + +const sink = new Writable({ + write(_chunk: Buffer, _enc, cb) { + cb(); + }, +}); +const terminal = readline.createInterface({ input: new PassThrough(), output: sink, terminal: true }); +terminal.write("abc"); +console.log("terminal", terminal.cursor, typeof terminal.cursor, terminal.cursor + 1); +terminal.close(); diff --git a/tests/corpus/7543-var-list-declarator-order.js b/tests/corpus/7543-var-list-declarator-order.js new file mode 100644 index 000000000..8fae06ff9 --- /dev/null +++ b/tests/corpus/7543-var-list-declarator-order.js @@ -0,0 +1,66 @@ +// A `var` read in a later declarator of the statement that stores it runs +// after the store; a read in the same or an earlier declarator sees undefined. +var base = { a: 1, b: "x" }, + more = { ...base, c: 2 }, + keys = Object.keys({ ...more }).join(","), + first = base.a + 1; +console.log(JSON.stringify(more), keys, first); + +function describeBase() { + return base.b + more.c; +} +var rules = { space: /\s+/, word: /\w+/ }, + merged = { ...rules, digit: /\d/ }, + described = describeBase(); +console.log(Object.keys(merged).join(","), merged.word.test("ab"), described); + +var table = { one: 1, two: 2 }, + lookup = function (key) { + return table[key]; + }, + arrow = (key) => ({ ...table })[key]; +console.log(lookup("one"), arrow("two")); + +var readLater = function () { + return later; + }, + early = readLater(), + later = { k: 1 }, + after = readLater().k; +console.log(early, after); + +var self = self ? 1 : 2; +console.log(self); + +for (var i = 0, j = i + 10; i < 2; i++) console.log(i, j); + +function nested() { + var inner = [1, 2, 3], + copy = [...inner, 4], + total = copy.reduce((sum, n) => sum + n, 0); + return { copy, total }; +} +console.log(JSON.stringify(nested())); + +function earlyInFunction() { + show("before"); + var point = { n: 1 }, + next = { m: point.n + 1 }; + show("after"); + function show(when) { + console.log(when, typeof point, point === undefined ? "none" : point.n, next === undefined ? "none" : next.m); + } + var check = () => console.log("check", items === undefined); + check(); + var items = [1]; + check(); +} +earlyInFunction(); + +function loopLists() { + for (var k = 0, sum = k; k < 3; k++) sum += k; + var doubled = sum * 2, + again = doubled + 1; + return [sum, doubled, again]; +} +console.log(JSON.stringify(loopLists())); diff --git a/tests/corpus/7545-string-regex-checked-arguments.js b/tests/corpus/7545-string-regex-checked-arguments.js new file mode 100644 index 000000000..a7ace9cdb --- /dev/null +++ b/tests/corpus/7545-string-regex-checked-arguments.js @@ -0,0 +1,89 @@ +// String search, match and replace with untyped pattern values: a RegExp +// is used as is, and any other value is converted as Node converts it. +const rules = JSON.parse('{"other":{}}'); +rules.other.space = /\s+/g; +rules.other.word = /o+/; +rules.other.text = "l"; +rules.other.dot = "."; +rules.other.none = undefined; +rules.other.nul = null; +rules.other.one = /(l)(o)/; +rules.other.many = /l+/g; +const t = " hello world"; + +console.log( + t.search(rules.other.word), + t.search(rules.other.text), + t.search(rules.other.none), + t.search(rules.other.nul), + t.search(rules.other.space), + "a.b".search(rules.other.dot), +); +console.log(rules.other.space.lastIndex); + +let conversions = 0; +rules.other.obj = { + toString() { + conversions++; + return "w"; + }, +}; +console.log(t.search(rules.other.obj), t.match(rules.other.obj)[0], conversions); + +const all = t.match(rules.other.many); +console.log(JSON.stringify(all), rules.other.many.lastIndex); +const pair = t.match(rules.other.one); +console.log(pair[0], pair[1], pair[2], pair.index, pair.length, pair.input === t); +const literal = t.match(rules.other.dot); +console.log(literal[0], literal.index, literal.length); +console.log(JSON.stringify(t.match(rules.other.none)), t.match(rules.other.zz)); +rules.other.zz = "zz"; +console.log(t.match(rules.other.zz)); +function first(text, re) { + const m = text.match(re); + return m ? m[0] : "none"; +} +console.log(first("abc", rules.other.dot), first("", rules.other.dot)); + +console.log(t.replace(rules.other.space, (m) => "[" + m.length + "]")); +console.log(t.replace(rules.other.word, (m, offset) => m.toUpperCase() + offset)); +console.log(t.replace(rules.other.text, (m, offset, subject) => "<" + m + offset + subject.length + ">")); +console.log(t.replaceAll(rules.other.text, (m) => m.toUpperCase())); +console.log(t.replaceAll(rules.other.space, () => "_")); +console.log(t.replace(rules.other.one, (m, p1, p2, offset) => p1 + "|" + p2 + "|" + offset)); +console.log(t.replace(rules.other.text, (m, offset, subject, extra) => typeof extra)); +console.log(t.replace(rules.other.text, (m) => m.length)); +try { + t.replaceAll(rules.other.word, (m) => m); +} catch (e) { + console.log(e.constructor.name, e.message); +} + +function tabs(text, width) { + let column = 0; + let out = ""; + for (const ch of text) { + if (ch === "\t") { + const pad = 4 - (column % 4); + out += " ".repeat(pad); + column += pad; + } else { + out += ch; + column++; + } + } + return out + width; +} +class Lexer { + constructor(table) { + this.rules = table; + } + indent(line, width) { + return line.replace(this.rules.leading, (w) => tabs(w, width)); + } + firstText(line) { + return line.search(this.rules.nonSpace); + } +} +const lexer = new Lexer({ leading: /^( {1,4}| {0,3}\t)/, nonSpace: /[^ ]/ }); +console.log(JSON.stringify(lexer.indent("\tabc", 2)), lexer.firstText(" x")); diff --git a/tests/corpus/7548-cjs-receiverless-builtin-calls/lib.cjs b/tests/corpus/7548-cjs-receiverless-builtin-calls/lib.cjs new file mode 100644 index 000000000..248e66270 --- /dev/null +++ b/tests/corpus/7548-cjs-receiverless-builtin-calls/lib.cjs @@ -0,0 +1,10 @@ +"use strict"; +Object.defineProperty(exports, "__esModule", { value: !0 }); +exports.label = exports.describe = void 0; +const node_util_1 = require("node:util"); +const node_path_1 = require("node:path"); +function describe(name, size) { + return (0, node_util_1.format)("%s holds %d (%s)", name, size, (0, node_path_1.basename)("/a/b/c.txt")); +} +exports.describe = describe; +exports.label = "lib"; diff --git a/tests/corpus/7548-cjs-receiverless-builtin-calls/main.cjs b/tests/corpus/7548-cjs-receiverless-builtin-calls/main.cjs new file mode 100644 index 000000000..6fd0b78a9 --- /dev/null +++ b/tests/corpus/7548-cjs-receiverless-builtin-calls/main.cjs @@ -0,0 +1,19 @@ +"use strict"; +// tsc and esbuild output: the __esModule stamp spelled { value: !0 }, and +// calls of builtin module functions without a receiver, (0, ns.member)(...). +const lib = require("./lib.cjs"); +const dc = require("node:diagnostics_channel"); +const util = require("node:util"); +const os = require("node:os"); +const path = require("node:path"); +console.log(lib.label, lib.describe("cache", 3)); +const metrics = (0, dc.channel)("cache:metrics"); +console.log(metrics.name, metrics.hasSubscribers); +const seen = []; +dc.subscribe("cache:metrics", (message) => seen.push(message.key)); +metrics.publish({ key: "a" }); +console.log(seen.join(","), metrics.hasSubscribers); +const tracing = (0, dc.tracingChannel)("cache"); +console.log(tracing.hasSubscribers); +console.log((0, util.inspect)({ a: [1, 2] }), (0, util.isDeepStrictEqual)([1], [1])); +console.log((0, path.join)("a", "b"), typeof (0, os.platform)()); diff --git a/tests/corpus/7549-cjs-minified-esmodule-stamp/main.cjs b/tests/corpus/7549-cjs-minified-esmodule-stamp/main.cjs new file mode 100644 index 000000000..c619a847a --- /dev/null +++ b/tests/corpus/7549-cjs-minified-esmodule-stamp/main.cjs @@ -0,0 +1,7 @@ +"use strict"; +// Minifiers spell `true` as `!0`: Object.defineProperty(exports, +// "__esModule", { value: !0 }) is the same interop stamp as { value: true }. +const stamped = require("./stamped.cjs"); +const moduleStamped = require("./module-stamped.cjs"); +console.log(stamped.answer, stamped.twice(4), moduleStamped.label); +console.log(typeof stamped.twice, moduleStamped.label.length); diff --git a/tests/corpus/7549-cjs-minified-esmodule-stamp/module-stamped.cjs b/tests/corpus/7549-cjs-minified-esmodule-stamp/module-stamped.cjs new file mode 100644 index 000000000..96a2aae7b --- /dev/null +++ b/tests/corpus/7549-cjs-minified-esmodule-stamp/module-stamped.cjs @@ -0,0 +1,3 @@ +"use strict"; +Object.defineProperty(module.exports, "__esModule", { value: !0 }); +module.exports.label = "module"; diff --git a/tests/corpus/7549-cjs-minified-esmodule-stamp/stamped.cjs b/tests/corpus/7549-cjs-minified-esmodule-stamp/stamped.cjs new file mode 100644 index 000000000..cd3b9c400 --- /dev/null +++ b/tests/corpus/7549-cjs-minified-esmodule-stamp/stamped.cjs @@ -0,0 +1,5 @@ +"use strict"; +Object.defineProperty(exports, "__esModule", { value: !0 }); +exports.answer = void 0; +exports.answer = 42; +exports.twice = (n) => n * 2; diff --git a/tests/corpus/7553-var-for-in-of-zero-iterations.js b/tests/corpus/7553-var-for-in-of-zero-iterations.js new file mode 100644 index 000000000..1a439f715 --- /dev/null +++ b/tests/corpus/7553-var-for-in-of-zero-iterations.js @@ -0,0 +1,18 @@ +// A for-in or for-of loop stores its var binding only when it iterates: +// after a loop that ran zero times, closures read undefined. +function f(o) { + for (var key in o) {} + const read = () => typeof key; + const same = () => key === undefined; + return [read(), same()]; +} +console.log(JSON.stringify(f({})), JSON.stringify(f({ a: 1 }))); +function g(xs) { + for (var x of xs) {} + const show = () => [typeof x, String(x)]; + return show(); +} +console.log(JSON.stringify(g([])), JSON.stringify(g([4]))); +for (var top in {}) {} +const readTop = () => typeof top; +console.log(readTop()); diff --git a/tests/corpus/7580-int32-arrays-across-calls.ts b/tests/corpus/7580-int32-arrays-across-calls.ts new file mode 100644 index 000000000..df180b264 --- /dev/null +++ b/tests/corpus/7580-int32-arrays-across-calls.ts @@ -0,0 +1,166 @@ +// Number arrays of proven int32s that reach other functions: passed to +// helpers called directly, along chains of helpers and recursion, and held +// in module globals that several functions read. A helper that also +// receives an array of other numbers or one that is not a fresh array, +// stores a fraction through its parameter, or escapes as a value keeps +// every array it can reach in doubles, as does a global an async function +// reads. Each case uses its own arrays and helpers. + +const data: number[] = []; +let seed = 7; +for (let i = 0; i < 2000; i++) { + seed = (Math.imul(seed, 48271) + 11) & 0x7fffffff; + data.push(seed % 1000); +} + +function fnv(values: number[]): number { + let h = 2166136261; + for (let i = 0; i < values.length; i++) { + h ^= values[i]! & 0xff; + h = Math.imul(h, 16777619) >>> 0; + } + return h; +} + +function histogram(values: number[], buckets: number): number[] { + const counts: number[] = new Array(buckets).fill(0); + for (let i = 0; i < values.length; i++) { + const b = values[i]! % buckets; + counts[b] = counts[b]! + 1; + } + return counts; +} + +// A helper that writes through its parameter and calls another helper. +function scale(values: number[], factor: number): number { + for (let i = 0; i < values.length; i++) values[i] = (values[i]! * factor) & 0xffff; + return sum(values); +} + +function sum(values: number[]): number { + let s = 0; + for (let i = 0; i < values.length; i++) s = (s + values[i]!) | 0; + return s; +} + +// Recursion passes the array on, and the elements index another array. +function chase(next: number[], at: number, steps: number): number { + if (steps === 0) return at; + return chase(next, next[at]!, steps - 1); +} + +function pushAll(target: number[], count: number): void { + for (let i = 0; i < count; i++) target.push((i * 3) & 0xffff); +} + +// `n` is a power of two: the masked elements are indexes of `perm`. +function locals(n: number): string { + const perm: number[] = []; + for (let i = 0; i < n; i++) perm.push((i * 37 + 11) & (n - 1)); + const copy: number[] = []; + pushAll(copy, n); + const s1 = scale(copy, 5); + return `locals ${chase(perm, 0, 1000)} ${s1} ${copy[n - 1]} ${copy.length} ${sum(perm)}`; +} + +// Read only through the element reader (`a[i] ?? 0`) and states. +function readSparse(values: number[], at: number): string { + return `${values[at] ?? -1} ${at in values} ${values.length}`; +} + +// The same helper receives an array of other numbers too: the arrays it +// can reach keep doubles. +function mean(values: number[]): number { + let s = 0; + for (let i = 0; i < values.length; i++) s += values[i]!; + return s / values.length; +} + +// A store through the parameter that is not an int32. +function halve(values: number[]): void { + for (let i = 0; i < values.length; i++) values[i] = values[i]! / 2; +} + +// A function used as a value keeps its parameters. +function first(values: number[]): number { + return values[0]!; +} + +let reassigned: number[] = [1, 2, 3]; +function regrow(n: number): number { + reassigned = []; + for (let i = 0; i < n; i++) reassigned.push(i & 15); + let s = 0; + for (let i = 0; i < reassigned.length; i++) s += reassigned[i]!; + return s; +} + +// Not a fresh array: the result of a call. +function total(values: number[]): number { + let s = 0; + for (let i = 0; i < values.length; i++) s += values[i]!; + return s; +} + +const awaited: number[] = [4, 5, 6]; +async function later(): Promise { + return awaited[0]! + awaited.length; +} + +// A helper whose record result its callers only project: the backend may +// call its result-slot entry instead, so its parameter keeps doubles. +interface Span { + lo: number; + hi: number; +} +function spanAt(values: number[], i: number): Span { + return { lo: values[i]!, hi: values[i + 1]! }; +} +const spans: number[] = [5, 9, 2, 7, 11]; +function spanTotal(): number { + let total = 0; + for (let i = 0; i < 4; i++) { + const span = spanAt(spans, i); + total += span.hi * 10 - span.lo; + } + return total; +} + +const ints: number[] = [3, 4, 5]; +const fractions: number[] = [0.5, 1.5, 2.5]; +const halved: number[] = [8, 6, 4]; +halve(halved); +const pick: (values: number[]) => number = first; +const viaValue: number[] = [9, 8, 7]; + +// An index that calls a function reassigning the global: the read uses the +// array the global held before the index ran, as in Node. +let swapped: number[] = [1, 2, 3]; +function swapIn(): number { + swapped = []; + for (let i = 0; i < 4; i++) swapped.push(i + 10); + return 1; +} +function fillSwapped(n: number): number { + swapped = []; + for (let i = 0; i < n; i++) swapped.push(i & 15); + let s = 0; + for (let i = 0; i < swapped.length; i++) s += swapped[i]!; + return s; +} +function readSwapped(): number | undefined { + return swapped[swapIn()]; +} + +console.log(`fnv ${fnv(data)} ${fnv(data)}`); +const h = histogram(data, 10); +console.log(`hist ${h.length} ${h[0]} ${h[9]} ${total(h)}`); +console.log(locals(128)); +console.log(readSparse(data, 5), readSparse(data, 5000)); +console.log(`mean ${mean(ints)} ${mean(fractions)}`); +console.log(`halved ${halved[0]} ${halved[2]} ${halved.length}`); +console.log(`value ${pick(viaValue)} ${viaValue.length}`); +console.log(`regrow ${regrow(40)} ${reassigned.length} ${reassigned[39]} ${regrow(5)}`); +console.log(`span ${spanTotal()}`); +console.log(`swap ${fillSwapped(6)} ${readSwapped()} ${swapped[1]}`); +void later().then((v) => console.log(`later ${v}`)); diff --git a/tests/corpus/7620-js-class-view-descriptors.js b/tests/corpus/7620-js-class-view-descriptors.js new file mode 100644 index 000000000..a0dc42813 --- /dev/null +++ b/tests/corpus/7620-js-class-view-descriptors.js @@ -0,0 +1,91 @@ +// An instance of a JS class with public fields keeps its own properties in +// a property bag, and untyped code that calls a method on it sees a view +// built from that bag: data properties with their attributes, accessors, +// index-like keys (which come first, in ascending order), symbols, and +// properties added or removed after construction. Every refresh of the view +// must copy the bag exactly. +'use strict'; + +class Box { + label = 'box'; + count = 0; + constructor(n) { + this.count = n; + } + describe() { + return `${this.label}:${this.count}`; + } + bump() { + this.count++; + return this; + } +} + +function call(target, method, ...args) { + return target[method](...args); +} + +function put(target, key, value) { + target[key] = value; +} + +function drop(target, key) { + delete target[key]; +} + +function show(title, target) { + const lines = []; + for (const key of Reflect.ownKeys(target)) { + const d = Object.getOwnPropertyDescriptor(target, key); + const kind = 'value' in d ? `value=${String(d.value)} writable=${d.writable}` : `get=${typeof d.get} set=${typeof d.set}`; + lines.push(` ${String(key)}: ${kind} enumerable=${d.enumerable} configurable=${d.configurable}`); + } + console.log(`${title}\n${lines.join('\n')}`); +} + +const box = new Box(2); +Object.defineProperty(box, 'hidden', { value: 42, enumerable: false, writable: false, configurable: true }); +Object.defineProperty(box, 'computed', { + get() { + return this.count * 10; + }, + set(v) { + this.count = v / 10; + }, + enumerable: true, + configurable: true, +}); +Object.defineProperty(box, 'getterOnly', { get: () => 'g', enumerable: false, configurable: false }); +put(box, 2, 'two'); +put(box, 1, 'one'); +put(box, 'extra', 'x'); +const tag = Symbol('tag'); +put(box, tag, 'symbol'); + +console.log(call(box, 'describe')); +show('after construction', box); +console.log('keys', Object.keys(box).join(',')); + +call(box, 'bump'); +put(box, 'computed', 70); +drop(box, 'extra'); +put(box, 'later', true); +console.log(call(box, 'describe')); +show('after changes', box); +console.log('computed', box.computed, 'hidden', box.hidden, 'getterOnly', box.getterOnly); + +// Many views in a row see the same, current properties. +let total = 0; +for (let i = 0; i < 5; i++) { + call(box, 'bump'); + total += Object.keys(box).length; +} +console.log('total keys', total, call(box, 'describe')); + +const other = new Box(7); +console.log(call(other, 'describe'), Object.keys(other).join(',')); +try { + call(box, 'missing'); +} catch (error) { + console.log(error instanceof TypeError, error.message); +} diff --git a/tests/corpus/7621-js-class-field-methods.js b/tests/corpus/7621-js-class-field-methods.js new file mode 100644 index 000000000..cd0a5be99 --- /dev/null +++ b/tests/corpus/7621-js-class-field-methods.js @@ -0,0 +1,96 @@ +// Untyped code calling methods on an instance of a JS class whose own +// properties live in a property bag: methods declared as arrow-function +// fields (own properties), prototype methods, prototype methods replaced or +// shadowed after construction, accessors on the prototype, and calls of +// members that are not functions. Each call must see what the instance +// holds at that moment and bind `this` to the instance. +'use strict'; + +class Context { + status = 200; + headers = {}; + label = 'ctx'; + header = (name, value) => { + this.headers[name] = value; + return this; + }; + json = (body) => `${this.status} ${JSON.stringify(body)} ${JSON.stringify(this.headers)}`; + constructor(label) { + this.label = label; + } + describe() { + return `${this.label}/${this.status}`; + } + get upper() { + return this.label.toUpperCase(); + } + shout() { + return this.upper + '!'; + } +} + +// Constant method names: the calls a framework's untyped code makes. +const calls = { + describe: (t) => t.describe(), + header: (t, name, value) => t.header(name, value), + json: (t, body) => t.json(body), + shout: (t) => t.shout(), + extra: (t, n) => t.extra(n), + status: (t) => t.status(), + missing: (t) => t.missing(), + upper: (t) => t.upper(), +}; + +function call(target, method, ...args) { + return calls[method](target, ...args); +} + +function put(target, key, value) { + target[key] = value; +} + +const c = new Context('first'); +console.log(call(c, 'describe')); +console.log(call(call(c, 'header', 'x-a', '1'), 'json', { ok: true })); +put(c, 'status', 404); +console.log(call(c, 'json', [1, 2]), call(c, 'describe'), call(c, 'shout')); + +// A field method replaced through untyped code, and an own property that +// shadows a prototype method. +put(c, 'json', (body) => `replaced ${body}`); +put(c, 'describe', function () { + return `own describe of ${this.label}`; +}); +console.log(call(c, 'json', 'x'), call(c, 'describe')); + +// A prototype method replaced after construction reaches every instance +// that does not shadow it. +const d = new Context('second'); +Context.prototype.describe = function () { + return `patched ${this.label}`; +}; +console.log(call(d, 'describe'), call(c, 'describe')); + +// A prototype method added after construction. +Context.prototype.extra = function (n) { + return `${this.label} extra ${n}`; +}; +console.log(call(d, 'extra', 3)); + +// Members that are not functions. +for (const name of ['status', 'missing', 'upper']) { + try { + call(d, name); + } catch (error) { + console.log(error instanceof TypeError, error.message); + } +} + +// Many calls in a row stay consistent with the instance's current state. +let out = ''; +for (let i = 0; i < 4; i++) { + call(d, 'header', `h${i}`, String(i)); + put(d, 'status', 200 + i); + out += call(d, 'json', i) + '\n'; +} +process.stdout.write(out); diff --git a/tests/corpus/7640-child-spawn-before-fast-exit.ts b/tests/corpus/7640-child-spawn-before-fast-exit.ts new file mode 100644 index 000000000..9c232d96c --- /dev/null +++ b/tests/corpus/7640-child-spawn-before-fast-exit.ts @@ -0,0 +1,20 @@ +// A child spawned from a 'close' listener that exits before the listener +// returns still emits 'spawn' before 'exit'. +import { spawn } from "node:child_process"; + +function busyWait(ms: number): void { + const end = Date.now() + ms; + while (Date.now() < end) {} +} + +const events: string[] = []; +spawn("sh", ["-c", "exit 0"], { stdio: "ignore" }).on("close", () => { + const child = spawn("sh", ["-c", "exit 4"], { stdio: "ignore" }); + child.on("spawn", () => events.push("spawn")); + child.on("exit", (code) => events.push(`exit ${code}`)); + child.on("close", (code) => { + events.push(`close ${code}`); + console.log(events.join(", ")); + }); + busyWait(300); +}); diff --git a/tests/corpus/7642-element-assignment-order.ts b/tests/corpus/7642-element-assignment-order.ts new file mode 100644 index 000000000..e020d05fc --- /dev/null +++ b/tests/corpus/7642-element-assignment-order.ts @@ -0,0 +1,45 @@ +// An element assignment evaluates its receiver and index before its value, +// also when the value is an optional element read or a call that assigns +// the bindings the receiver and index read. +const a = [10, 20, 30, 40]; +const b = [1, 2, 3, 4]; +let n = 0; +a[n++] = b[n]!; +console.log(a.join(), n); +a[n] = b[n++]; +console.log(a.join(), n); + +const optional: (number | undefined)[] = [undefined, 5, undefined]; +let k = 0; +optional[k++] = optional[k]; +console.log(optional, k); + +// A call in the value assigns the receiver and the index through closures. +let target: (number | undefined)[] = [0, 0, 0]; +const first = target; +let at = 0; +function swap(): number | undefined { + target = [9, 9, 9]; + at = 2; + return b[3]; +} +target[at] = swap(); +console.log(first, target, at); + +// The index and receiver are not evaluated twice. +let reads = 0; +function index(): number { + reads++; + return 1; +} +const c: (number | undefined)[] = [0, 0]; +c[index()] = optional[1]; +console.log(c, reads); + +// The index assigns the receiver's binding: the store uses the receiver +// evaluated before it. +let receiver: (number | undefined)[] = [0, 0]; +const original = receiver; +const replacement: (number | undefined)[] = [7, 7]; +receiver[((receiver = replacement), 1)] = optional[1]; +console.log(original, replacement, receiver === replacement); diff --git a/tests/corpus/7733-property-bag-late-prototype.js b/tests/corpus/7733-property-bag-late-prototype.js new file mode 100644 index 000000000..f88212b62 --- /dev/null +++ b/tests/corpus/7733-property-bag-late-prototype.js @@ -0,0 +1,52 @@ +// A class whose property bag is first created without prototype data, while a local +// class with an untyped base rewrites the shared bag dispatch: the store gains the +// prototype later, so inherited methods stay readable (hono's SmartRouter with zod). +function $constructor(name, initializer, params) { + const Parent = params?.Parent ?? Object; + class Definition extends Parent {} + Object.defineProperty(Definition, "name", { value: name }); + function _(def) { + const inst = params?.Parent ? new Definition() : this; + inst.def = def; + initializer(inst, def); + return inst; + } + Object.defineProperty(_, "name", { value: name }); + return _; +} +class Smart { + name = "S"; + #inner = []; + constructor(init) { this.#inner = init.routers; } + add(x) { this.#inner.push(x); } + match(m) { + const r = this.#inner[0]; + this.match = r.match.bind(r); + this.name = `S + ${r.name}`; + return "first " + m; + } +} +class Inner { name = "Inner"; match(m) { return "inner " + m; } } +class App { + router; + constructor(options = {}) { + const { strict, ...rest } = options; + Object.assign(this, rest); + this.router = options.router ?? new Smart({ routers: [new Inner()] }); + } + dispatch(m) { return this.router.match(m); } +} +const app = new App(); +console.log(Object.keys(app.router).join(",")); +const $Err = $constructor("$Err", (inst, def) => { inst.issues = def.issues; }, { Parent: Error }); +const e = new $Err({ issues: [1] }); +console.log(Object.keys(e).join(","), e.issues.length); +const run = [(a) => a.dispatch("a"), (a) => a.dispatch("b")]; +for (const f of run) console.log(f(app), app.router.name); +const holder = JSON.parse("{}"); +holder.r = new Smart({ routers: [new Inner()] }); +console.log(typeof holder.r.match, "match" in holder.r, Object.keys(holder.r).join(",")); +console.log(holder.r.match("c"), holder.r.match("d"), holder.r.name); +const scratch = JSON.parse("{\"name\":\"x\",\"other\":1}"); +delete scratch.name; +console.log(Object.keys(scratch).join(",")); diff --git a/tests/corpus/7734-caught-instanceof-class-binding.js b/tests/corpus/7734-caught-instanceof-class-binding.js new file mode 100644 index 000000000..4332daab0 --- /dev/null +++ b/tests/corpus/7734-caught-instanceof-class-binding.js @@ -0,0 +1,42 @@ +// A catch binding tested against a class held by a binding (var/const class expression), +// as hono's SmartRouter tests UnsupportedPathError. +var UnsupportedPathError = class extends Error {}; +class Strict { + add(p) { if (p.includes("*")) throw new UnsupportedPathError(); return p; } +} +class Loose { add(p) { return "loose " + p; } } +function pick(p) { + const routers = [new Strict(), new Loose()]; + for (const r of routers) { + try { + return r.add(p); + } catch (e) { + if (e instanceof UnsupportedPathError) continue; + throw e; + } + } +} +console.log(pick("/a"), pick("/b/*")); +try { throw new Error("x"); } catch (e) { console.log(e instanceof UnsupportedPathError, e instanceof Error); } +const u = new UnsupportedPathError("m"); +console.log(u instanceof UnsupportedPathError, u.message, u.name, u instanceof Error); + +// A binding that holds a subclass of the class it was declared with: the +// test reads the class the binding holds at run time. +class BaseError extends Error {} +class DerivedError extends BaseError {} +class LeafError extends DerivedError {} +var Current = BaseError; +function caughtIs(value) { + try { + throw value; + } catch (e) { + return e instanceof Current; + } +} +const samples = () => [new BaseError("b"), new DerivedError("d"), new LeafError("l"), new Error("e"), "text"]; +console.log("base", samples().map(caughtIs).join(",")); +Current = DerivedError; +console.log("derived", samples().map(caughtIs).join(",")); +Current = LeafError; +console.log("leaf", samples().map(caughtIs).join(",")); diff --git a/tests/corpus/7738-private-logical-assignment-untyped.js b/tests/corpus/7738-private-logical-assignment-untyped.js new file mode 100644 index 000000000..b0ceaea86 --- /dev/null +++ b/tests/corpus/7738-private-logical-assignment-untyped.js @@ -0,0 +1,56 @@ +// Logical assignments to a private field of another instance whose type +// JavaScript leaves open, the way a router trie numbers its parameter +// nodes on first use (`next.#varIndex ??= context.varIndex++`). They read +// and write the brand-checked field, and evaluate the right side only +// when they store it. +class Node { + #varIndex; + #label = ""; + #count = 0; + #children = Object.create(null); + + child(key, context) { + let next = this.#children[key]; + if (!next) next = this.#children[key] = new Node(); + next.#varIndex ??= context.varIndex++; + next.#label ||= `node:${key}`; + next.#count &&= next.#count + 1; + next.#count ||= 1; + return next; + } + + describe() { + const kind = typeof this.#varIndex === "number" ? "param" : "static"; + return `${kind} ${this.#varIndex} ${this.#label} ${this.#count}`; + } + + static reset(node, other) { + other.#label &&= ""; + other.#varIndex ??= -1; + return [node.#label, other.#label, other.#varIndex]; + } + + static peek(value) { + return value.#varIndex ??= 99; + } +} + +const context = { varIndex: 0 }; +const root = new Node(); +const a = root.child("a", context); +const b = root.child("b", context); +const again = root.child("a", context); +console.log(a === again, context.varIndex); +console.log(root.describe()); +console.log(a.describe()); +console.log(b.describe()); +console.log(JSON.stringify(Node.reset(root, a))); +console.log(a.describe()); + +for (const value of [{}, null]) { + try { + Node.peek(value); + } catch (error) { + console.log(error instanceof TypeError, error.message); + } +} diff --git a/tests/corpus/7740-handle-method-values.js b/tests/corpus/7740-handle-method-values.js new file mode 100644 index 000000000..dffcba306 --- /dev/null +++ b/tests/corpus/7740-handle-method-values.js @@ -0,0 +1,126 @@ +// Methods of native Web objects read as values in untyped code: hono's +// HonoRequest calls `raw[key]()` with a computed body-method name, and code +// probes `typeof x.json` or `"get" in headers`. Each method is one shared +// function that runs on its `this`; another receiver fails Node's brand check. +const report = (label, e) => console.log(label, e.name, e.message, e.code); +// An untyped holder, as values reach a JavaScript library. +const untyped = (value) => { + const holder = JSON.parse("{}"); + holder.value = value; + return holder.value; +}; + +async function bodies() { + const holder = JSON.parse("{}"); + holder.req = new Request("http://example.test/books", { method: "POST", body: '{"title":"Dune"}' }); + const cache = {}; + const cachedBody = (key) => { + const raw = holder.req; + if (cache[key]) return cache[key]; + return (cache[key] = raw[key]()); + }; + console.log(await cachedBody("json")); + for (const key of ["text", "arrayBuffer", "bytes"]) { + const res = new Response("abc"); + const value = await res[key](); + console.log(key, typeof value, value instanceof ArrayBuffer, value.length ?? value.byteLength); + } +} + +function shape() { + const req = JSON.parse("{}"); + req.value = new Request("http://example.test/"); + const r = req.value; + console.log(typeof r.json, typeof r.text, typeof r.blobby, typeof r.url); + console.log("json" in r, "url" in r, "toString" in r, "missing" in r); + const other = JSON.parse("{}"); + other.res = new Response("x"); + console.log(r.json === r.json, r.json === other.res.json, r.json === r.text); + console.log(r.json.name, r.json.length, r.arrayBuffer.name); + const box = { h: untyped(new Headers({ a: "1" })) }; + console.log(box.h.append.length, box.h.forEach.length, box.h.get.name, "has" in box.h, "nope" in box.h); + const name = "get"; + console.log(box.h[name]("a"), box.h["has"]("b")); + box.h["set"]("b", "2"); + const entries = []; + box.h["forEach"]((v, k) => entries.push(k + "=" + v)); + console.log(entries.join(",")); +} + +async function receivers() { + const box = { + h: untyped(new Headers({ a: "1" })), + req: untyped(new Request("http://example.test/", { method: "POST", body: "x" })), + }; + const get = box.h.get; + console.log(get.call(box.h, "a"), get.bind(box.h)("a"), get.apply(box.h, ["a"])); + try { + get("a"); + } catch (e) { + report("unbound get", e); + } + try { + get.call(box.req, "a"); + } catch (e) { + report("foreign get", e); + } + const text = box.req.text; + const pending = text(); + console.log("text returns", pending instanceof Promise); + await pending.then( + () => console.log("unexpected"), + (e) => report("unbound text", e), + ); + await text.call(box.h).catch((e) => report("foreign text", e)); + console.log(await text.call(box.req)); + const signal = { s: untyped(new AbortController().signal) }; + const throwIfAborted = signal.s.throwIfAborted; + try { + throwIfAborted(); + } catch (e) { + report("unbound throwIfAborted", e); + } + try { + signal.s.addEventListener.call(box.h, "abort", () => {}); + } catch (e) { + report("foreign addEventListener", e); + } + // The signal's EventTarget methods are EventTarget's own: one function + // shared with every target, which runs on any EventTarget receiver. + const target = untyped(new EventTarget()); + const add = signal.s.addEventListener; + console.log(add === target.addEventListener, signal.s.dispatchEvent === target.dispatchEvent); + console.log(add.name, add.length, "addEventListener" in signal.s, "throwIfAborted" in target); + const heard = (event) => console.log("heard", event.type, event.target === target); + add.call(target, "ping", heard); + console.log(signal.s.dispatchEvent.call(target, new Event("ping"))); + signal.s.removeEventListener.call(target, "ping", heard); + target.dispatchEvent(new Event("ping")); + try { + throwIfAborted.call(target); + } catch (e) { + report("target throwIfAborted", e); + } +} + +async function streams() { + const box = { body: untyped(new Response("stream").body) }; + const key = "getReader"; + const reader = box.body[key](); + const read = reader.read; + const first = await read.call(reader); + console.log(first.done, new TextDecoder().decode(first.value)); + await read().catch((e) => report("unbound read", e)); + try { + box.body.getReader.call(reader); + } catch (e) { + report("foreign getReader", e); + } +} + +(async () => { + await bodies(); + shape(); + await receivers(); + await streams(); +})(); diff --git a/tests/corpus/7741-abort-signal-listener-value.js b/tests/corpus/7741-abort-signal-listener-value.js new file mode 100644 index 000000000..1a8a201d7 --- /dev/null +++ b/tests/corpus/7741-abort-signal-listener-value.js @@ -0,0 +1,10 @@ +// An AbortSignal's EventTarget methods read as values, in a program that +// never adds a listener or makes an EventTarget of its own: the shared +// functions are released at exit like every other value. +const holder = JSON.parse("{}"); +holder.controller = new AbortController(); +const signal = holder.controller.signal; +const add = signal.addEventListener; +console.log(typeof add, add.name, add.length, signal.dispatchEvent.name); +console.log("removeEventListener" in signal, signal.removeEventListener.length); +console.log(signal.aborted); diff --git a/tests/corpus/7742-response-clone-tee.ts b/tests/corpus/7742-response-clone-tee.ts new file mode 100644 index 000000000..56c5e74e6 --- /dev/null +++ b/tests/corpus/7742-response-clone-tee.ts @@ -0,0 +1,212 @@ +// Response.clone and Request.clone on checked-dynamic handles: the body +// tees, so the original and the clone each read every chunk; a clone of a +// used or locked body throws; canceling both branches cancels the source +// with both reasons; the clone's headers are a copy. +const enc = new TextEncoder(); +const dec = new TextDecoder(); + +function describe(e: unknown): string { + const err = e as { name?: string; message?: string }; + return `${String(err.name)}: ${String(err.message)}`; +} + +async function readAll(stream: any): Promise { + const reader = stream.getReader(); + let out = ""; + for (;;) { + const { done, value } = await reader.read(); + if (done) return out; + out += dec.decode(value); + } +} + +async function main(): Promise { + const r: any = new Response("hello world", { status: 201, headers: { "x-a": "1" } }); + const c: any = r.clone(); + console.log(c.status, c.headers.get("x-a"), c.bodyUsed, r.bodyUsed, c.body === r.body); + console.log(await c.text(), "|", await r.text()); + console.log("used", c.bodyUsed, r.bodyUsed); + try { + r.clone(); + } catch (e) { + console.log("clone used:", describe(e), e instanceof TypeError); + } + + const locked: any = new Response("abc"); + locked.body.getReader(); + try { + locked.clone(); + } catch (e) { + console.log("clone locked:", describe(e)); + } + + const empty: any = new Response(null, { status: 204, statusText: "Nothing" }); + const emptyClone: any = empty.clone(); + console.log("null body", emptyClone.body, emptyClone.status, emptyClone.statusText, JSON.stringify(await emptyClone.text())); + + // Headers are copied: changing the clone's leaves the original's alone. + const h: any = new Response("x", { headers: { "x-b": "2" } }); + const hc: any = h.clone(); + hc.headers.set("x-b", "3"); + console.log("headers", h.headers.get("x-b"), hc.headers.get("x-b"), h.headers === hc.headers); + + // A pulled source: both branches see each chunk (the same object). + let n = 0; + const source: any = new ReadableStream({ + start(): void { + console.log("start"); + }, + pull(controller: any): void { + n++; + console.log("pull", n); + if (n <= 3) controller.enqueue(enc.encode(`chunk${n};`)); + else controller.close(); + }, + }); + const streamed: any = new Response(source); + const streamedClone: any = streamed.clone(); + console.log("cloned"); + const r1 = streamed.body.getReader(); + const r2 = streamedClone.body.getReader(); + const a = await r1.read(); + const b = await r2.read(); + console.log("same chunk", a.value === b.value, dec.decode(a.value)); + console.log("rest 1", await readAllFrom(r1)); + console.log("rest 2", await readAllFrom(r2)); + + // One branch read to the end first; the other buffers. + const big: any = new Response(new ReadableStream({ + start(controller: any): void { + controller.enqueue(enc.encode("a")); + controller.enqueue(enc.encode("b")); + controller.close(); + }, + })); + const bigClone: any = big.clone(); + console.log("clone first", await bigClone.text()); + console.log("then original", await readAll(big.body)); + + // Canceling both branches cancels the source with both reasons. + const cancelSource: any = new ReadableStream({ + pull(controller: any): void { + controller.enqueue(enc.encode("z")); + }, + cancel(reason: unknown): void { + console.log("source cancel", JSON.stringify(reason)); + }, + }); + const cr: any = new Response(cancelSource); + const crc: any = cr.clone(); + const p1 = cr.body.cancel("first"); + console.log("one canceled"); + const p2 = crc.body.cancel("second"); + await p1; + await p2; + console.log("both canceled", cr.bodyUsed, crc.bodyUsed); + + // An erroring source errors both branches. + const failing: any = new ReadableStream({ + pull(controller: any): void { + controller.error(new Error("source failed")); + }, + }); + const fr: any = new Response(failing); + const frc: any = fr.clone(); + for (const branch of [fr, frc]) { + try { + await branch.text(); + } catch (e) { + console.log("branch error:", describe(e)); + } + } + + // Request.clone: the body tees and the signal follows the original's. + const q: any = new Request("http://example.test/path", { method: "POST", body: "data" }); + const qc: any = q.clone(); + console.log("request", qc.method, qc.url, await qc.text(), await q.text(), qc.signal === q.signal); + try { + q.clone(); + } catch (e) { + console.log("request clone used:", describe(e)); + } + const ac = new AbortController(); + const q2: any = new Request("http://example.test/", { signal: ac.signal }); + const q2c: any = q2.clone(); + console.log("before abort", q2c.signal.aborted); + ac.abort("why"); + console.log("after abort", q2c.signal.aborted, q2c.signal.reason); + const q3: any = new Request("http://example.test/", { signal: AbortSignal.abort("early") }); + console.log("aborted clone", q3.clone().signal.aborted, q3.clone().signal.reason); + + // clone is a method value like the body readers. + console.log(typeof r.clone, "clone" in r, r.clone === c.clone); +} + +async function readAllFrom(reader: any): Promise { + let out = ""; + for (;;) { + const { done, value } = await reader.read(); + if (done) return out; + out += dec.decode(value); + } +} + + +// Tee branches error when their source errors: after both buffered a +// chunk and with no source read pending, reads reject once the error's +// reaction ran (a read issued at once still takes the buffered chunk, as in +// Node); a clone of an errored body errors both branches. +async function readFirst(label: string, r: any): Promise { + try { + const chunk = await r.body.getReader().read(); + console.log(label, "read", JSON.stringify(chunk.value ? [...chunk.value] : null), chunk.done); + } catch (error) { + console.log(label, "rejects", (error as Error).message); + } +} + +function erroring(): { res: any; error: (reason: Error) => void } { + let ctrl: ReadableStreamDefaultController | undefined; + const source = new ReadableStream({ + start(c) { + ctrl = c; + c.enqueue(new Uint8Array([1, 2])); + }, + }); + return { res: new Response(source), error: (reason) => ctrl!.error(reason) }; +} + +const tick = (): Promise => new Promise((resolve) => setTimeout(resolve, 10)); + +async function sourceErrors(): Promise { + // Error, then read after the reaction: both branches reject. + { + const { res, error } = erroring(); + const clone = res.clone(); + await tick(); + error(new Error("late")); + await tick(); + await readFirst("later original", res); + await readFirst("later clone", clone); + } + // Error, then read at once: the first read takes the buffered chunk. + { + const { res, error } = erroring(); + const clone = res.clone(); + await tick(); + error(new Error("soon")); + await readFirst("at once original", res); + await readFirst("at once clone", clone); + } + // A body that errored before the clone. + { + const { res, error } = erroring(); + error(new Error("before")); + const clone = res.clone(); + await readFirst("errored original", res); + await readFirst("errored clone", clone); + } +} + +await main(); +await sourceErrors(); diff --git a/tests/corpus/7743-transform-stream-identity.ts b/tests/corpus/7743-transform-stream-identity.ts new file mode 100644 index 000000000..3664c8ff1 --- /dev/null +++ b/tests/corpus/7743-transform-stream-identity.ts @@ -0,0 +1,239 @@ +// The identity TransformStream: writes wait for reads (the readable side's +// high-water mark is 0), the writer's desiredSize/ready track the writable +// side's queue, close/abort/cancel propagate across the pair (in either +// order when they race), and the readable side feeds a Response body (the +// StreamingApi pattern of Hono's stream helpers). +const enc = new TextEncoder(); +const dec = new TextDecoder(); + +function describe(e: unknown): string { + if (e instanceof Error) { + const code = (e as NodeJS.ErrnoException).code; + return `${e.name}: ${e.message}${code === undefined ? "" : ` [${code}]`}`; + } + return `value ${String(e)}`; +} + +async function basics(): Promise { + const ts = new TransformStream(); + const writer = ts.writable.getWriter(); + const reader = ts.readable.getReader(); + console.log("desired", writer.desiredSize, ts.writable.locked, ts.readable.locked); + const written = writer.write("a").then(() => console.log("write a done")); + console.log("desired after write", writer.desiredSize); + void writer.ready.then(() => console.log("ready")); + const first = await reader.read(); + console.log("read", first.done, first.value); + await written; + console.log("desired", writer.desiredSize); + const closing = writer.close().then(() => console.log("closed")); + const last = await reader.read(); + console.log("read", last.done, last.value); + await closing; + await writer.closed; + console.log("writer.closed resolved"); + try { + await writer.write("b"); + } catch (e) { + console.log("write after close:", describe(e)); + } + try { + ts.writable.getWriter(); + } catch (e) { + console.log("getWriter locked:", describe(e)); + } + writer.releaseLock(); + try { + await writer.write("c"); + } catch (e) { + console.log("released write:", describe(e)); + } + try { + await writer.closed; + } catch (e) { + console.log("released closed:", describe(e)); + } +} + +async function abortWriter(): Promise { + const ts = new TransformStream(); + const writer = ts.writable.getWriter(); + const reader = ts.readable.getReader(); + const pending = writer.write("z").catch((e: unknown) => console.log("pending write:", describe(e))); + await writer.abort("boom").then(() => console.log("abort resolved")); + await pending; + try { + await reader.read(); + } catch (e) { + console.log("read after abort:", describe(e)); + } + try { + await writer.closed; + } catch (e) { + console.log("closed after abort:", describe(e)); + } + try { + await writer.ready; + } catch (e) { + console.log("ready after abort:", describe(e)); + } + console.log("desired after abort", writer.desiredSize); +} + +async function cancelReadable(): Promise { + const ts = new TransformStream(); + const writer = ts.writable.getWriter(); + const pending = writer.write("q").catch((e: unknown) => console.log("write rejected by cancel:", describe(e))); + await ts.readable.cancel("nope").then(() => console.log("cancel resolved")); + await pending; + try { + await writer.closed; + } catch (e) { + console.log("writer.closed after cancel:", describe(e)); + } +} + +async function intoResponse(): Promise { + const ts = new TransformStream(); + const writer = ts.writable.getWriter(); + void (async (): Promise => { + for (const s of ["x", "y", "z"]) { + await writer.write(enc.encode(s)); + console.log("wrote", s); + } + await writer.close(); + console.log("writer closed"); + })(); + console.log("text", await new Response(ts.readable).text()); +} + +/** Hono's StreamingApi: a writer on the writable side and a pull-based + * ReadableStream over a reader of the readable side. */ +interface StreamingApi { + readonly responseReadable: ReadableStream; + write(input: string): Promise; + close(): Promise; + state(): string; +} + +function streamingApi(writable: WritableStream, readable: ReadableStream): StreamingApi { + const writer = writable.getWriter(); + const reader = readable.getReader(); + let closed = false; + let aborted = false; + const responseReadable = new ReadableStream({ + async pull(controller): Promise { + const { done, value } = await reader.read(); + if (done) controller.close(); + else controller.enqueue(value); + }, + cancel: (): void => { + if (!closed) aborted = true; + }, + }); + return { + responseReadable, + async write(input: string): Promise { + try { + await writer.write(enc.encode(input)); + } catch { + // a closed or aborted stream drops the write + } + }, + async close(): Promise { + closed = true; + try { + await writer.close(); + } catch { + // already closed + } + }, + state: (): string => `closed=${String(closed)} aborted=${String(aborted)}`, + }; +} + +async function streamedResponse(): Promise { + const { readable, writable } = new TransformStream(); + const api = streamingApi(writable, readable); + void (async (): Promise => { + try { + for (let i = 0; i < 3; i++) { + await api.write(`data: ${i}\n\n`); + } + } finally { + await api.close(); + } + })(); + const response = new Response(api.responseReadable, { headers: { "content-type": "text/event-stream" } }); + const reader = response.body!.getReader(); + let chunks = 0; + let text = ""; + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + chunks++; + text += dec.decode(value); + } + console.log("streamed", chunks, JSON.stringify(text), api.state()); +} + +async function writableMethods(): Promise { + const a = new TransformStream(); + await a.writable.abort("r").then(() => console.log("writable.abort ok")); + const b = new TransformStream(); + const writer = b.writable.getWriter(); + try { + await b.writable.close(); + } catch (e) { + console.log("close locked:", describe(e)); + } + try { + await b.writable.abort(); + } catch (e) { + console.log("abort locked:", describe(e)); + } + writer.releaseLock(); + const reader = b.readable.getReader(); + const closing = b.writable.close().then(() => console.log("writable.close ok")); + const end = await reader.read(); + console.log("readable done", end.done); + await closing; +} + +// writable.abort() and readable.cancel() share the transform's finish +// promise: whichever runs second answers the first one's result. +function settled(label: string, p: Promise): Promise { + return p.then( + () => console.log(label, "fulfilled"), + (e: unknown) => console.log(label, "rejected", describe(e)), + ); +} + +async function abortCancelRaces(): Promise { + const a = new TransformStream(); + const aWriter = a.writable.getWriter(); + await new Promise((r) => setTimeout(r, 0)); + const aborted = settled("abort then cancel: abort", aWriter.abort("abort first")); + const cancelled = settled("abort then cancel: cancel", a.readable.cancel("cancel second")); + await aborted; + await cancelled; + const b = new TransformStream(); + const bWriter = b.writable.getWriter(); + await new Promise((r) => setTimeout(r, 0)); + const cancelledFirst = settled("cancel then abort: cancel", b.readable.cancel("cancel first")); + const abortedSecond = settled("cancel then abort: abort", bWriter.abort("abort second")); + await cancelledFirst; + await abortedSecond; +} + +async function main(): Promise { + await basics(); + await abortWriter(); + await cancelReadable(); + await intoResponse(); + await streamedResponse(); + await writableMethods(); + await abortCancelRaces(); +} + +await main(); diff --git a/tests/corpus/7746-key-order-iterable-iterator.ts b/tests/corpus/7746-key-order-iterable-iterator.ts index 13f27ee83..d9d4ea15a 100644 --- a/tests/corpus/7746-key-order-iterable-iterator.ts +++ b/tests/corpus/7746-key-order-iterable-iterator.ts @@ -56,3 +56,54 @@ const ordered = [...newest(listed)]; console.log(JSON.stringify(ordered), ordered[0] === listed[1]); const step = newest(listed).next(); console.log(step.done, JSON.stringify(step.value)); + +// Deleting the current entry before leaving a loop over an iterator-typed +// method: the iterator parks after that entry and continues with the next. +class Store { + readonly rows = new Map(); + readonly tags = new Set(); + values(): IterableIterator { + return this.rows.values(); + } + keys(): IterableIterator { + return this.rows.keys(); + } + tagValues(): IterableIterator { + return this.tags.values(); + } + anyPairs(): IterableIterator { + return this.rows.entries(); + } + unknownPairs(): IterableIterator { + return this.rows.entries(); + } + tagAny(): IterableIterator { + return this.tags.entries(); + } +} +const store = new Store(); +for (const [k, v] of [["a", 1], ["b", 2], ["c", 3]] as const) store.rows.set(k, v); +for (const t of ["x", "y", "z"]) store.tags.add(t); +console.log(JSON.stringify([...store.anyPairs()]), JSON.stringify([...store.unknownPairs()])); +console.log(JSON.stringify([...store.tagAny()])); +const values = store.values(); +for (const v of values) { + if (v === 2) { + store.rows.delete("b"); + break; + } +} +console.log([...values].join(), [...store.rows.keys()].join()); +for (const k of store.keys()) { + store.rows.delete(k); + break; +} +console.log([...store.rows.keys()].join()); +const tags = store.tagValues(); +for (const t of tags) { + if (t === "y") { + store.tags.delete("y"); + break; + } +} +console.log([...tags].join(), [...store.tags].join()); diff --git a/tests/corpus/7747-text-codec-untyped-slot.js b/tests/corpus/7747-text-codec-untyped-slot.js new file mode 100644 index 000000000..cdf8345da --- /dev/null +++ b/tests/corpus/7747-text-codec-untyped-slot.js @@ -0,0 +1,69 @@ +// TextEncoder and TextDecoder instances held in untyped storage (hono's +// StreamingApi keeps `this.encoder = new TextEncoder()` in a JavaScript +// class field) and called from dynamic code, with their identity, streaming +// state, properties, method values and argument errors. +class StreamingApi { + /** @type {any} */ encoder; + /** @type {any} */ decoder; + /** @type {any} */ latin; + /** @type {any} */ strict; + constructor() { + this.encoder = new TextEncoder(); + this.decoder = new TextDecoder(); + this.latin = new TextDecoder("latin1"); + this.strict = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }); + } + write(input) { + if (typeof input === "string") input = this.encoder.encode(input); + return input; + } +} + +const show = (error) => console.log(error.name, error.code); +const api = new StreamingApi(); +console.log(api.write("héllo 😀")); +console.log(api.encoder.encoding, typeof api.encoder, api.encoder === api.encoder); +console.log(String(api.encoder), JSON.stringify(api.encoder), JSON.stringify({ d: api.decoder })); +console.log(api.encoder.encode(), api.encoder.encode(42), api.encoder.encode(undefined)); +const dest = new Uint8Array(4); +console.log(api.encoder.encodeInto("héllo", dest), dest); +try { + api.encoder.encodeInto(1, dest); +} catch (error) { + show(error); +} + +console.log(api.decoder.encoding, api.decoder.fatal, api.decoder.ignoreBOM); +console.log(api.latin.encoding, api.strict.fatal, api.strict.ignoreBOM); +const euro = api.encoder.encode("€uro"); +const head = api.decoder.decode(euro.subarray(0, 2), { stream: true }); +console.log(JSON.stringify(head), api.decoder.decode(euro.subarray(2))); +console.log(api.latin.decode(new Uint8Array([0xe9, 0x41]))); +console.log(JSON.stringify(api.strict.decode(new Uint8Array([0xef, 0xbb, 0xbf, 0x41])))); +try { + api.strict.decode(new Uint8Array([0xff])); +} catch (error) { + show(error); +} +try { + api.decoder.decode("text"); +} catch (error) { + show(error); +} +console.log(JSON.stringify(api.decoder.decode()), JSON.stringify(api.decoder.decode(undefined, null))); + +const encode = api.encoder.encode; +try { + encode("x"); +} catch (error) { + console.log(error.name, error.code, error.message); +} +console.log(typeof api.encoder.encode, api.encoder.encode.length, api.encoder.encodeInto.length); +console.log(api.decoder.decode.length, encode === api.encoder.encode); +console.log(api.encoder.encode.call(api.encoder, "ab")); +console.log("encode" in api.encoder, "decode" in api.decoder, api.encoder.missing); + +console.log(api.encoder instanceof TextEncoder, api.decoder instanceof TextDecoder); +console.log(api.encoder instanceof TextDecoder, JSON.parse("{}") instanceof TextEncoder); +const codecs = [api.encoder, api.decoder]; +console.log(codecs.indexOf(api.decoder), codecs.includes(api.encoder)); diff --git a/tests/corpus/7748-npm-any-bytes-receiver/main.ts b/tests/corpus/7748-npm-any-bytes-receiver/main.ts new file mode 100644 index 000000000..58301765a --- /dev/null +++ b/tests/corpus/7748-npm-any-bytes-receiver/main.ts @@ -0,0 +1,33 @@ +// Typed arrays the checker types `any` in package JavaScript: hono's etag +// digest (hono/dist/middleware/etag/digest.js) merges multi-chunk bodies +// through `nextChunk.set(...)`, where `nextChunk`'s initializer depends on +// `chunk`, which is assigned `nextChunk`. Method calls and property reads on +// such values run on the native bytes, as in Node. +import { generateDigest } from "etag-digest"; +import { grow } from "chunk-buffers"; + +const enc = new TextEncoder(); +const sha1 = (body: Uint8Array): Promise => crypto.subtle.digest({ name: "SHA-1" }, body); + +function chunked(parts: string[]): ReadableStream { + let i = 0; + return new ReadableStream({ + pull(controller): void { + if (i < parts.length) controller.enqueue(enc.encode(parts[i++]!)); + else controller.close(); + }, + }); +} + +async function main(): Promise { + console.log("single", await generateDigest(chunked([JSON.stringify({ books: 400 })]), sha1)); + console.log("chunked", await generateDigest(chunked(["alpha\n", "beta\n", "gamma\n", "delta\n"]), sha1)); + const large = ["x".repeat(200000), "y".repeat(100000), "z".repeat(5)]; + console.log("large", await generateDigest(chunked(large), sha1)); + console.log("exact", await generateDigest(chunked(["a".repeat(262144), "b"]), sha1)); + console.log("empty", await generateDigest(chunked([]), sha1)); + console.log("null", await generateDigest(null, sha1)); + console.log(grow([new Uint8Array([1, 2, 3]), new Uint8Array([4]), new Uint8Array([5])])); +} + +await main(); diff --git a/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/index.d.ts b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/index.d.ts new file mode 100644 index 000000000..26fe364a0 --- /dev/null +++ b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/index.d.ts @@ -0,0 +1 @@ +export declare function grow(parts: Uint8Array[]): Uint8Array | undefined; diff --git a/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/index.js b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/index.js new file mode 100644 index 000000000..a0a1fea79 --- /dev/null +++ b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/index.js @@ -0,0 +1,18 @@ +// `next` is referenced by its own initializer's inference (through +// `chunk = next`), so the checker types it `any` while it is a Uint8Array. +export function grow(parts) { + let chunk; + for (const value of parts) { + if (!chunk) chunk = value.slice(0); + else { + const next = new Uint8Array(new ArrayBuffer(Math.max(8, chunk.byteLength * 2))); + next.set(chunk.subarray(0, 2)); + next.set([9], 2); + next[3] = next.byteLength; + console.log(next.length, next[0], next.subarray(1, 3), next.at(-1), next.byteOffset); + console.log(next.slice(2, 4), next.buffer.byteLength, next.BYTES_PER_ELEMENT); + chunk = next; + } + } + return chunk; +} diff --git a/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/package.json b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/package.json new file mode 100644 index 000000000..74e354d34 --- /dev/null +++ b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/chunk-buffers/package.json @@ -0,0 +1 @@ +{ "name": "chunk-buffers", "version": "1.0.0", "type": "module", "main": "index.js", "types": "index.d.ts" } diff --git a/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/index.d.ts b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/index.d.ts new file mode 100644 index 000000000..79d9dc396 --- /dev/null +++ b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/index.d.ts @@ -0,0 +1 @@ +export declare const generateDigest: (stream: ReadableStream | null, generator: (body: Uint8Array) => ArrayBuffer | Promise) => Promise; diff --git a/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/index.js b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/index.js new file mode 100644 index 000000000..bf438a5ea --- /dev/null +++ b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/index.js @@ -0,0 +1,60 @@ +// hono/dist/middleware/etag/digest.js (hono 4.13.13) +const mergeBuffers = (buffer1, buffer2) => { + if (!buffer1) return buffer2; + const merged = new Uint8Array(new ArrayBuffer(buffer1.byteLength + buffer2.byteLength)); + merged.set(new Uint8Array(buffer1), 0); + merged.set(buffer2, buffer1.byteLength); + return merged; +}; +const CHUNK_SIZE = 262144; +const generateDigest = async (stream, generator) => { + if (!stream) return null; + let result = void 0; + let chunk; + let chunkLength = 0; + const digest = async (body) => { + result = await generator(mergeBuffers(result, body)); + }; + const reader = stream.getReader(); + for (;;) { + const { value, done } = await reader.read(); + if (done) break; + let offset = 0; + while (offset < value.byteLength) { + const remaining = value.byteLength - offset; + if (chunkLength === 0 && remaining >= CHUNK_SIZE) { + await digest(value.subarray(offset, offset + CHUNK_SIZE)); + offset += CHUNK_SIZE; + continue; + } + const requiredLength = chunkLength + remaining; + if (requiredLength < CHUNK_SIZE) { + if (!chunk) chunk = value.slice(offset); + else { + if (chunk.byteLength < requiredLength) { + const nextChunk = new Uint8Array(new ArrayBuffer(Math.min(CHUNK_SIZE, Math.max(requiredLength, chunk.byteLength * 2)))); + nextChunk.set(chunk.subarray(0, chunkLength)); + chunk = nextChunk; + } + chunk.set(value.subarray(offset), chunkLength); + } + chunkLength = requiredLength; + break; + } + const length = CHUNK_SIZE - chunkLength; + if (chunk?.byteLength !== CHUNK_SIZE) { + const nextChunk = new Uint8Array(/* @__PURE__ */ new ArrayBuffer(CHUNK_SIZE)); + if (chunk) nextChunk.set(chunk.subarray(0, chunkLength)); + chunk = nextChunk; + } + chunk.set(value.subarray(offset, offset + length), chunkLength); + await digest(chunk); + chunkLength = 0; + offset += length; + } + } + if (chunk && chunkLength > 0) await digest(chunk.subarray(0, chunkLength)); + if (!result) return null; + return Array.prototype.map.call(new Uint8Array(result), (x) => x.toString(16).padStart(2, "0")).join(""); +}; +export { generateDigest }; diff --git a/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/package.json b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/package.json new file mode 100644 index 000000000..f15495eed --- /dev/null +++ b/tests/corpus/7748-npm-any-bytes-receiver/node_modules/etag-digest/package.json @@ -0,0 +1 @@ +{ "name": "etag-digest", "version": "1.0.0", "type": "module", "main": "index.js", "types": "index.d.ts" } diff --git a/tests/corpus/7749-text-codec-unknown-roundtrip.ts b/tests/corpus/7749-text-codec-unknown-roundtrip.ts new file mode 100644 index 000000000..b7c18b3ef --- /dev/null +++ b/tests/corpus/7749-text-codec-unknown-roundtrip.ts @@ -0,0 +1,36 @@ +// TypeScript codecs crossing 'unknown' slots: boxed by reference and +// checked back, keeping identity and a decoder's streaming state. +const holder: { codec: unknown; decoder: unknown } = { + codec: new TextEncoder(), + decoder: new TextDecoder("utf-16le"), +}; + +const encoder = holder.codec as TextEncoder; +console.log(encoder.encode("typed"), encoder === (holder.codec as TextEncoder)); +const decoder = holder.decoder as TextDecoder; +const bytes = new Uint8Array([0x68, 0x00, 0x69]); +console.log(JSON.stringify(decoder.decode(bytes, { stream: true }))); +console.log(JSON.stringify(decoder.decode(new Uint8Array([0x00]))), decoder.encoding); + +function store(value: unknown): unknown[] { + return [value, value]; +} +const pair = store(encoder); +console.log(pair[0] === pair[1], pair[0] === holder.codec); + +const maybe: unknown = Math.random() > 2 ? 1 : new TextDecoder(); +if (typeof maybe === "object" && maybe !== null) console.log((maybe as TextDecoder).encoding); + +// Codecs inside arrays and records cross as their handles too. +const codecs: TextEncoder[] = [encoder, new TextEncoder()]; +const asUnknown: unknown = codecs; +const back = asUnknown as TextEncoder[]; +console.log(back.length, back[0] === encoder, back[1]!.encode("x")); +const streaming = new TextDecoder(); +const nested: { name: string; decoder: TextDecoder } = { name: "nested", decoder: streaming }; +const boxed: unknown = nested; +const unboxed = boxed as { name: string; decoder: TextDecoder }; +console.log(unboxed.name, unboxed.decoder === streaming); +console.log(JSON.stringify(unboxed.decoder.decode(new Uint8Array([0xe2, 0x82]), { stream: true }))); +console.log(JSON.stringify(streaming.decode(new Uint8Array([0xac])))); +console.log(JSON.stringify(boxed)); diff --git a/tests/corpus/7750-key-order-javascript-spread-sources/main.ts b/tests/corpus/7750-key-order-javascript-spread-sources/main.ts index 727f8e367..073ff6c5e 100644 --- a/tests/corpus/7750-key-order-javascript-spread-sources/main.ts +++ b/tests/corpus/7750-key-order-javascript-spread-sources/main.ts @@ -103,3 +103,20 @@ showView("replaced view", repo.view(repo.require("y"))); const again = repo.view(checked); showView("again", again); console.log(native.version, checked.version, again.availability.onLoan); + +// An integer-named key the spread source lacks comes first, as Node +// creates integer keys first whatever the literal's order. +interface Ranked { + year: number; + title: string; + tags: readonly string[]; + id: string; + "1": string; +} +function rank(source: Draft, id: string): Ranked { + return { ...source, id, "1": "gold" }; +} +for (const ranked of [rank(draft, "r1"), rank({ title: "Native", year: 1990, tags: [] }, "r2")]) { + console.log("ranked", JSON.stringify(ranked)); + console.log(" keys", Object.keys(ranked).join()); +} diff --git a/tests/corpus/7790-js-class-integrity-layouts.js b/tests/corpus/7790-js-class-integrity-layouts.js new file mode 100644 index 000000000..f889c02f8 --- /dev/null +++ b/tests/corpus/7790-js-class-integrity-layouts.js @@ -0,0 +1,139 @@ +"use strict"; +// Object.freeze, Object.seal and Object.preventExtensions next to JavaScript +// classes whose fields all exist from construction. The integrity state of +// a class instance lives beside its fields, so classes keep fixed layouts +// whether or not a program freezes values; frozen instances still refuse +// writes and additions as in Node. + +class Point { + constructor(x, y) { + this.x = x; + this.y = y; + } + norm() { + return Math.abs(this.x) + Math.abs(this.y); + } +} + +class Point3 extends Point { + constructor(x, y, z) { + super(x, y); + this.z = z; + } + norm() { + return super.norm() + Math.abs(this.z); + } +} + +class Status { + constructor() { + this.value = "valid"; + } + dirty() { + if (this.value === "valid") this.value = "dirty"; + } +} + +/** @param {any} target @param {string} key @param {unknown} value */ +function put(target, key, value) { + target[key] = value; +} + +/** @param {any} value */ +function describe(value) { + const keys = Object.keys(value); + const parts = []; + for (const key in value) parts.push(`${key}=${String(value[key])}`); + return `${keys.join(",")} | ${parts.join(" ")} | ${JSON.stringify(value)}`; +} + +/** @param {() => void} fn */ +function attempt(label, fn) { + try { + fn(); + console.log(label, "ok"); + } catch (e) { + console.log(label, e instanceof TypeError ? "TypeError" : "other", e.message); + } +} + +// The zod shape: freeze whatever value a result carries. Here it is always +// plain data, never a class instance. +/** @param {{ status: string, value: any }} data */ +function freezeResult(data) { + if (data.status === "valid") data.value = Object.freeze(data.value); + return data; +} + +const results = [ + freezeResult({ status: "valid", value: { a: 1, list: [1, 2] } }), + freezeResult({ status: "valid", value: [3, 4] }), + freezeResult({ status: "aborted", value: { b: 2 } }), + freezeResult({ status: "valid", value: "text" }), +]; +for (const result of results) console.log(result.status, Object.isFrozen(result.value), JSON.stringify(result.value)); +/** @param {() => void} fn */ +function throws(fn) { + try { + fn(); + return false; + } catch (e) { + return e instanceof TypeError; + } +} +console.log(throws(() => put(results[0].value, "a", 9)), throws(() => put(results[2].value, "b", 3))); +console.log(JSON.stringify(results[2].value)); + +// Class instances that only ever travel through untyped code. +const status = new Status(); +status.dirty(); +const p = new Point(1, -2); +const q = new Point3(3, 4, -5); +put(p, "tag", "p"); +put(q, "w", 6); +console.log(describe(status), status.value); +console.log(describe(p), p.norm()); +console.log(describe(q), q.norm()); +console.log(Object.isFrozen(p), Object.isExtensible(q), Object.isSealed(status)); + +// Statically typed freeze, seal and preventExtensions of instances. +const frozen = new Point(5, 6); +Object.freeze(frozen); +attempt("frozen field write", () => { + /** @type {Point} */ (frozen).x = 50; +}); +attempt("frozen field write untyped", () => put(frozen, "y", 60)); +attempt("frozen addition", () => put(frozen, "extra", 1)); +console.log(describe(frozen), Object.isFrozen(frozen), Object.isSealed(frozen), Object.isExtensible(frozen)); +console.log(JSON.stringify(Object.getOwnPropertyDescriptor(frozen, "x"))); + +const sealed = new Point3(1, 2, 3); +put(sealed, "before", true); +Object.seal(sealed); +attempt("sealed field write", () => { + sealed.z = 30; +}); +attempt("sealed added write", () => put(sealed, "before", false)); +attempt("sealed addition", () => put(sealed, "after", 1)); +console.log(describe(sealed), Object.isFrozen(sealed), Object.isSealed(sealed), Object.isExtensible(sealed)); +console.log(JSON.stringify(Object.getOwnPropertyDescriptor(sealed, "z"))); + +const closed = new Status(); +Object.preventExtensions(closed); +closed.dirty(); +attempt("closed addition", () => put(closed, "more", 1)); +console.log(describe(closed), Object.isFrozen(closed), Object.isExtensible(closed)); + +// A subclass instance frozen through its base type. +/** @type {Point} */ +const asBase = new Point3(7, 8, 9); +Object.freeze(asBase); +attempt("frozen subclass field", () => put(asBase, "z", 90)); +console.log(describe(asBase), asBase.norm(), Object.isFrozen(asBase)); + +// Fresh instances of the same classes stay writable. +const fresh = new Point3(0, 0, 0); +fresh.x = 1; +put(fresh, "y", 2); +put(fresh, "added", 3); +console.log(describe(fresh), fresh.norm(), Object.isFrozen(fresh)); diff --git a/tests/corpus/7791-js-class-accessor-layouts.js b/tests/corpus/7791-js-class-accessor-layouts.js new file mode 100644 index 000000000..6c87989cb --- /dev/null +++ b/tests/corpus/7791-js-class-accessor-layouts.js @@ -0,0 +1,85 @@ +"use strict"; +// Getters and setters on JavaScript classes. An accessor whose name no +// field uses lives on the prototype beside the fields; an accessor that +// shares a field's name decides what an assignment does. Both behave as in +// Node, for direct use and for instances passed to untyped code. + +class LazyPath { + constructor(parent, value, path, key) { + this._cachedPath = []; + this.parent = parent; + this.data = value; + this._path = path; + this._key = key; + } + get path() { + if (!this._cachedPath.length) { + if (Array.isArray(this._key)) this._cachedPath.push(...this._path, ...this._key); + else this._cachedPath.push(...this._path, this._key); + } + return this._cachedPath; + } +} + +class Temperature { + constructor(celsius) { + this.celsius = celsius; + } + get fahrenheit() { + return this.celsius * 1.8 + 32; + } + set fahrenheit(value) { + this.celsius = (value - 32) / 1.8; + } +} + +class Labeled extends Temperature { + constructor(celsius, label) { + super(celsius); + this.label = label; + } + get description() { + return `${this.label}: ${this.celsius.toFixed(1)}C`; + } +} + +/** @param {any} target @param {string} key @param {unknown} value */ +function put(target, key, value) { + target[key] = value; +} + +/** @param {any} value */ +function describe(value) { + const parts = []; + for (const key in value) parts.push(key); + return `${Object.keys(value).join(",")} | ${parts.join(",")} | ${JSON.stringify(value)}`; +} + +/** @param {any} value @param {string} key */ +function probe(value, key) { + return `${key}: in=${key in value} own=${Object.prototype.hasOwnProperty.call(value, key)} value=${JSON.stringify(value[key])}`; +} + +const root = { common: true }; +const paths = []; +for (let i = 0; i < 3; i++) paths.push(new LazyPath(root, { n: i }, ["items"], i)); +paths.push(new LazyPath(root, null, ["a"], ["b", "c"])); +for (const item of paths) console.log(item.path.join("."), item.data && item.data.n, item.parent === root); +console.log(describe(paths[0])); +console.log(probe(paths[1], "path"), probe(paths[1], "_key")); +put(paths[2], "extra", 1); +console.log(describe(paths[2]), paths[2].path.length); + +const t = new Temperature(100); +console.log(t.fahrenheit, describe(t)); +t.fahrenheit = 32; +console.log(t.celsius, t.fahrenheit); +put(t, "fahrenheit", 212); +console.log(t.celsius, probe(t, "fahrenheit"), probe(t, "celsius")); +put(t, "note", "added"); +console.log(describe(t)); + +const l = new Labeled(20, "room"); +console.log(l.description, l.fahrenheit, describe(l)); +l.fahrenheit = 50; +console.log(l.description, probe(l, "description")); diff --git a/tests/corpus/7792-untyped-property-reads-mixed-receivers.js b/tests/corpus/7792-untyped-property-reads-mixed-receivers.js new file mode 100644 index 000000000..6a2785541 --- /dev/null +++ b/tests/corpus/7792-untyped-property-reads-mixed-receivers.js @@ -0,0 +1,111 @@ +"use strict"; +// One untyped property read site meets class instances (fixed and tracked +// layouts, getters, own properties added later), plain objects, arrays, +// strings, functions, class constructors with statics, null prototypes and +// optional chains. Each receiver kind answers as in Node. + +class Status { + constructor() { + this.value = "valid"; + } +} + +class Lazy { + constructor(key) { + this.key = key; + } + get value() { + return `lazy:${this.key}`; + } + get length() { + return 7; + } +} + +class Tracked { + constructor(list) { + list.push(this); + this.value = list.length; + } + static value = "static"; +} + +function plainFunction() {} +plainFunction.value = "fn"; + +/** @param {any} target */ +function readValue(target) { + return target.value; +} +/** @param {any} target */ +function readLength(target) { + return target.length; +} +/** @param {any} target */ +function readOptional(target) { + return target?.value; +} +/** @param {any} target @param {unknown} value */ +function writeValue(target, value) { + target.value = value; +} + +const list = []; +const added = new Status(); +writeValue(added, "dirty"); +/** @type {any} */ +const extra = new Lazy("x"); +extra.other = 1; +const nullProto = Object.create(null); +nullProto.value = "bare"; +const inherited = Object.create({ value: "from prototype" }); +const receivers = [ + new Status(), + added, + new Lazy("k"), + extra, + new Tracked(list), + { value: 1, length: 2 }, + { other: true }, + [1, 2, 3], + "text", + 42, + true, + plainFunction, + Tracked, + nullProto, + inherited, +]; +for (const receiver of receivers) { + console.log(typeof receiver, JSON.stringify(readValue(receiver)), JSON.stringify(readLength(receiver)), JSON.stringify(readOptional(receiver))); +} +console.log(readOptional(null), readOptional(undefined)); +for (const bad of [null, undefined]) { + try { + readValue(bad); + } catch (e) { + console.log(e instanceof TypeError, e.message); + } +} + +const plain = { value: 0 }; +const arr = []; +writeValue(plain, 5); +writeValue(arr, "on array"); +writeValue(inherited, "own now"); +writeValue(plainFunction, "fn2"); +const status = new Status(); +writeValue(status, "aborted"); +console.log(plain.value, arr.value, Object.keys(inherited), plainFunction.value, status.value, JSON.stringify(status)); +try { + writeValue("text", 1); +} catch (e) { + console.log(e instanceof TypeError, e.message); +} +const frozen = Object.freeze({ value: "frozen" }); +try { + writeValue(frozen, 2); +} catch (e) { + console.log(e instanceof TypeError); +} +console.log(frozen.value); diff --git a/tests/corpus/7793-untyped-key-writes.js b/tests/corpus/7793-untyped-key-writes.js new file mode 100644 index 000000000..93158145e --- /dev/null +++ b/tests/corpus/7793-untyped-key-writes.js @@ -0,0 +1,83 @@ +"use strict"; +// Untyped property writes through one site: new keys append in order, +// existing keys keep their position, own and inherited setters run, +// getter-only and read-only properties refuse the write, and +// non-extensible objects refuse new keys, as in Node. + +/** @param {any} target @param {string} key @param {unknown} value */ +function put(target, key, value) { + target[key] = value; +} + +/** @param {() => void} fn */ +function attempt(label, fn) { + try { + fn(); + console.log(label, "ok"); + } catch (e) { + console.log(label, e instanceof TypeError ? "TypeError" : String(e)); + } +} + +const order = {}; +for (const key of ["b", "a", "c", "a", "b", "d"]) put(order, key, key.charCodeAt(0)); +console.log(JSON.stringify(order), Object.keys(order).join("")); + +const log = []; +const withSetter = { + _v: 0, + get v() { + return this._v; + }, + set v(value) { + log.push(`own ${value}`); + this._v = value * 2; + }, + get onlyGetter() { + return "g"; + }, +}; +put(withSetter, "v", 4); +console.log(withSetter.v, JSON.stringify(log)); +attempt("getter-only", () => put(withSetter, "onlyGetter", 1)); + +const proto = { + /** @param {number} value */ + set inherited(value) { + log.push(`inherited ${value} on ${this === child}`); + }, +}; +Object.defineProperty(proto, "fixed", { value: 1, writable: false }); +const child = Object.create(proto); +put(child, "inherited", 7); +attempt("inherited read-only", () => put(child, "fixed", 2)); +put(child, "own", 3); +console.log(JSON.stringify(log), Object.keys(child).join(","), child.fixed); + +/** @type {any} */ +const sealed = { x: 1 }; +Object.seal(sealed); +put(sealed, "x", 2); +attempt("sealed add", () => put(sealed, "y", 3)); +/** @type {any} */ +const closed = { x: 1 }; +Object.preventExtensions(closed); +put(closed, "x", 5); +attempt("closed add", () => put(closed, "z", 1)); +const frozen = Object.freeze({ x: 1 }); +attempt("frozen write", () => put(frozen, "x", 9)); +console.log(JSON.stringify(sealed), JSON.stringify(closed), JSON.stringify(frozen)); + +const readOnly = {}; +Object.defineProperty(readOnly, "k", { value: "kept", writable: false, enumerable: true }); +attempt("own read-only", () => put(readOnly, "k", "changed")); +console.log(readOnly.k); + +const error = new Error("boom"); +put(error, "cause", "why"); +put(error, "code", "E_X"); +console.log(error.cause, error.code); + +const wide = {}; +for (let i = 0; i < 40; i++) put(wide, `k${i % 25}`, i); +console.log(Object.keys(wide).length, wide.k0, wide.k24, Object.keys(wide).slice(0, 3).join(",")); diff --git a/tests/corpus/7794-untyped-array-iteration.js b/tests/corpus/7794-untyped-array-iteration.js new file mode 100644 index 000000000..ba7fe0810 --- /dev/null +++ b/tests/corpus/7794-untyped-array-iteration.js @@ -0,0 +1,86 @@ +"use strict"; +// for...of, spread and destructuring over untyped arrays: elements are read +// live, so pushes and truncation during the loop show, holes read as +// undefined, an own Symbol.iterator replaces the builtin one, and keys() +// and entries() keep their own shapes, as in Node. + +/** @param {any} list */ +function collect(list) { + const out = []; + for (const item of list) out.push(item); + return out; +} + +/** @param {any} list */ +function growWhileIterating(list) { + const seen = []; + for (const item of list) { + seen.push(item); + if (item === 2) list.push(99); + if (seen.length > 10) break; + } + return seen; +} + +/** @param {any} list */ +function shrinkWhileIterating(list) { + const seen = []; + for (const item of list) { + seen.push(item); + list.length = 2; + } + return seen; +} + +const parsed = JSON.parse('[1, "two", {"three": 3}, [4], null, true]'); +console.log(JSON.stringify(collect(parsed))); +console.log(JSON.stringify(collect([]))); +console.log(JSON.stringify(growWhileIterating(JSON.parse("[1, 2, 3]")))); +console.log(JSON.stringify(shrinkWhileIterating(JSON.parse("[1, 2, 3, 4, 5]")))); + +/** @type {any} */ +const holes = [1, , 3]; +holes.length = 5; +console.log(collect(holes).map((x) => String(x)).join(",")); + +/** @type {any} */ +const custom = JSON.parse("[1, 2, 3]"); +custom[Symbol.iterator] = function* () { + yield "custom"; + yield this.length; +}; +console.log(JSON.stringify(collect(custom))); + +/** @type {any} */ +const data = JSON.parse('[10, 20, 30]'); +console.log(JSON.stringify([...data]), JSON.stringify([...data.keys()]), JSON.stringify([...data.entries()])); +const [first, , third = "default", fourth = "missing"] = data; +console.log(first, third, fourth); + +let total = 0; +for (const value of data) { + if (value === 20) continue; + total += value; +} +console.log(total); + +/** @param {any} list */ +function nested(list) { + const pairs = []; + for (const a of list) for (const b of list) pairs.push(`${a}${b}`); + return pairs.join(" "); +} +console.log(nested(JSON.parse('["x", "y"]'))); + +/** @param {any} list */ +function earlyReturn(list) { + for (const item of list) if (item > 1) return item; + return -1; +} +console.log(earlyReturn(JSON.parse("[0, 1, 2, 3]")), earlyReturn(JSON.parse("[0]"))); + +try { + console.log(collect(null)); +} catch (e) { + console.log(e instanceof TypeError, e.message); +} diff --git a/tests/corpus/7795-untyped-global-constructors.js b/tests/corpus/7795-untyped-global-constructors.js new file mode 100644 index 000000000..0b464b7b2 --- /dev/null +++ b/tests/corpus/7795-untyped-global-constructors.js @@ -0,0 +1,29 @@ +"use strict"; +// Global constructors read as values in untyped code: instanceof tests, +// identity comparisons and constructor names, as zod's type sniffing uses +// them, answer as in Node for every global the lookup knows. + +/** @param {unknown} data */ +function kind(data) { + if (typeof Map !== "undefined" && data instanceof Map) return "map"; + if (typeof Set !== "undefined" && data instanceof Set) return "set"; + if (typeof Date !== "undefined" && data instanceof Date) return "date"; + if (data instanceof TypeError) return "type-error"; + if (data instanceof RangeError) return "range-error"; + if (data instanceof Error) return "error"; + if (data instanceof Promise) return "promise"; + if (Array.isArray(data)) return "array"; + return typeof data; +} + +const values = [new Map(), new Set([1]), new Date(0), new TypeError("t"), new RangeError("r"), new SyntaxError("s"), new Error("e"), Promise.resolve(1), [1], { a: 1 }, "s", 1, null]; +for (const value of values) console.log(kind(value)); + +const ctors = [Map, Set, Date, Error, TypeError, RangeError, SyntaxError, ReferenceError, EvalError, URIError, Object, Promise, WeakMap, WeakSet, RegExp]; +console.log(ctors.map((ctor) => ctor.name).join(",")); +console.log(ctors.every((ctor, i) => ctors.indexOf(ctor) === i)); +const again = [Map, Set, Date, Error, TypeError]; +console.log(again.every((ctor, i) => ctor === ctors[i])); +/** @type {any} */ +const err = new TypeError("x"); +console.log(err.constructor === TypeError, err.constructor === Error, err instanceof Error); diff --git a/tests/corpus/7981-typed-array-paired-loops.ts b/tests/corpus/7981-typed-array-paired-loops.ts new file mode 100644 index 000000000..d13eecbd6 --- /dev/null +++ b/tests/corpus/7981-typed-array-paired-loops.ts @@ -0,0 +1,57 @@ +// Two-counter loops over typed arrays (`for (let i = lo, j = hi; i < j; +// i++, j--)`), with starts inside the arrays and outside them: past the +// end, negative, fractional and NaN. Views over one buffer, other element +// types, inclusive conditions and steps other than one. + +function reverse(a: Int32Array, lo: number, hi: number): string { + for (let i = lo, j = hi; i < j; i++, j--) { + const t = a[i]!; + a[i] = a[j]!; + a[j] = t; + } + return Array.from(a).join(); +} + +function mirror(a: Float64Array, b: Float64Array, hi: number): number { + let sum = 0; + for (let i = 0, j = hi; i <= j; i++, j--) { + b[i] = a[j]! * 2; + sum += a[i]! - b[j]!; + } + return sum; +} + +function skipping(a: Uint8ClampedArray, hi: number): string { + for (let i = 0, j = hi; i < j; i += 2, j -= 3) { + a[i] = a[j]! + 200; + a[j] = i - 10; + } + return Array.from(a).join(); +} + +function palindrome(text: Uint16Array, n: number): boolean { + for (let i = 0, j = n - 1; i < j; i++, j--) if (text[i] !== text[j]) return false; + return true; +} + +console.log(reverse(new Int32Array([1, 2, 3, 4, 5, 6]), 0, 5)); +console.log(reverse(new Int32Array([1, 2, 3, 4, 5, 6]), 1, 3)); +console.log(reverse(new Int32Array([1, 2, 3, 4]), 0, 6)); +console.log(reverse(new Int32Array([1, 2, 3, 4]), -2, 2)); +console.log(reverse(new Int32Array([1, 2, 3, 4]), 0.5, 3.5)); +console.log(reverse(new Int32Array([1, 2, 3, 4]), 0, NaN)); +console.log(reverse(new Int32Array([1, 2, 3, 4]), 3, 0)); + +const buffer = new ArrayBuffer(48); +const whole = new Float64Array(buffer); +for (let k = 0; k < whole.length; k++) whole[k] = k + 0.5; +const tail = new Float64Array(buffer, 16, 3); +console.log(mirror(whole, tail, 4), Array.from(whole).join()); +console.log(mirror(tail, tail, 2), Array.from(whole).join()); +console.log(mirror(tail, whole, 7), Array.from(whole).join()); + +console.log(skipping(new Uint8ClampedArray([10, 20, 30, 40, 50, 60, 70, 80]), 7)); +console.log(skipping(new Uint8ClampedArray([10, 20, 30]), 9)); + +const word = new Uint16Array([114, 97, 99, 101, 99, 97, 114]); +console.log(palindrome(word, 7), palindrome(word, 6), palindrome(word, 9), palindrome(word, 0)); diff --git a/tests/corpus/957-builtins-namespace.ts b/tests/corpus/957-builtins-namespace.ts index 4b27bfe79..f6980cda6 100644 --- a/tests/corpus/957-builtins-namespace.ts +++ b/tests/corpus/957-builtins-namespace.ts @@ -17,7 +17,7 @@ function tail(p: string): string { return p.slice(i + 1); } -const scratch = "tmp-957-" + tail(process.argv[1]); +const scratch = "tmp-957-" + tail(process.argv[1]) + "-" + process.pid; if (fs.existsSync(scratch)) { fs.rmSync(scratch + "/a.txt"); fs.rmdirSync(scratch); diff --git a/tests/corpus/992-fs-roundtrip.ts b/tests/corpus/992-fs-roundtrip.ts index a039c936f..92b527fb1 100644 --- a/tests/corpus/992-fs-roundtrip.ts +++ b/tests/corpus/992-fs-roundtrip.ts @@ -32,7 +32,7 @@ function freshDir(dir: string): void { } mkdirSync(dir); } -const scratch = "tmp-992-" + tail(process.argv[1]); +const scratch = "tmp-992-" + tail(process.argv[1]) + "-" + process.pid; freshDir(scratch); const file = scratch + "/notes.txt"; diff --git a/tests/corpus/993-fs-readdir.ts b/tests/corpus/993-fs-readdir.ts index e948d3878..b0409f2ef 100644 --- a/tests/corpus/993-fs-readdir.ts +++ b/tests/corpus/993-fs-readdir.ts @@ -26,7 +26,7 @@ function freshDir(dir: string): void { } mkdirSync(dir); } -const scratch = "tmp-993-" + tail(process.argv[1]); +const scratch = "tmp-993-" + tail(process.argv[1]) + "-" + process.pid; freshDir(scratch); console.log(readdirSync(scratch).length); // fresh dir is empty diff --git a/tests/corpus/994-fs-errors.ts b/tests/corpus/994-fs-errors.ts index 74a1ed74e..d317c1aba 100644 --- a/tests/corpus/994-fs-errors.ts +++ b/tests/corpus/994-fs-errors.ts @@ -30,7 +30,7 @@ function freshDir(dir: string): void { } mkdirSync(dir); } -const scratch = "tmp-994-" + tail(process.argv[1]); +const scratch = "tmp-994-" + tail(process.argv[1]) + "-" + process.pid; freshDir(scratch); let caught = 0; diff --git a/tests/corpus/996-fs-rc-stress.ts b/tests/corpus/996-fs-rc-stress.ts index d6a8f0ba3..063845309 100644 --- a/tests/corpus/996-fs-rc-stress.ts +++ b/tests/corpus/996-fs-rc-stress.ts @@ -28,7 +28,7 @@ function freshDir(dir: string): void { } mkdirSync(dir); } -const scratch = "tmp-996-" + tail(process.argv[1]); +const scratch = "tmp-996-" + tail(process.argv[1]) + "-" + process.pid; freshDir(scratch); // Contents into an array, back through map/filter/join. diff --git a/tests/corpus/997-fs-modules/store.ts b/tests/corpus/997-fs-modules/store.ts index 65e59df1b..1b2a0f1fe 100644 --- a/tests/corpus/997-fs-modules/store.ts +++ b/tests/corpus/997-fs-modules/store.ts @@ -17,7 +17,7 @@ export function scratchDir(tag: string): string { while (i >= 0 && script.charAt(i) !== "/" && script.charAt(i) !== "\\") { i = i - 1; } - const dir = tag + script.slice(i + 1); + const dir = tag + script.slice(i + 1) + "-" + process.pid; if (existsSync(dir)) { for (const name of readdirSync(dir)) { rmSync(dir + "/" + name); diff --git a/tests/corpus/class-brand-probe-keys.ts b/tests/corpus/class-brand-probe-keys.ts new file mode 100644 index 000000000..77b5ff138 --- /dev/null +++ b/tests/corpus/class-brand-probe-keys.ts @@ -0,0 +1,68 @@ +// Untyped reads of class instances dispatch on the capsule's class key. +// These classes have keys of equal length, keys that share their last +// eight bytes, and short keys, so every stage of the brand probe decides. +class FooHandlerA { + kind = "foo-a"; + label(): string { + return `FooHandlerA:${this.kind}`; + } +} +class BarHandlerA { + kind = "bar-a"; + label(): string { + return `BarHandlerA:${this.kind}`; + } +} +class FooHandlerB extends FooHandlerA { + override kind = "foo-b"; + extra = 2; +} +class Q { + kind = "q"; + label(): string { + return "Q"; + } +} +class R { + kind = "r"; + label(): string { + return "R"; + } +} +class LongerClassNameWithSharedTail { + kind = "long"; + label(): string { + return "long"; + } +} +class OtherClassNameWithSharedTail { + kind = "other"; + label(): string { + return "other"; + } +} + +const values: any[] = [ + new FooHandlerA(), + new BarHandlerA(), + new FooHandlerB(), + new Q(), + new R(), + new LongerClassNameWithSharedTail(), + new OtherClassNameWithSharedTail(), + { kind: "plain", label: () => "plain" }, +]; + +for (const value of values) { + const extra = value.extra; + console.log(value.kind, value.label(), extra === undefined ? "-" : extra, value.constructor.name); +} + +// Writes through the untyped view reach the right instance. +for (const value of values) value.kind = `${value.kind}!`; +console.log(values.map((value) => value.kind).join(",")); + +// Computed reads go through the same tables. +const key: string = ["kind", "label"][values.length % 2]; +console.log(values.map((value) => typeof value[key]).join(",")); +console.log(values.filter((value) => value instanceof FooHandlerA).length); diff --git a/tests/corpus/fetch-headers-copy-cache.ts b/tests/corpus/fetch-headers-copy-cache.ts new file mode 100644 index 000000000..cbbc7fc6d --- /dev/null +++ b/tests/corpus/fetch-headers-copy-cache.ts @@ -0,0 +1,125 @@ +// Headers copies and the cached sort-and-combine list: copies are +// independent of their source, reads see every mutation, and live +// iterators re-read the current list after a change. +function dump(label: string, h: Headers): void { + const parts: string[] = []; + h.forEach((value, name) => parts.push(`${name}=${value}`)); + console.log(`${label}: ${parts.join(" | ")}`); +} + +// The corpus lib has no DOM iterable typings for Headers, so iterators +// are reached through an untyped view. +const iter = (h: Headers): any => h; + +const source = new Headers({ "X-B": " two ", "x-a": "one", Accept: "text/html" }); +source.append("accept", "application/json"); +source.append("Set-Cookie", "a=1"); +source.append("set-cookie", "b=2"); +dump("source", source); + +// A Headers copy keeps the combined values and stays separate. +const copy = new Headers(source); +source.set("x-a", "changed"); +copy.append("x-c", "three"); +dump("source after set", source); +dump("copy after append", copy); +console.log(JSON.stringify(copy.get("accept")), JSON.stringify(copy.get("set-cookie"))); +console.log(JSON.stringify(copy.getSetCookie())); + +// Copying into an empty Response and a Request, then appending a repeat. +const response = new Response("body", { headers: copy, status: 201 }); +response.headers.append("accept", "*/*"); +response.headers.append("cookie", "c=3"); +response.headers.append("Cookie", "d=4"); +dump("response", response.headers); +dump("copy unchanged", copy); +const clone = new Response(response.body, response); +dump("response clone", clone.headers); +const request = new Request("http://example.test/", { headers: copy }); +console.log("request x-b", JSON.stringify(request.headers.get("X-B"))); + +// Every read after a mutation sees it: get, has, iteration, spread. +const h = new Headers([["x-z", "z"], ["x-m", "m"]]); +console.log([...iter(h).entries()].map(([n, v]) => `${n}:${v}`).join(",")); +h.delete("x-z"); +console.log([...iter(h).keys()].join(","), h.has("x-z")); +h.delete("x-missing"); +console.log([...iter(h).values()].join(",")); +h.append("x-a", "first"); +console.log([...iter(h).entries()].map(([n, v]) => `${n}:${v}`).join(",")); +h.set("X-M", "again"); +console.log(JSON.stringify(h.get("x-m")), [...iter(h).keys()].join(",")); + +// A live iterator: mutations between steps change what comes next. +const it = iter(h).entries(); +console.log(JSON.stringify(it.next().value)); +h.append("x-0", "before"); +console.log(JSON.stringify(it.next().value)); +h.delete("x-m"); +h.append("x-zz", "after"); +console.log(JSON.stringify(it.next()), JSON.stringify(it.next())); + +// forEach that mutates as it walks. +const walk = new Headers([["b", "2"], ["d", "4"]]); +const seen: string[] = []; +walk.forEach((value, name) => { + seen.push(`${name}=${value}`); + if (name === "b") { + walk.append("c", "3"); + walk.append("a", "1"); + } + if (name === "c") walk.set("d", "four"); +}); +console.log(seen.join(" | ")); + +// Invalid input in a copy source is still rejected (plain object source). +try { + new Response("x", { headers: { "bad name": "v" } }); +} catch (error) { + console.log((error as Error).name, (error as Error).message); +} +try { + new Headers(source).append("x", "a\nb"); +} catch (error) { + console.log((error as Error).name, (error as Error).message); +} + +// Appending an empty value leaves the separator's space at the end of the +// combined value; a copy trims it again, as Node's does. A leading +// separator has no whitespace to trim. +const spaced = new Headers(); +spaced.append("a", "x"); +spaced.append("a", ""); +spaced.append("b", ""); +spaced.append("b", "y"); +spaced.append("c", ""); +spaced.append("c", ""); +spaced.append("cookie", "k=v"); +spaced.append("cookie", ""); +const spacedList = (headers: Headers): string => { + const out: string[] = []; + headers.forEach((value, name) => out.push(`${name}=${JSON.stringify(value)}`)); + return out.join(" "); +}; +console.log(spacedList(spaced)); +console.log(spacedList(new Headers(spaced))); +console.log(spacedList(new Headers(new Headers(spaced)))); +console.log(spacedList(new Response("", { headers: spaced }).headers)); +console.log(spacedList(new Request("http://example.test/", { headers: spaced }).headers)); + +// The same through duplicate constructor entries, in a Headers object and +// in a Response's own headers. +const entries: [string, string][] = [ + ["a", "x"], + ["a", ""], + ["c", ""], + ["c", ""], + ["d", "plain"], +]; +const built = new Headers(entries); +console.log(spacedList(built)); +console.log(spacedList(new Headers(built))); +const builtResponse = new Response("", { headers: entries }); +console.log(spacedList(builtResponse.headers)); +console.log(spacedList(new Headers(builtResponse.headers))); +console.log(spacedList(new Request("http://example.test/", { headers: built }).headers)); diff --git a/tests/corpus/string-last-index-of-ascii.ts b/tests/corpus/string-last-index-of-ascii.ts new file mode 100644 index 000000000..8dcf405f5 --- /dev/null +++ b/tests/corpus/string-last-index-of-ascii.ts @@ -0,0 +1,21 @@ +// lastIndexOf on ASCII receivers: every position form, empty and long +// needles, needles with non-ASCII text, and receivers built at run time. +const ids = ["bk_00042", "mb_00410", "plain", "_lead", "trail_", "a_b_c_d", ""]; +for (const id of ids) console.log(JSON.stringify(id), id.lastIndexOf("_"), Number(id.slice(id.lastIndexOf("_") + 1))); + +const text = "abcabcabc"; +const positions = [Infinity, -Infinity, NaN, -0, 0, 1, 2.9, 3, 5, 8, 9, 100, -5]; +for (const needle of ["abc", "c", "a", "", "abcabcabcabc", "cab", "x", "é", "\ud83d", "😀"]) { + const row = positions.map((position) => text.lastIndexOf(needle, position)); + console.log(JSON.stringify(needle), text.lastIndexOf(needle), row.join(",")); +} + +// Receivers built at run time (concatenation, repeat, join, slices). +const built = ["x", "y"].join("-").repeat(3) + String(42); +console.log(built, built.lastIndexOf("-"), built.lastIndexOf("x-", 3), built.lastIndexOf("42", 6)); +const slice = built.slice(2, 9); +console.log(slice, slice.lastIndexOf("y"), slice.lastIndexOf("y", 0), slice.lastIndexOf("-y-")); + +// A non-ASCII receiver keeps UTF-16 positions. +const mixed = "é_ü_😀_z"; +console.log(mixed.lastIndexOf("_"), mixed.lastIndexOf("_", 3), mixed.lastIndexOf("😀"), mixed.lastIndexOf("\ude00")); diff --git a/tests/diagnostics/stream-runtime-option-order.ts b/tests/diagnostics/stream-runtime-option-order.ts new file mode 100644 index 000000000..357907d5d --- /dev/null +++ b/tests/diagnostics/stream-runtime-option-order.ts @@ -0,0 +1,42 @@ +import { Writable } from "node:stream"; + +// Runtime decodeStrings and defaultEncoding values are read before the +// other option members are evaluated, while Node evaluates the members in +// order: a value after a member with effects refuses, and so does a value +// with effects after a member that reads a property those effects change. +let encoding = (process.env["STREAM_ENCODING"] ?? "utf8") as BufferEncoding; +let decode = process.env["STREAM_DECODE"] !== "0"; +function bump(): number { + encoding = "hex"; + decode = false; + return 16; +} +const sink = new Writable({ + highWaterMark: bump(), + defaultEncoding: encoding, + write(_chunk, _encoding, done) { + done(); + }, +}); +const other = new Writable({ + highWaterMark: bump(), + decodeStrings: decode, + write(_chunk, _encoding, done) { + done(); + }, +}); +const sizes = { hwm: 16 }; +function pickEncoding(): BufferEncoding { + sizes.hwm = 1; + return "hex"; +} +const third = new Writable({ + highWaterMark: sizes.hwm, + defaultEncoding: pickEncoding(), + write(_chunk, _encoding, done) { + done(); + }, +}); +sink.end(); +other.end(); +third.end(); diff --git a/tests/fixtures/fetch/static-stream/main.mts b/tests/fixtures/fetch/static-stream/main.mts index 9ff8c050b..0ae400bc3 100644 --- a/tests/fixtures/fetch/static-stream/main.mts +++ b/tests/fixtures/fetch/static-stream/main.mts @@ -259,6 +259,22 @@ try { console.log("consumed request reader:", (error as Error).name); } +// A TransformStream's readable side as the request body: fetch's upload +// reads it, so writes to the writable side do not wait on backpressure. +const uploadTransform = new TransformStream(); +const uploadWriter = uploadTransform.writable.getWriter(); +const transformPosted = fetch(`${process.argv[2]}/post-echo`, { + method: "POST", + body: uploadTransform.readable, + duplex: "half", +}); +for (const part of ["trans", "form ", "body"]) { + await uploadWriter.write(new TextEncoder().encode(part)); +} +await uploadWriter.close(); +const transformResult = (await (await transformPosted).json()) as { body: string }; +console.log("transform request body:", transformResult.body); + const prelockedRequestBody = ReadableStream.from([ Buffer.from("prelocked request"), ]); diff --git a/tests/fixtures/node-types/fenced.ts b/tests/fixtures/node-types/fenced.ts index 27e68a618..7c468de01 100644 --- a/tests/fixtures/node-types/fenced.ts +++ b/tests/fixtures/node-types/fenced.ts @@ -74,7 +74,7 @@ http2.connect("https://localhost"); * secure heap the generic hint, and setFips the FIPS truth (getFips() lowers to 0). */ import { createCipheriv, publicEncrypt, secureHeapUsed, setFips } from "node:crypto"; publicEncrypt("key", Buffer.alloc(1)); -createCipheriv("aes-128-ccm", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); +createCipheriv("aes-128-ocb", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); secureHeapUsed(); setFips(false); fetch("https://example.invalid/", { diff --git a/tests/fixtures/npm/cases/7546-minified-esm-rules/main.ts b/tests/fixtures/npm/cases/7546-minified-esm-rules/main.ts new file mode 100644 index 000000000..0c94acb0d --- /dev/null +++ b/tests/fixtures/npm/cases/7546-minified-esm-rules/main.ts @@ -0,0 +1,22 @@ +// A one-line minified ES module. Its module-level var statement spreads +// earlier declarators into later ones, its tokenizer reads regexes through +// an untyped rules table (search, match and replace with callbacks), and +// one function receives both typed and untyped link objects. +import { render } from "gtminimark"; + +const doc = [ + "# Title \\*one\\*", + "## Sub", + '[a](/x "T\\*")', + "[b](/y)", + '[r]: /ref "Ref"', + "[n]: /none", + "[R]", + "[n]", + "[z]", + "\tcode *em* line", + " three spaces *x* and *y*", + "plain words here", +].join("\n"); +console.log(render(doc)); +console.log(render(doc, { pedantic: true })); diff --git a/tests/fixtures/npm/cases/7547-minified-cjs-chunks/main.ts b/tests/fixtures/npm/cases/7547-minified-cjs-chunks/main.ts new file mode 100644 index 000000000..ad2303f47 --- /dev/null +++ b/tests/fixtures/npm/cases/7547-minified-cjs-chunks/main.ts @@ -0,0 +1,12 @@ +// An identifier-minified esbuild CommonJS bundle (lru-cache's shape): its +// __commonJS helper and chunk parameters lost their names, the chunks +// stamp __esModule with { value: !0 }, and they call node:util and +// node:diagnostics_channel functions in the (0, ns.member)(...) form. +import { Cache } from "gtminicache"; + +const cache = new Cache(2); +cache.set("a", 1).set("b", 2); +cache.get("a"); +cache.set("c", 3); +console.log(cache.keys().join(","), cache.get("b"), cache.get("c")); +console.log(cache.describe("cache")); diff --git a/tests/fixtures/npm/cases/hono-node-server/main.ts b/tests/fixtures/npm/cases/hono-node-server/main.ts new file mode 100644 index 000000000..3cca4aa44 --- /dev/null +++ b/tests/fixtures/npm/cases/hono-node-server/main.ts @@ -0,0 +1,387 @@ +// @hono/node-server 2.x (compiled from scriptc's built-in implementation) +// serving a fetch handler: a raw HTTP client prints each exchange's status +// line, wire-order headers (Date dropped) and body, covering buffered and +// streamed bodies, header forms, cookies, request bodies, URL handling, +// HEAD, empty statuses, handler errors, a client that disconnects during +// a streamed body, and request bodies the handler reads, leaves unread or +// rejects while the upload is still arriving. +import * as http from "node:http"; +import * as net from "node:net"; +import type { AddressInfo } from "node:net"; +import { getRequestListener, RequestError, serve } from "@hono/node-server"; + +function delay(ms: number): Promise { + return new Promise((resolve) => { + setTimeout(resolve, ms); + }); +} + +function slowStream(parts: string[]): ReadableStream { + const encoder = new TextEncoder(); + let index = 0; + return new ReadableStream({ + async pull(controller) { + await delay(5); + if (index < parts.length) controller.enqueue(encoder.encode(parts[index++]!)); + else controller.close(); + }, + }); +} + +function readyStream(parts: string[]): ReadableStream { + const encoder = new TextEncoder(); + return new ReadableStream({ + start(controller) { + for (const part of parts) controller.enqueue(encoder.encode(part)); + controller.close(); + }, + }); +} + +// A stream that produces until it is cancelled (bounded, so a missed cancel +// ends the program instead of hanging it). +let reportCancel: (reason: string) => void = () => {}; +const endlessCancelled = new Promise((resolve) => { + reportCancel = resolve; +}); +function endlessStream(): ReadableStream { + const encoder = new TextEncoder(); + let pulls = 0; + return new ReadableStream({ + async pull(controller) { + await delay(5); + if (++pulls > 400) controller.close(); + else controller.enqueue(encoder.encode("tick\n")); + }, + cancel(reason) { + reportCancel(`cancelled with ${String(reason)}`); + }, + }); +} + +async function handler(request: Request): Promise { + const url = new URL(request.url); + switch (url.pathname) { + case "/echo": { + const body = await request.text(); + const headers = new Headers({ "content-type": "application/json" }); + headers.set("x-echo", "yes"); + return new Response( + JSON.stringify({ + method: request.method, + url: request.url.replace(/:\d+\//, ":PORT/"), + type: request.headers.get("content-type"), + agent: request.headers.get("x-agent"), + body, + }), + { status: 201, headers }, + ); + } + case "/headers-object": { + const headers = new Headers(); + headers.set("x-b", "2"); + headers.set("X-A", "1"); + headers.set("content-type", "text/html; charset=utf-8"); + return new Response("

hi

", { status: 202, headers }); + } + case "/cookies": { + const headers = new Headers(); + headers.append("set-cookie", "a=1; Path=/"); + headers.set("x-before", "b"); + headers.append("set-cookie", "b=2; HttpOnly"); + headers.set("content-type", "text/plain"); + return new Response("cookies", { headers }); + } + case "/no-content": + return new Response(null, { status: 204, headers: new Headers({ "x-empty": "1" }) }); + case "/declared-length": { + const headers = new Headers({ "content-length": "5", "content-type": "text/plain" }); + return new Response("12345", { headers }); + } + case "/ready-stream": { + const headers = new Headers({ "content-type": "text/plain" }); + return new Response(readyStream(["one ", "two"]), { headers }); + } + case "/slow-stream": { + const headers = new Headers({ "content-type": "application/x-ndjson" }); + return new Response(slowStream(['{"n":1}\n', '{"n":2}\n', '{"n":3}\n']), { headers }); + } + case "/chunked-stream": { + const headers = new Headers({ "content-type": "text/event-stream", "transfer-encoding": "chunked" }); + return new Response(readyStream(["data: a\n\n", "data: b\n\n"]), { headers }); + } + case "/bytes": { + const headers = new Headers({ "content-type": "application/octet-stream" }); + return new Response(new Uint8Array([104, 105, 33]), { headers }); + } + case "/null-body": { + return new Response(null, { status: 404, headers: new Headers({ "x-missing": "yes" }) }); + } + case "/reject": + throw new Error("handler failed"); + case "/light-text": + return new Response("light"); + case "/light-record": + return new Response("rec", { status: 201, headers: { "X-Custom": "a", "Content-Type": "text/html" } }); + case "/light-json": + return Response.json({ ok: true, n: [1, 2] }); + case "/light-json-headers": + return Response.json({ ok: true }, { status: 201, headers: { "X-Test": "yes" } }); + case "/light-json-typed": + return Response.json([1], { headers: { "Content-Type": "application/vnd.test+json" } }); + case "/endless": + return new Response(endlessStream(), { headers: new Headers({ "content-type": "text/plain" }) }); + case "/light-touched": { + const r = new Response("touched"); + r.headers.set("x-touched", "1"); + return r; + } + case "/light-record-touched": { + const r = new Response("rt", { headers: { "X-Rec": "1" } }); + r.headers.append("x-more", "2"); + return r; + } + case "/materialized": { + const r = new Response("m", { headers: { "X-M": "1" } }); + return new Response(r.body, r); + } + case "/light-stream": + return new Response(readyStream(["s1 ", "s2"])); + case "/light-null": + return new Response(null, { status: 202 }); + case "/light-length": + return new Response("abc", { headers: { "Content-Length": "3" } }); + case "/light-bytes": + return new Response(new Uint8Array([111, 107])); + default: { + const headers = new Headers({ "content-type": "text/plain; charset=UTF-8" }); + return new Response(`path ${url.pathname}${url.search}`, { headers }); + } + } +} + +interface Exchange { + method: string; + path: string; + headers?: Record; + body?: string; +} + +function exchange(port: number, ex: Exchange): Promise { + return new Promise((resolve, reject) => { + const headers: Record = { ...(ex.headers ?? {}) }; + if (ex.body !== undefined) headers["content-length"] = String(Buffer.byteLength(ex.body)); + const req = http.request( + { host: "127.0.0.1", port, method: ex.method, path: ex.path, headers: headers }, + (res) => { + const chunks: Buffer[] = []; + res.on("data", (chunk: Buffer) => { + chunks.push(chunk); + }); + res.on("end", () => { + const lines: string[] = [`HTTP/${res.httpVersion} ${res.statusCode} ${res.statusMessage}`]; + for (let i = 0; i < res.rawHeaders.length; i += 2) { + const name = res.rawHeaders[i]!; + if (name.toLowerCase() === "date") continue; + lines.push(`${name}: ${res.rawHeaders[i + 1]}`); + } + resolve(`### ${ex.method} ${ex.path}\n${lines.join("\n")}\n\n${Buffer.concat(chunks).toString("utf8")}\n`); + }); + }, + ); + req.on("error", reject); + if (ex.body !== undefined) req.write(ex.body); + req.end(); + }); +} + +const exchanges: Exchange[] = [ + { method: "GET", path: "/plain?q=1" }, + { method: "POST", path: "/echo", headers: { "content-type": "text/plain", "x-agent": "probe" }, body: "payload body" }, + { method: "PUT", path: "/echo", headers: { "content-type": "application/json" }, body: '{"k":[1,2]}' }, + { method: "GET", path: "/headers-object" }, + { method: "GET", path: "/cookies" }, + { method: "GET", path: "/no-content" }, + { method: "GET", path: "/declared-length" }, + { method: "GET", path: "/ready-stream" }, + { method: "GET", path: "/slow-stream" }, + { method: "GET", path: "/chunked-stream" }, + { method: "GET", path: "/bytes" }, + { method: "GET", path: "/null-body" }, + { method: "HEAD", path: "/plain" }, + { method: "GET", path: "/light-text" }, + { method: "HEAD", path: "/light-text" }, + { method: "GET", path: "/light-record" }, + { method: "HEAD", path: "/light-record" }, + { method: "GET", path: "/light-json" }, + { method: "GET", path: "/light-json-headers" }, + { method: "GET", path: "/light-json-typed" }, + { method: "GET", path: "/light-touched" }, + { method: "GET", path: "/light-record-touched" }, + { method: "GET", path: "/materialized" }, + { method: "GET", path: "/light-stream" }, + { method: "GET", path: "/light-null" }, + { method: "GET", path: "/light-length" }, + { method: "GET", path: "/light-bytes" }, + { method: "GET", path: "/reject" }, + { method: "GET", path: "/a/../b/./c?x=%20y" }, + { method: "GET", path: "/upper", headers: { host: "Example.COM:8080" } }, + { method: "GET", path: "/bad-host", headers: { host: "bad host" } }, + { method: "get", path: "/lower-method" }, +]; + +/** Reads the first chunk of /endless, then destroys the response: the + * server cancels the response's stream. */ +function disconnectMidStream(port: number): Promise { + return new Promise((resolve) => { + const req = http.request({ host: "127.0.0.1", port, method: "GET", path: "/endless" }, (res) => { + res.once("data", () => { + res.destroy(); + }); + }); + req.end(); + const timeout = new Promise((done) => { + setTimeout(() => done("not cancelled"), 3000); + }); + void Promise.race([endlessCancelled, timeout]).then((outcome) => { + resolve(`### disconnect GET /endless\n${outcome}\n`); + }); + }); +} + +const server = serve({ fetch: handler, port: 0, hostname: "127.0.0.1" }, (info: AddressInfo) => { + console.log("listening", info.family, info.address, typeof info.port); + void (async () => { + for (const ex of exchanges) console.log(await exchange(info.port, ex)); + console.log(await disconnectMidStream(info.port)); + server.close(() => { + console.log("closed"); + startSecond(); + }); + })(); +}); + +// getRequestListener with an error handler, on a plain node:http server. +const listener = getRequestListener( + async (request) => { + if (request.url.endsWith("/fail")) throw new RequestError("listener failure"); + return new Response("ok", { headers: new Headers({ "content-type": "text/plain" }) }); + }, + { + errorHandler: (err) => + new Response(`handled: ${err instanceof Error ? `${err.name} ${err.message}` : "?"}`, { + status: 503, + headers: new Headers({ "content-type": "text/plain" }), + }), + }, +); +function startSecond(): void { + const second = http.createServer((req, res) => { + void listener(req, res); + }); + second.listen(0, "127.0.0.1", () => { + const port = (second.address() as AddressInfo).port; + void (async () => { + console.log(await exchange(port, { method: "GET", path: "/ok" })); + console.log(await exchange(port, { method: "GET", path: "/fail" })); + second.close(() => { + console.log("second closed"); + startUploads(); + }); + })(); + }); +} + +// Request bodies stream to the handler. /size reads the whole body. The +// other routes leave it unread: /unread answers at once (the package's +// cached-response path) and /unread-async through a promise, and /reject +// answers 413 after 300 ms while the client keeps uploading, so the server +// must stop reading the socket. +const MiB = 1024 * 1024; + +interface UploadResult { + status: string; + body: string; + sentBeforeResponse: number; +} + +/** A chunked POST over a raw socket: 64 KiB chunks, each written after + * the previous write's callback, until `limit` bytes are out (then the + * last chunk, when `finish` is set) or the response arrives. */ +function chunkedUpload(port: number, path: string, limit: number, finish: boolean): Promise { + return new Promise((resolve) => { + const socket = net.connect(port, "127.0.0.1"); + const frame = Buffer.concat([Buffer.from("10000\r\n"), Buffer.alloc(65536, 97), Buffer.from("\r\n")]); + let sent = 0; + let sentBeforeResponse = -1; + let response = ""; + let settled = false; + const settle = (): void => { + if (settled) return; + settled = true; + const headEnd = response.indexOf("\r\n\r\n"); + const status = response.slice(0, response.indexOf("\r\n")); + resolve({ status, body: headEnd < 0 ? "" : response.slice(headEnd + 4), sentBeforeResponse }); + socket.destroy(); + }; + const writeNext = (): void => { + if (sentBeforeResponse >= 0 || socket.destroyed) return; + if (sent >= limit) { + if (finish) socket.write("0\r\n\r\n"); + return; + } + sent += 65536; + socket.write(frame, () => { + writeNext(); + }); + }; + socket.on("connect", () => { + socket.write(`POST ${path} HTTP/1.1\r\nHost: 127.0.0.1\r\nTransfer-Encoding: chunked\r\n\r\n`); + writeNext(); + }); + socket.on("data", (chunk: Buffer) => { + response += chunk.toString("latin1"); + if (sentBeforeResponse < 0 && response.includes("\r\n\r\n")) sentBeforeResponse = sent; + const match = /content-length: (\d+)/i.exec(response); + const headEnd = response.indexOf("\r\n\r\n"); + if (match !== null && headEnd >= 0 && response.length - headEnd - 4 >= Number(match[1])) settle(); + }); + socket.on("error", () => settle()); + socket.on("close", () => settle()); + }); +} + +function startUploads(): void { + const server = serve( + { + fetch: (request: Request): Response | Promise => { + const url = new URL(request.url); + if (url.pathname === "/size") + return request.arrayBuffer().then((body) => new Response(`size ${body.byteLength}`)); + if (url.pathname === "/reject") + return delay(300).then(() => new Response("too large", { status: 413 })); + if (url.pathname === "/unread-async") return Promise.resolve(new Response("unread async")); + return new Response(`unread ${url.pathname}`); + }, + port: 0, + hostname: "127.0.0.1", + }, + (info: AddressInfo) => { + void (async () => { + const read = await chunkedUpload(info.port, "/size", 3 * MiB, true); + console.log(`### chunked POST /size\n${read.status}\n${read.body}\n`); + console.log(await exchange(info.port, { method: "POST", path: "/unread", body: "x".repeat(300000) })); + console.log(await exchange(info.port, { method: "GET", path: "/after-unread" })); + console.log(await exchange(info.port, { method: "POST", path: "/unread-async", body: "y".repeat(300000) })); + console.log(await exchange(info.port, { method: "GET", path: "/after-unread-async" })); + const before = process.memoryUsage().rss; + const rejected = await chunkedUpload(info.port, "/reject", 512 * MiB, false); + const grown = process.memoryUsage().rss - before; + console.log(`### chunked POST /reject\n${rejected.status}\n${rejected.body}`); + console.log("client stopped before 32 MiB:", rejected.sentBeforeResponse >= 0 && rejected.sentBeforeResponse < 32 * MiB); + console.log("server memory grew under 64 MiB:", grown < 64 * MiB); + server.close(() => console.log("uploads closed")); + })(); + }, + ); +} diff --git a/tests/fixtures/npm/cases/hono-node-server/package.json b/tests/fixtures/npm/cases/hono-node-server/package.json new file mode 100644 index 000000000..5ffd9800b --- /dev/null +++ b/tests/fixtures/npm/cases/hono-node-server/package.json @@ -0,0 +1 @@ +{ "type": "module" } diff --git a/tests/fixtures/npm/cases/types-boundary-prototype/main.ts b/tests/fixtures/npm/cases/types-boundary-prototype/main.ts new file mode 100644 index 000000000..6645535e4 --- /dev/null +++ b/tests/fixtures/npm/cases/types-boundary-prototype/main.ts @@ -0,0 +1,13 @@ +// A user interface extending a package interface types values the +// package's constructors create with a prototype (effect's Request.tagged +// pattern): they stay JavaScript values, so the package still finds its +// prototype members when the value goes back to it. +import { run, tagged, type Req } from "reqtag"; + +interface Double extends Req { + readonly _tag: "Double"; + readonly value: number; +} +const Double = tagged("Double"); +const request = Double({ value: 21 }); +console.log(run(request), request.value * 2, request._tag); diff --git a/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/index.d.ts b/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/index.d.ts new file mode 100644 index 000000000..731b6a07f --- /dev/null +++ b/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/index.d.ts @@ -0,0 +1,8 @@ +export declare const ReqTypeId: unique symbol; +export interface Req { + readonly [ReqTypeId]: { readonly _A: A }; +} +export declare function tagged & { readonly _tag: string }>( + tag: R["_tag"], +): (args: Omit) => R; +export declare function run(request: Req): string; diff --git a/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/index.js b/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/index.js new file mode 100644 index 000000000..196bf555e --- /dev/null +++ b/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/index.js @@ -0,0 +1,15 @@ +// Requests share a prototype that carries a symbol-keyed marker and a +// method, as effect's Request.tagged constructors do. +export const ReqTypeId = Symbol.for("reqtag/Req"); +const proto = { + [ReqTypeId]: true, + describe() { + return `${this._tag}(${this.value})`; + }, +}; +export function tagged(tag) { + return (args) => Object.assign(Object.create(proto), args, { _tag: tag }); +} +export function run(request) { + return request[ReqTypeId] === true ? request.describe() : "not a request"; +} diff --git a/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/package.json b/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/package.json new file mode 100644 index 000000000..e461981ee --- /dev/null +++ b/tests/fixtures/npm/cases/types-boundary-prototype/node_modules/reqtag/package.json @@ -0,0 +1 @@ +{ "name": "reqtag", "version": "1.0.0", "type": "module", "main": "index.js", "types": "index.d.ts" } diff --git a/tests/fixtures/npm/cases/types-boundary-prototype/package.json b/tests/fixtures/npm/cases/types-boundary-prototype/package.json new file mode 100644 index 000000000..ac2988811 --- /dev/null +++ b/tests/fixtures/npm/cases/types-boundary-prototype/package.json @@ -0,0 +1 @@ +{ "name": "types-boundary-prototype-case", "private": true, "type": "module" } diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/LICENSE b/tests/fixtures/npm/node_modules/@hono/node-server/LICENSE new file mode 100644 index 000000000..5f84afaff --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2022 - present, Yusuke Wada and Hono contributors + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/README.md b/tests/fixtures/npm/node_modules/@hono/node-server/README.md new file mode 100644 index 000000000..25af664e8 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/README.md @@ -0,0 +1,450 @@ +# Node.js Adapter for Hono + +This adapter `@hono/node-server` allows you to run your Hono application on Node.js. +Initially, Hono wasn't designed for Node.js, but with this adapter, you can now use Hono on Node.js. It utilizes web standard APIs implemented in Node.js. + +## Benchmarks + +Hono is 3.1 times faster than Express. + +Express v5: + +```txt +$ bombardier -d 10s --fasthttp http://localhost:3000/ + +Statistics Avg Stdev Max + Reqs/sec 24380.41 1409.74 26604.88 + Latency 5.12ms 2.95ms 305.21ms + HTTP codes: + 1xx - 0, 2xx - 243915, 3xx - 0, 4xx - 0, 5xx - 0 + others - 0 + Throughput: 6.98MB/s +``` + +Hono + `@hono/node-server`: + +```txt +$ bombardier -d 10s --fasthttp http://localhost:3000/ + +Statistics Avg Stdev Max + Reqs/sec 74428.96 3815.27 88228.32 + Latency 1.68ms 621.25us 106.20ms + HTTP codes: + 1xx - 0, 2xx - 744378, 3xx - 0, 4xx - 0, 5xx - 0 + others - 0 + Throughput: 16.82MB/s +``` + +## Requirements + +It works on Node.js versions greater than 20.x. + +## Installation + +You can install it from the npm registry: + +```sh +npm install @hono/node-server +pnpm add @hono/node-server +``` + +## Usage + +Just import `@hono/node-server` at the top and write the code as usual. +The same code that runs on Cloudflare Workers, Deno, and Bun will work. + +```ts +import { serve } from '@hono/node-server' +import { Hono } from 'hono' + +const app = new Hono() +app.get('/', (c) => c.text('Hono meets Node.js')) + +serve(app, (info) => { + console.log(`Listening on http://localhost:${info.port}`) // Listening on http://localhost:3000 +}) +``` + +## WebSocket + +You can upgrade WebSocket connections with `upgradeWebSocket` from `@hono/node-server`. +To enable this, install `ws` (and `@types/ws`) in your project, then create and provide a `WebSocketServer` as shown in the example below. + +```ts +import { serve, upgradeWebSocket } from '@hono/node-server' +import { WebSocketServer } from 'ws' +import { Hono } from 'hono' + +const app = new Hono() + +app.get( + '/ws', + upgradeWebSocket(() => ({ + onMessage(event, ws) { + ws.send(event.data) + }, + })) +) + +const wss = new WebSocketServer({ noServer: true }) +serve({ + fetch: app.fetch, + websocket: { server: wss }, +}) +``` + +For example, run it using `ts-node`. Then an HTTP server will be launched. The default port is `3000`. + +```sh +ts-node ./index.ts +``` + +Open `http://localhost:3000` with your browser. + +## Options + +### `port` + +```ts +serve({ + fetch: app.fetch, + port: 8787, // Port number, default is 3000 +}) +``` + +### `createServer` + +```ts +import { createServer } from 'node:https' +import fs from 'node:fs' + +//... + +serve({ + fetch: app.fetch, + createServer: createServer, + serverOptions: { + key: fs.readFileSync('test/fixtures/keys/agent1-key.pem'), + cert: fs.readFileSync('test/fixtures/keys/agent1-cert.pem'), + }, +}) +``` + +### `overrideGlobalObjects` + +The default value is `true`. The Node.js Adapter rewrites the global Request/Response and uses a lightweight Request/Response to improve performance. If you don't want to do that, set `false`. + +```ts +serve({ + fetch: app.fetch, + overrideGlobalObjects: false, +}) +``` + +### `autoCleanupIncoming` + +The default value is `true`. The Node.js Adapter automatically cleans up (explicitly call `destroy()` method) if application is not finished to consume the incoming request. If you don't want to do that, set `false`. + +If the application accepts connections from arbitrary clients, this cleanup must be done otherwise incomplete requests from clients may cause the application to stop responding. If your application only accepts connections from trusted clients, such as in a reverse proxy environment and there is no process that returns a response without reading the body of the POST request all the way through, you can improve performance by setting it to `false`. + +```ts +serve({ + fetch: app.fetch, + autoCleanupIncoming: false, +}) +``` + +### `websocket` + +provide a websocket server to enable websocket support. + +```ts +import { serve, upgradeWebSocket } from '@hono/node-server' +import { WebSocketServer } from 'ws' + +// ... +const wss = new WebSocketServer({ noServer: true }) + +serve({ + fetch: app.fetch, + websocket: { server: wss }, +}) +``` + +## Middleware + +Most built-in middleware also works with Node.js. +Read [the documentation](https://hono.dev/middleware/builtin/basic-auth) and use the Middleware of your liking. + +```ts +import { serve } from '@hono/node-server' +import { Hono } from 'hono' +import { prettyJSON } from 'hono/pretty-json' + +const app = new Hono() + +app.get('*', prettyJSON()) +app.get('/', (c) => c.json({ 'Hono meets': 'Node.js' })) + +serve(app) +``` + +## Serve Static Middleware + +Use Serve Static Middleware that has been created for Node.js. + +```ts +import { serveStatic } from '@hono/node-server/serve-static' + +//... + +app.use('/static/*', serveStatic({ root: './' })) +``` + +If using a relative path, `root` will be relative to the current working directory from which the app was started. + +This can cause confusion when running your application locally. + +Imagine your project structure is: + +``` +my-hono-project/ + src/ + index.ts + static/ + index.html +``` + +Typically, you would run your app from the project's root directory (`my-hono-project`), +so you would need the following code to serve the `static` folder: + +```ts +app.use('/static/*', serveStatic({ root: './static' })) +``` + +Notice that `root` here is not relative to `src/index.ts`, rather to `my-hono-project`. + +Only `GET` and `HEAD` requests are served. Requests with other methods are passed to the next middleware. + +### Options + +#### `allowPercentInPath` + +By default, `serveStatic` skips request paths that still contain `%` after routing. This prevents a second decoding step from serving a file under a different path than the one checked by route middleware. Set `allowPercentInPath: true` to keep the previous behavior when serving files whose names contain percent signs. + +If you restrict access to only some paths under `root` (for example, `/static/admin/*`), enabling this option may let requests bypass those restrictions. Apply access control to the same route pattern as `serveStatic` (for example, `/static/*`), before the static middleware: + +```ts +app.use('/static/*', authMiddleware) +app.use('/static/*', serveStatic({ root: './static', allowPercentInPath: true })) +``` + +#### `rewriteRequestPath` + +If you want to serve files in `./.foojs` with the request path `/__foo/*`, you can write like the following. + +```ts +app.use( + '/__foo/*', + serveStatic({ + root: './.foojs/', + rewriteRequestPath: (path: string) => path.replace(/^\/__foo/, ''), + }) +) +``` + +#### `onFound` + +You can specify handling when the requested file is found with `onFound`. + +```ts +app.use( + '/static/*', + serveStatic({ + // ... + onFound: (_path, c) => { + c.header('Cache-Control', `public, immutable, max-age=31536000`) + }, + }) +) +``` + +#### `onNotFound` + +The `onNotFound` is useful for debugging. You can write a handle for when a file is not found. + +```ts +app.use( + '/static/*', + serveStatic({ + root: './non-existent-dir', + onNotFound: (path, c) => { + console.log(`${path} is not found, request to ${c.req.path}`) + }, + }) +) +``` + +#### `precompressed` + +The `precompressed` option checks if files with extensions like `.br` or `.gz` are available and serves them based on the `Accept-Encoding` header. It prioritizes Brotli, then Zstd, and Gzip. If none are available, it serves the original file. + +```ts +app.use( + '/static/*', + serveStatic({ + precompressed: true, + }) +) +``` + +## ConnInfo Helper + +You can use the [ConnInfo Helper](https://hono.dev/docs/helpers/conninfo) by importing `getConnInfo` from `@hono/node-server/conninfo`. + +```ts +import { getConnInfo } from '@hono/node-server/conninfo' + +app.get('/', (c) => { + const info = getConnInfo(c) // info is `ConnInfo` + return c.text(`Your remote address is ${info.remote.address}`) +}) +``` + +## Accessing Node.js API + +You can access the Node.js API from `c.env` in Node.js. For example, if you want to specify a type, you can write the following. + +```ts +import { serve } from '@hono/node-server' +import type { HttpBindings } from '@hono/node-server' +import { Hono } from 'hono' + +const app = new Hono<{ Bindings: HttpBindings }>() + +app.get('/', (c) => { + return c.json({ + remoteAddress: c.env.incoming.socket.remoteAddress, + }) +}) + +serve(app) +``` + +The APIs that you can get from `c.env` are as follows. + +```ts +type HttpBindings = { + incoming: IncomingMessage + outgoing: ServerResponse +} + +type Http2Bindings = { + incoming: Http2ServerRequest + outgoing: Http2ServerResponse +} +``` + +## Early Hints Middleware + +You can send HTTP 103 Early Hints to instruct browsers to preload or preconnect resources before the final response is prepared. The middleware is supported under Node.js bindings (HTTP/1.1 and HTTP/2). + +### Usage + +Import `earlyHints` from `@hono/node-server/early-hints`: + +#### Static links + +```ts +import { serve } from '@hono/node-server' +import { earlyHints } from '@hono/node-server/early-hints' +import { Hono } from 'hono' + +const app = new Hono() + +app.use( + earlyHints({ + link: '; rel=preload; as=style', + }) +) + +app.get('/', (c) => { + return c.html('

Hello Hono!

') +}) + +serve(app) +``` + +#### Dynamic links + +```ts +app.use( + earlyHints({ + link: (c) => + c.req.query('theme') === 'dark' + ? '; rel=preload; as=style' + : '; rel=preload; as=style', + }) +) +``` + +> [!NOTE] +> Early Hints are sent only for requests that look like document navigations. If `Sec-Fetch-Mode` or `Sec-Fetch-Dest` is present with a value other than `navigate` or `document`, for example a `fetch()` or XHR call from a browser, a subresource request, or an iframe navigation, the middleware skips the hints and continues to the handler. Requests without these headers, such as `curl` or `fetch()` from a JavaScript runtime, are treated as navigations and do receive Early Hints. + +## Direct response from Node.js API + +You can directly respond to the client from the Node.js API. +In that case, the response from Hono should be ignored, so return `RESPONSE_ALREADY_SENT`. + +> [!NOTE] +> This feature can be used when migrating existing Node.js applications to Hono, but we recommend using Hono's API for new applications. + +```ts +import { serve } from '@hono/node-server' +import type { HttpBindings } from '@hono/node-server' +import { RESPONSE_ALREADY_SENT } from '@hono/node-server/utils/response' +import { Hono } from 'hono' + +const app = new Hono<{ Bindings: HttpBindings }>() + +app.get('/', (c) => { + const { outgoing } = c.env + outgoing.writeHead(200, { 'Content-Type': 'text/plain' }) + outgoing.end('Hello World\n') + + return RESPONSE_ALREADY_SENT +}) + +serve(app) +``` + +## Listen to a UNIX domain socket + +You can configure the HTTP server to listen to a UNIX domain socket instead of a TCP port. + +```ts +import { createAdaptorServer } from '@hono/node-server' + +// ... + +const socketPath = '/tmp/example.sock' + +const server = createAdaptorServer(app) +server.listen(socketPath, () => { + console.log(`Listening on ${socketPath}`) +}) +``` + +## Related projects + +- Hono - +- Hono GitHub repository - + +## Authors + +- Yusuke Wada +- Taku Amano + +## License + +MIT diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.cjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.cjs new file mode 100644 index 000000000..b41116486 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.cjs @@ -0,0 +1,22 @@ +Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' }); + +//#region src/conninfo.ts +/** +* ConnInfo Helper for Node.js +* @param c Context +* @returns ConnInfo +*/ +const getConnInfo = (c) => { + const bindings = c.env.server ? c.env.server : c.env; + const address = bindings.incoming.socket.remoteAddress; + const port = bindings.incoming.socket.remotePort; + const family = bindings.incoming.socket.remoteFamily; + return { remote: { + address, + port, + addressType: family === "IPv4" ? "IPv4" : family === "IPv6" ? "IPv6" : void 0 + } }; +}; + +//#endregion +exports.getConnInfo = getConnInfo; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.d.cts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.d.cts new file mode 100644 index 000000000..ed43a48f0 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.d.cts @@ -0,0 +1,11 @@ +import { GetConnInfo } from "hono/conninfo"; + +//#region src/conninfo.d.ts +/** + * ConnInfo Helper for Node.js + * @param c Context + * @returns ConnInfo + */ +declare const getConnInfo: GetConnInfo; +//#endregion +export { getConnInfo }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.d.mts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.d.mts new file mode 100644 index 000000000..ed43a48f0 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.d.mts @@ -0,0 +1,11 @@ +import { GetConnInfo } from "hono/conninfo"; + +//#region src/conninfo.d.ts +/** + * ConnInfo Helper for Node.js + * @param c Context + * @returns ConnInfo + */ +declare const getConnInfo: GetConnInfo; +//#endregion +export { getConnInfo }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.mjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.mjs new file mode 100644 index 000000000..29ec46d0d --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/conninfo.mjs @@ -0,0 +1,20 @@ +//#region src/conninfo.ts +/** +* ConnInfo Helper for Node.js +* @param c Context +* @returns ConnInfo +*/ +const getConnInfo = (c) => { + const bindings = c.env.server ? c.env.server : c.env; + const address = bindings.incoming.socket.remoteAddress; + const port = bindings.incoming.socket.remotePort; + const family = bindings.incoming.socket.remoteFamily; + return { remote: { + address, + port, + addressType: family === "IPv4" ? "IPv4" : family === "IPv6" ? "IPv6" : void 0 + } }; +}; + +//#endregion +export { getConnInfo }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/constants-BLSFu_RU.mjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/constants-BLSFu_RU.mjs new file mode 100644 index 000000000..db7c31e48 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/constants-BLSFu_RU.mjs @@ -0,0 +1,5 @@ +//#region src/utils/response/constants.ts +const X_ALREADY_SENT = "x-hono-already-sent"; + +//#endregion +export { X_ALREADY_SENT as t }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/constants-BXAKTxRC.cjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/constants-BXAKTxRC.cjs new file mode 100644 index 000000000..8888a5804 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/constants-BXAKTxRC.cjs @@ -0,0 +1,11 @@ + +//#region src/utils/response/constants.ts +const X_ALREADY_SENT = "x-hono-already-sent"; + +//#endregion +Object.defineProperty(exports, 'X_ALREADY_SENT', { + enumerable: true, + get: function () { + return X_ALREADY_SENT; + } +}); \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.cjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.cjs new file mode 100644 index 000000000..aa3ceba6a --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.cjs @@ -0,0 +1,35 @@ +Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' }); + +//#region src/early-hints.ts +/** +* Early Hints middleware for Node.js +* Automatically sends a 103 Early Hints informational response with the specified Link header(s). +* +* @param options EarlyHintsOptions +* @returns MiddlewareHandler +*/ +const earlyHints = (options) => { + let warned = false; + return async (c, next) => { + const mode = c.req.header("Sec-Fetch-Mode"); + const dest = c.req.header("Sec-Fetch-Dest"); + if (mode && mode !== "navigate" || dest && dest !== "document") return next(); + const env = c.env || {}; + const outgoing = ("server" in env ? env.server : env)?.outgoing; + if (typeof outgoing?.writeEarlyHints !== "function") { + if (!warned) { + console.warn("Early Hints Middleware is not supported because writeEarlyHints is not defined."); + warned = true; + } + return await next(); + } + if (!outgoing.headersSent) { + const link = typeof options.link === "function" ? options.link(c) : options.link; + if (link !== void 0 && (Array.isArray(link) ? link.length > 0 : Boolean(link))) outgoing.writeEarlyHints({ link }); + } + await next(); + }; +}; + +//#endregion +exports.earlyHints = earlyHints; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.d.cts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.d.cts new file mode 100644 index 000000000..9bd27d5e3 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.d.cts @@ -0,0 +1,16 @@ +import { Context, Env, MiddlewareHandler } from "hono"; + +//#region src/early-hints.d.ts +type EarlyHintsOptions = { + link: string | string[] | ((c: Context) => string | string[] | undefined); +}; +/** + * Early Hints middleware for Node.js + * Automatically sends a 103 Early Hints informational response with the specified Link header(s). + * + * @param options EarlyHintsOptions + * @returns MiddlewareHandler + */ +declare const earlyHints: (options: EarlyHintsOptions) => MiddlewareHandler; +//#endregion +export { EarlyHintsOptions, earlyHints }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.d.mts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.d.mts new file mode 100644 index 000000000..9bd27d5e3 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.d.mts @@ -0,0 +1,16 @@ +import { Context, Env, MiddlewareHandler } from "hono"; + +//#region src/early-hints.d.ts +type EarlyHintsOptions = { + link: string | string[] | ((c: Context) => string | string[] | undefined); +}; +/** + * Early Hints middleware for Node.js + * Automatically sends a 103 Early Hints informational response with the specified Link header(s). + * + * @param options EarlyHintsOptions + * @returns MiddlewareHandler + */ +declare const earlyHints: (options: EarlyHintsOptions) => MiddlewareHandler; +//#endregion +export { EarlyHintsOptions, earlyHints }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.mjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.mjs new file mode 100644 index 000000000..7999a3d0c --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/early-hints.mjs @@ -0,0 +1,33 @@ +//#region src/early-hints.ts +/** +* Early Hints middleware for Node.js +* Automatically sends a 103 Early Hints informational response with the specified Link header(s). +* +* @param options EarlyHintsOptions +* @returns MiddlewareHandler +*/ +const earlyHints = (options) => { + let warned = false; + return async (c, next) => { + const mode = c.req.header("Sec-Fetch-Mode"); + const dest = c.req.header("Sec-Fetch-Dest"); + if (mode && mode !== "navigate" || dest && dest !== "document") return next(); + const env = c.env || {}; + const outgoing = ("server" in env ? env.server : env)?.outgoing; + if (typeof outgoing?.writeEarlyHints !== "function") { + if (!warned) { + console.warn("Early Hints Middleware is not supported because writeEarlyHints is not defined."); + warned = true; + } + return await next(); + } + if (!outgoing.headersSent) { + const link = typeof options.link === "function" ? options.link(c) : options.link; + if (link !== void 0 && (Array.isArray(link) ? link.length > 0 : Boolean(link))) outgoing.writeEarlyHints({ link }); + } + await next(); + }; +}; + +//#endregion +export { earlyHints }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.cjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.cjs new file mode 100644 index 000000000..e578de054 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.cjs @@ -0,0 +1,1340 @@ +Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' }); +const require_constants = require('./constants-BXAKTxRC.cjs'); +let node_http = require("node:http"); +let node_http2 = require("node:http2"); +let node_stream = require("node:stream"); +let hono_ws = require("hono/ws"); + +//#region src/error.ts +var RequestError = class extends Error { + constructor(message, options) { + super(message, options); + this.name = "RequestError"; + } +}; + +//#endregion +//#region src/headers.ts +const nonJoinedHeaders = new Set([ + "age", + "authorization", + "content-length", + "content-type", + "etag", + "expires", + "from", + "host", + "if-modified-since", + "if-unmodified-since", + "last-modified", + "location", + "max-forwards", + "proxy-authorization", + "referer", + "retry-after", + "server", + "user-agent" +]); +const validHeaderName = /^[!#$%&'*+\-.^_`|~\dA-Za-z]+$/; +const isHttpWhitespace = (code) => code === 9 || code === 10 || code === 13 || code === 32; +const normalizeHeaderValue = (value) => { + if (!isHttpWhitespace(value.charCodeAt(0)) && !isHttpWhitespace(value.charCodeAt(value.length - 1))) return value; + let start = 0; + let end = value.length; + while (start < end && isHttpWhitespace(value.charCodeAt(start))) start++; + while (end > start && isHttpWhitespace(value.charCodeAt(end - 1))) end--; + return value.slice(start, end); +}; +const forbiddenHeaderValue = /[\0\r\n]/; +const GlobalHeaders = globalThis.Headers; +const materializeHeaders = (rawHeaders, HeadersCtor = GlobalHeaders) => { + const headers = new HeadersCtor(); + for (let i = 0; i < rawHeaders.length; i += 2) { + const name = rawHeaders[i]; + if (!name.startsWith(":")) headers.append(name, rawHeaders[i + 1]); + } + return headers; +}; +var RequestHeaders = class { + #incoming; + #rawHeaders; + #headers; + #invalidValue; + constructor(incoming) { + this.#incoming = incoming; + if (incoming instanceof node_http2.Http2ServerRequest) this.#rawHeaders = incoming.rawHeaders.slice(); + } + get #lazyRawHeaders() { + return this.#rawHeaders ??= this.#incoming.rawHeaders.slice(); + } + get #native() { + if (!this.#headers) { + this.#headers = materializeHeaders(this.#lazyRawHeaders); + this.#rawHeaders = void 0; + } + return this.#headers; + } + #normalizedName(name) { + if (typeof name !== "string") return; + if (!validHeaderName.test(name)) throw new TypeError(`Invalid header name: ${name}`); + return name.toLowerCase(); + } + #lookupHttp1(lowerName) { + const headers = this.#incoming instanceof node_http2.Http2ServerRequest ? void 0 : this.#incoming.headers; + if (!headers || nonJoinedHeaders.has(lowerName) || lowerName === "set-cookie" || lowerName === "__proto__") return; + if (!Object.hasOwn(headers, lowerName)) return null; + const rawValue = headers[lowerName]; + if (typeof rawValue === "string") { + const value = normalizeHeaderValue(rawValue); + return forbiddenHeaderValue.test(value) ? void 0 : value; + } + } + #lookup(rawHeaders, lowerName) { + const separator = lowerName === "cookie" ? "; " : ", "; + let value = null; + for (let i = 0; i < rawHeaders.length; i += 2) { + const rawName = rawHeaders[i]; + if (rawName.length === lowerName.length && rawName.toLowerCase() === lowerName) { + const rawValue = normalizeHeaderValue(rawHeaders[i + 1]); + if (forbiddenHeaderValue.test(rawValue)) { + this.#invalidValue = true; + return; + } + value = value === null ? rawValue : value + separator + rawValue; + } + } + return value; + } + append(name, value) { + this.#native.append(name, value); + } + delete(name) { + this.#native.delete(name); + } + get(name) { + const lowerName = this.#normalizedName(name); + if (lowerName && !this.#headers && !this.#invalidValue) { + const http1Value = this.#lookupHttp1(lowerName); + if (http1Value !== void 0) return http1Value; + const value = this.#lookup(this.#lazyRawHeaders, lowerName); + if (value !== void 0) return value; + } + return this.#native.get(name); + } + has(name) { + const lowerName = this.#normalizedName(name); + if (lowerName && !this.#headers && !this.#invalidValue) { + const http1Value = this.#lookupHttp1(lowerName); + if (http1Value !== void 0) return http1Value !== null; + const value = this.#lookup(this.#lazyRawHeaders, lowerName); + if (value !== void 0) return value !== null; + } + return this.#native.has(name); + } + set(name, value) { + this.#native.set(name, value); + } + getSetCookie() { + return this.#native.getSetCookie(); + } + keys() { + return this.#native.keys(); + } + values() { + return this.#native.values(); + } + entries() { + return this.#native.entries(); + } + forEach(callback, thisArg) { + this.#native.forEach((value, key) => { + callback.call(thisArg, value, key, this); + }); + } + [Symbol.iterator]() { + return this.entries(); + } +}; +Object.defineProperty(RequestHeaders.prototype, Symbol.for("nodejs.util.inspect.custom"), { value: function(depth, options, inspectFn) { + return `Headers (lightweight) ${inspectFn(Object.fromEntries(this), { + ...options, + depth: depth == null ? null : depth - 1 + })}`; +} }); +Object.setPrototypeOf(RequestHeaders.prototype, GlobalHeaders.prototype); +const newHeadersFromIncoming = (incoming) => globalThis.Headers === GlobalHeaders ? new RequestHeaders(incoming) : materializeHeaders(incoming.rawHeaders, globalThis.Headers); + +//#endregion +//#region src/url.ts +const reValidRequestUrl = /^\/[!#$&-;=?-\[\]_a-z~]*$/; +const reDotSegment = /\/\.\.?(?:[/?#]|$)/; +const reValidHost = /^[a-z0-9._-]+(?::(?:[1-5]\d{3,4}|[6-9]\d{3}))?$/; +const buildUrl = (scheme, host, incomingUrl) => { + const url = `${scheme}://${host}${incomingUrl}`; + if (!reValidHost.test(host)) { + const urlObj = new URL(url); + if (urlObj.hostname.length !== host.length && urlObj.hostname !== (host.includes(":") ? host.replace(/:\d+$/, "") : host).toLowerCase()) throw new RequestError("Invalid host header"); + return urlObj.href; + } else if (incomingUrl.length === 0) return url + "/"; + else { + if (incomingUrl.charCodeAt(0) !== 47) throw new RequestError("Invalid URL"); + if (!reValidRequestUrl.test(incomingUrl) || reDotSegment.test(incomingUrl)) return new URL(url).href; + return url; + } +}; + +//#endregion +//#region src/request.ts +const toRequestError = (e) => { + if (e instanceof RequestError) return e; + return new RequestError(e.message, { cause: e }); +}; +const GlobalRequest = global.Request; +var Request$1 = class extends GlobalRequest { + constructor(input, options) { + if (typeof input === "object" && getRequestCache in input) { + const hasReplacementBody = options !== void 0 && "body" in options && options.body != null; + if (input[bodyConsumedDirectlyKey] && !hasReplacementBody) throw new TypeError("Cannot construct a Request with a Request object that has already been used."); + input = input[getRequestCache](); + } + if (typeof (options?.body)?.getReader !== "undefined") options.duplex ??= "half"; + super(input, options); + } +}; +const wrapBodyStream = Symbol("wrapBodyStream"); +const byteExactEncodings = new Set([ + "latin1", + "binary", + "hex", + "base64", + "base64url" +]); +const isByteExactEncoding = (encoding) => encoding === null || byteExactEncodings.has(encoding); +const bodyBufferedBeforeDisconnectKey = Symbol("bodyBufferedBeforeDisconnect"); +const bodyBufferedLengthBeforeDisconnectKey = Symbol("bodyBufferedLengthBeforeDisconnect"); +const toBufferChunk = (chunk, encoding) => Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk, encoding ?? "utf8"); +const isRecoverableDisconnectedIncoming = (incoming) => !(incoming instanceof node_http2.Http2ServerRequest) && !!incoming.complete && !!incoming.readableAborted && typeof incoming.read === "function" && isByteExactEncoding(incoming.readableEncoding); +const recordBodyBufferedBeforeDisconnect = (incoming) => { + if (incoming.readableDidRead || !isRecoverableDisconnectedIncoming(incoming)) return; + const incomingWithRecovery = incoming; + incomingWithRecovery[bodyBufferedLengthBeforeDisconnectKey] ??= incoming.readableLength; +}; +const readBodyBufferedBeforeDisconnect = (incoming, chunks) => { + if (incoming.readableDidRead && !chunks || !isRecoverableDisconnectedIncoming(incoming)) return; + const incomingWithRecovery = incoming; + if (incomingWithRecovery[bodyBufferedBeforeDisconnectKey] !== void 0) return incomingWithRecovery[bodyBufferedBeforeDisconnectKey]; + let result; + const errored = incoming.errored; + if (errored && errored.code !== "ECONNRESET") result = errored; + else if (incomingWithRecovery[bodyBufferedLengthBeforeDisconnectKey] !== void 0 && incoming.readableLength !== incomingWithRecovery[bodyBufferedLengthBeforeDisconnectKey]) result = newBodyUnusableError(); + else { + const bodyChunks = chunks ?? []; + const chunk = incoming.read(); + if (chunk !== null) bodyChunks.push(toBufferChunk(chunk, incoming.readableEncoding)); + const buffer = bodyChunks.length === 1 ? bodyChunks[0] : Buffer.concat(bodyChunks); + result = buffer; + const contentLength = incoming.headers["content-length"]; + if (typeof contentLength === "string" && /^\d+$/.test(contentLength)) { + const expectedLength = Number(contentLength); + if (Number.isSafeInteger(expectedLength) && buffer.length !== expectedLength) result = newBodyUnusableError(); + } + } + incomingWithRecovery[bodyBufferedBeforeDisconnectKey] = result; + return result; +}; +const enqueueBufferedBody = (controller, buffered) => { + if (buffered instanceof Error) { + controller.error(buffered); + return; + } + if (buffered.length > 0) controller.enqueue(buffered); + controller.close(); +}; +const newRequestFromIncoming = (method, url, headers, incoming, abortController) => { + const init = { + method, + headers, + signal: abortController.signal + }; + if (method === "TRACE") { + init.method = "GET"; + const req = new Request$1(url, init); + Object.defineProperty(req, "method", { get() { + return "TRACE"; + } }); + return req; + } + if (!(method === "GET" || method === "HEAD")) if ("rawBody" in incoming && incoming.rawBody instanceof Buffer) init.body = new ReadableStream({ start(controller) { + controller.enqueue(incoming.rawBody); + controller.close(); + } }); + else if (incoming[wrapBodyStream]) { + let reader; + init.body = new ReadableStream({ async pull(controller) { + try { + if (!reader) { + const buffered = readBodyBufferedBeforeDisconnect(incoming); + if (buffered !== void 0) { + enqueueBufferedBody(controller, buffered); + return; + } + } + reader ||= node_stream.Readable.toWeb(incoming).getReader(); + const { done, value } = await reader.read(); + if (done) controller.close(); + else controller.enqueue(value); + } catch (error) { + controller.error(error); + } + } }); + } else { + const buffered = readBodyBufferedBeforeDisconnect(incoming); + if (buffered !== void 0) init.body = new ReadableStream({ start(controller) { + enqueueBufferedBody(controller, buffered); + } }); + else init.body = node_stream.Readable.toWeb(incoming); + } + return new Request$1(url, init); +}; +const getRequestCache = Symbol("getRequestCache"); +const requestCache = Symbol("requestCache"); +const incomingKey = Symbol("incomingKey"); +const urlKey = Symbol("urlKey"); +const methodKey = Symbol("methodKey"); +const headersKey = Symbol("headersKey"); +const abortControllerKey = Symbol("abortControllerKey"); +const getAbortController = Symbol("getAbortController"); +const abortRequest = Symbol("abortRequest"); +const bodyBufferKey = Symbol("bodyBuffer"); +const bodyReadPromiseKey = Symbol("bodyReadPromise"); +const bodyConsumedDirectlyKey = Symbol("bodyConsumedDirectly"); +const bodyLockReaderKey = Symbol("bodyLockReader"); +const abortReasonKey = Symbol("abortReason"); +const newBodyUnusableError = () => { + return /* @__PURE__ */ new TypeError("Body is unusable"); +}; +const rejectBodyUnusable = () => { + return Promise.reject(newBodyUnusableError()); +}; +const textDecoder = new TextDecoder(); +const consumeBodyDirectOnce = (request) => { + if (request[bodyConsumedDirectlyKey]) return rejectBodyUnusable(); + request[bodyConsumedDirectlyKey] = true; +}; +const toArrayBuffer = (buf) => { + return buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength); +}; +const contentType = (request) => { + return (request[headersKey] ||= newHeadersFromIncoming(request[incomingKey])).get("content-type") || ""; +}; +const methodTokenRegExp = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/; +const normalizeIncomingMethod = (method) => { + if (typeof method !== "string" || method.length === 0) return "GET"; + switch (method) { + case "DELETE": + case "GET": + case "HEAD": + case "OPTIONS": + case "PATCH": + case "POST": + case "PUT": + case "QUERY": return method; + } + const upper = method.toUpperCase(); + switch (upper) { + case "DELETE": + case "GET": + case "HEAD": + case "OPTIONS": + case "POST": + case "PUT": return upper; + default: return method; + } +}; +const validateDirectReadMethod = (method) => { + if (!methodTokenRegExp.test(method)) return /* @__PURE__ */ new TypeError(`'${method}' is not a valid HTTP method.`); + const normalized = method.toUpperCase(); + if (normalized === "CONNECT" || normalized === "TRACK" || normalized === "TRACE" && method !== "TRACE") return /* @__PURE__ */ new TypeError(`'${method}' HTTP method is unsupported.`); +}; +const readBodyWithFastPath = (request, method, fromBuffer) => { + if (request[bodyConsumedDirectlyKey]) return rejectBodyUnusable(); + const methodName = request.method; + if (methodName === "GET" || methodName === "HEAD") return request[getRequestCache]()[method](); + const methodValidationError = validateDirectReadMethod(methodName); + if (methodValidationError) return Promise.reject(methodValidationError); + if (request[requestCache]) { + if (methodName !== "TRACE") return request[requestCache][method](); + } + const alreadyUsedError = consumeBodyDirectOnce(request); + if (alreadyUsedError) return alreadyUsedError; + const raw = readRawBodyIfAvailable(request); + if (raw) { + const result = Promise.resolve(fromBuffer(raw, request)); + request[bodyBufferKey] = void 0; + return result; + } + return readBodyDirect(request).then((buf) => { + const result = fromBuffer(buf, request); + request[bodyBufferKey] = void 0; + return result; + }); +}; +const readRawBodyIfAvailable = (request) => { + const incoming = request[incomingKey]; + if ("rawBody" in incoming && incoming.rawBody instanceof Buffer) return incoming.rawBody; +}; +const normalizeAbortError = (request, incoming) => { + if (incoming.errored) return incoming.errored; + const reason = request[abortReasonKey]; + if (reason !== void 0) return reason instanceof Error ? reason : new Error(String(reason)); + return /* @__PURE__ */ new Error("Client connection prematurely closed."); +}; +const readBodyDirect = (request) => { + if (request[bodyBufferKey]) return Promise.resolve(request[bodyBufferKey]); + if (request[bodyReadPromiseKey]) return request[bodyReadPromiseKey]; + const incoming = request[incomingKey]; + if (incoming.readableDidRead) return rejectBodyUnusable(); + const buffered = readBodyBufferedBeforeDisconnect(incoming); + if (buffered !== void 0) { + if (buffered instanceof Error) return Promise.reject(buffered); + request[bodyBufferKey] = buffered; + return Promise.resolve(buffered); + } + const promise = new Promise((resolve, reject) => { + const chunks = []; + let settled = false; + const finish = (callback) => { + if (settled) return; + settled = true; + cleanup(); + callback(); + }; + const recoverCompleteBodyAfterDisconnect = (error) => { + const streamError = incoming.errored ?? error; + if (!isRecoverableDisconnectedIncoming(incoming) || streamError && streamError.code !== "ECONNRESET") return false; + finish(() => { + const recovered = readBodyBufferedBeforeDisconnect(incoming, chunks); + if (recovered instanceof Error) reject(recovered); + else if (recovered === void 0) reject(error ?? normalizeAbortError(request, incoming)); + else { + request[bodyBufferKey] = recovered; + resolve(recovered); + } + }); + return true; + }; + const onData = (chunk) => { + chunks.push(toBufferChunk(chunk, incoming.readableEncoding)); + }; + const onEnd = () => { + finish(() => { + const buffer = chunks.length === 1 ? chunks[0] : Buffer.concat(chunks); + request[bodyBufferKey] = buffer; + resolve(buffer); + }); + }; + const onError = (error) => { + if (recoverCompleteBodyAfterDisconnect(error)) return; + finish(() => { + reject(error); + }); + }; + const onClose = () => { + if (incoming.readableEnded) { + onEnd(); + return; + } + if (recoverCompleteBodyAfterDisconnect()) return; + finish(() => { + reject(normalizeAbortError(request, incoming)); + }); + }; + const cleanup = () => { + incoming.off("data", onData); + incoming.off("end", onEnd); + incoming.off("error", onError); + incoming.off("close", onClose); + request[bodyReadPromiseKey] = void 0; + }; + incoming.on("data", onData); + incoming.on("end", onEnd); + incoming.on("error", onError); + incoming.on("close", onClose); + queueMicrotask(() => { + if (settled) return; + if (incoming.readableEnded) onEnd(); + else if (incoming.errored) onError(incoming.errored); + else if (incoming.destroyed) onClose(); + }); + }); + request[bodyReadPromiseKey] = promise; + return promise; +}; +const requestPrototype = { + get method() { + return this[methodKey]; + }, + get url() { + return this[urlKey]; + }, + get headers() { + return this[headersKey] ||= newHeadersFromIncoming(this[incomingKey]); + }, + [abortRequest](reason) { + if (this[abortReasonKey] === void 0) this[abortReasonKey] = reason; + const abortController = this[abortControllerKey]; + if (abortController && !abortController.signal.aborted) abortController.abort(reason); + }, + [getAbortController]() { + this[abortControllerKey] ||= new AbortController(); + if (this[abortReasonKey] !== void 0 && !this[abortControllerKey].signal.aborted) this[abortControllerKey].abort(this[abortReasonKey]); + return this[abortControllerKey]; + }, + [getRequestCache]() { + const abortController = this[getAbortController](); + if (this[requestCache]) return this[requestCache]; + const method = this.method; + if (this[bodyConsumedDirectlyKey] && !(method === "GET" || method === "HEAD")) { + this[bodyBufferKey] = void 0; + const init = { + method: method === "TRACE" ? "GET" : method, + headers: this.headers, + signal: abortController.signal + }; + if (method !== "TRACE") { + init.body = new ReadableStream({ start(c) { + c.close(); + } }); + init.duplex = "half"; + } + const req = new Request$1(this[urlKey], init); + if (method === "TRACE") Object.defineProperty(req, "method", { get() { + return "TRACE"; + } }); + return this[requestCache] = req; + } + return this[requestCache] = newRequestFromIncoming(this.method, this[urlKey], this.headers, this[incomingKey], abortController); + }, + get body() { + if (!this[bodyConsumedDirectlyKey]) return this[getRequestCache]().body; + const request = this[getRequestCache](); + if (!this[bodyLockReaderKey] && request.body) this[bodyLockReaderKey] = request.body.getReader(); + return request.body; + }, + get bodyUsed() { + if (this[bodyConsumedDirectlyKey]) return true; + if (this[requestCache]) return this[requestCache].bodyUsed; + return false; + } +}; +Object.defineProperty(requestPrototype, "signal", { get() { + return this[getAbortController]().signal; +} }); +[ + "cache", + "credentials", + "destination", + "integrity", + "mode", + "redirect", + "referrer", + "referrerPolicy", + "keepalive" +].forEach((k) => { + Object.defineProperty(requestPrototype, k, { get() { + return this[getRequestCache]()[k]; + } }); +}); +["clone", "formData"].forEach((k) => { + Object.defineProperty(requestPrototype, k, { value: function() { + if (this[bodyConsumedDirectlyKey]) { + if (k === "clone") throw newBodyUnusableError(); + return rejectBodyUnusable(); + } + return this[getRequestCache]()[k](); + } }); +}); +Object.defineProperty(requestPrototype, "text", { value: function() { + return readBodyWithFastPath(this, "text", (buf) => textDecoder.decode(buf)); +} }); +Object.defineProperty(requestPrototype, "arrayBuffer", { value: function() { + return readBodyWithFastPath(this, "arrayBuffer", (buf) => toArrayBuffer(buf)); +} }); +Object.defineProperty(requestPrototype, "blob", { value: function() { + return readBodyWithFastPath(this, "blob", (buf, request) => { + const type = contentType(request); + const init = type ? { headers: { "content-type": type } } : void 0; + return new Response(buf, init).blob(); + }); +} }); +Object.defineProperty(requestPrototype, "json", { value: function() { + if (this[bodyConsumedDirectlyKey]) return rejectBodyUnusable(); + return this.text().then(JSON.parse); +} }); +Object.defineProperty(requestPrototype, Symbol.for("nodejs.util.inspect.custom"), { value: function(depth, options, inspectFn) { + return `Request (lightweight) ${inspectFn({ + method: this.method, + url: this.url, + headers: this.headers, + nativeRequest: this[requestCache] + }, { + ...options, + depth: depth == null ? null : depth - 1 + })}`; +} }); +Object.setPrototypeOf(requestPrototype, Request$1.prototype); +const newRequest = (incoming, defaultHostname) => { + const req = Object.create(requestPrototype); + req[incomingKey] = incoming; + req[methodKey] = normalizeIncomingMethod(incoming.method); + const incomingUrl = incoming.url || ""; + if (incomingUrl[0] !== "/" && (incomingUrl.startsWith("http://") || incomingUrl.startsWith("https://"))) { + if (incoming instanceof node_http2.Http2ServerRequest) throw new RequestError("Absolute URL for :path is not allowed in HTTP/2"); + try { + req[urlKey] = new URL(incomingUrl).href; + } catch (e) { + throw new RequestError("Invalid absolute URL", { cause: e }); + } + return req; + } + const host = (incoming instanceof node_http2.Http2ServerRequest ? incoming.authority : incoming.headers.host) || defaultHostname; + if (!host) throw new RequestError("Missing host header"); + let scheme; + if (incoming instanceof node_http2.Http2ServerRequest) { + scheme = incoming.scheme; + if (!(scheme === "http" || scheme === "https")) throw new RequestError("Unsupported scheme"); + } else scheme = incoming.socket && incoming.socket.encrypted ? "https" : "http"; + try { + req[urlKey] = buildUrl(scheme, host, incomingUrl); + } catch (e) { + if (e instanceof RequestError) throw e; + else throw new RequestError("Invalid URL", { cause: e }); + } + return req; +}; + +//#endregion +//#region src/response.ts +const defaultContentType = "text/plain; charset=UTF-8"; +const responseCache = Symbol("responseCache"); +const getResponseCache = Symbol("getResponseCache"); +const cacheKey = Symbol("cache"); +const GlobalResponse = global.Response; +var Response$1 = class Response$1 { + #body; + #init; + [getResponseCache]() { + const cache = this[cacheKey]; + const liveHeaders = cache && cache[2] instanceof Headers ? cache[2] : void 0; + delete this[cacheKey]; + return this[responseCache] ||= new GlobalResponse(this.#body, liveHeaders ? { + status: this.#init?.status, + statusText: this.#init?.statusText, + headers: liveHeaders + } : this.#init); + } + constructor(body, init) { + let headers; + this.#body = body; + if (init instanceof GlobalResponse) { + const cachedGlobalResponse = init[responseCache]; + if (cachedGlobalResponse) { + this.#init = cachedGlobalResponse; + this[getResponseCache](); + return; + } + this.#init = init instanceof Response$1 ? init.#init : init; + headers = new Headers(init.headers); + } else this.#init = init; + if (body == null || typeof body === "string" || typeof body?.getReader !== "undefined" || body instanceof Blob || body instanceof Uint8Array) this[cacheKey] = [ + init?.status || 200, + body ?? null, + headers || init?.headers + ]; + } + get headers() { + const cache = this[cacheKey]; + if (cache) { + if (!(cache[2] instanceof Headers)) cache[2] = new Headers(cache[2] || (cache[1] === null ? void 0 : { "content-type": defaultContentType })); + return cache[2]; + } + return this[getResponseCache]().headers; + } + get status() { + return this[cacheKey]?.[0] ?? this[getResponseCache]().status; + } + get ok() { + const status = this.status; + return status >= 200 && status < 300; + } +}; +[ + "body", + "bodyUsed", + "redirected", + "statusText", + "trailers", + "type", + "url" +].forEach((k) => { + Object.defineProperty(Response$1.prototype, k, { get() { + return this[getResponseCache]()[k]; + } }); +}); +[ + "arrayBuffer", + "blob", + "clone", + "formData", + "json", + "text" +].forEach((k) => { + Object.defineProperty(Response$1.prototype, k, { value: function() { + return this[getResponseCache]()[k](); + } }); +}); +Object.defineProperty(Response$1.prototype, Symbol.for("nodejs.util.inspect.custom"), { value: function(depth, options, inspectFn) { + return `Response (lightweight) ${inspectFn({ + status: this.status, + headers: this.headers, + ok: this.ok, + nativeResponse: this[responseCache] + }, { + ...options, + depth: depth == null ? null : depth - 1 + })}`; +} }); +Object.setPrototypeOf(Response$1, GlobalResponse); +Object.setPrototypeOf(Response$1.prototype, GlobalResponse.prototype); +const validRedirectUrl = /^https?:\/\/[!#-;=?-[\]_a-z~A-Z]+$/; +const parseRedirectUrl = (url) => { + if (url instanceof URL) return url.href; + if (validRedirectUrl.test(url)) return url; + return new URL(url).href; +}; +const validRedirectStatuses = new Set([ + 301, + 302, + 303, + 307, + 308 +]); +Object.defineProperty(Response$1, "redirect", { + value: function redirect(url, status = 302) { + if (!validRedirectStatuses.has(status)) throw new RangeError("Invalid status code"); + return new Response$1(null, { + status, + headers: { location: parseRedirectUrl(url) } + }); + }, + writable: true, + configurable: true +}); +Object.defineProperty(Response$1, "json", { + value: function json(data, init) { + const body = JSON.stringify(data); + if (body === void 0) throw new TypeError("The data is not JSON serializable"); + const initHeaders = init?.headers; + let headers; + if (initHeaders) { + headers = new Headers(initHeaders); + if (!headers.has("content-type")) headers.set("content-type", "application/json"); + } else headers = { "content-type": "application/json" }; + return new Response$1(body, { + status: init?.status ?? 200, + statusText: init?.statusText, + headers + }); + }, + writable: true, + configurable: true +}); + +//#endregion +//#region src/utils.ts +async function readWithoutBlocking(readPromise) { + return Promise.race([readPromise, Promise.resolve().then(() => Promise.resolve(void 0))]); +} +function writeFromReadableStreamDefaultReader(reader, writable, currentReadPromise) { + const cancel = (error) => { + reader.cancel(error).catch(() => {}); + }; + writable.on("close", cancel); + writable.on("error", cancel); + (currentReadPromise ?? reader.read()).then(flow, handleStreamError); + return reader.closed.finally(() => { + writable.off("close", cancel); + writable.off("error", cancel); + }); + function handleStreamError(error) { + if (error) writable.destroy(error); + } + function onDrain() { + reader.read().then(flow, handleStreamError); + } + function flow({ done, value }) { + try { + if (done) writable.end(); + else if (!writable.write(value)) writable.once("drain", onDrain); + else return reader.read().then(flow, handleStreamError); + } catch (e) { + handleStreamError(e); + } + } +} +function writeFromReadableStream(stream, writable) { + if (stream.locked) throw new TypeError("ReadableStream is locked."); + else if (writable.destroyed) return; + return writeFromReadableStreamDefaultReader(stream.getReader(), writable); +} +const buildOutgoingHttpHeaders = (headers, defaultContentType) => { + const res = {}; + if (!(headers instanceof Headers)) headers = new Headers(headers ?? void 0); + if (headers.has("set-cookie")) { + const cookies = []; + for (const [k, v] of headers) if (k === "set-cookie") cookies.push(v); + else res[k] = v; + if (cookies.length > 0) res["set-cookie"] = cookies; + } else for (const [k, v] of headers) res[k] = v; + if (defaultContentType) res["content-type"] ??= defaultContentType; + return res; +}; + +//#endregion +//#region src/listener.ts +const outgoingEnded = Symbol("outgoingEnded"); +const incomingDraining = Symbol("incomingDraining"); +const DRAIN_TIMEOUT_MS = 500; +const MAX_DRAIN_BYTES = 64 * 1024 * 1024; +const drainIncoming = (incoming) => { + const incomingWithDrainState = incoming; + if (incoming.destroyed || incomingWithDrainState[incomingDraining]) return; + incomingWithDrainState[incomingDraining] = true; + if (incoming instanceof node_http2.Http2ServerRequest) { + try { + incoming.stream?.close?.(node_http2.constants.NGHTTP2_NO_ERROR); + } catch {} + return; + } + let bytesRead = 0; + const cleanup = () => { + clearTimeout(timer); + incoming.off("data", onData); + incoming.off("end", cleanup); + incoming.off("error", cleanup); + }; + const forceClose = () => { + cleanup(); + const socket = incoming.socket; + if (socket && !socket.destroyed) { + if (typeof socket.destroySoon === "function") socket.destroySoon(); + else if (typeof socket.destroy === "function") socket.destroy(); + } + }; + const timer = setTimeout(forceClose, DRAIN_TIMEOUT_MS); + timer.unref?.(); + const onData = (chunk) => { + bytesRead += chunk.length; + if (bytesRead > MAX_DRAIN_BYTES) forceClose(); + }; + incoming.on("data", onData); + incoming.on("end", cleanup); + incoming.on("error", cleanup); + incoming.resume(); +}; +const makeCloseHandler = (req, incoming, outgoing, needsBodyCleanup) => () => { + if (incoming.errored) { + recordBodyBufferedBeforeDisconnect(incoming); + req[abortRequest](incoming.errored.toString()); + } else if (!outgoing.writableFinished || incoming instanceof node_http2.Http2ServerRequest && incoming.aborted) { + recordBodyBufferedBeforeDisconnect(incoming); + req[abortRequest]("Client connection prematurely closed."); + } + if (needsBodyCleanup && !incoming.readableEnded) setTimeout(() => { + if (!incoming.readableEnded) setTimeout(() => { + drainIncoming(incoming); + }); + }); +}; +const isImmediateCacheableResponse = (res) => { + if (!(cacheKey in res)) return false; + const body = res[cacheKey][1]; + return body === null || typeof body === "string" || body instanceof Uint8Array; +}; +const handleRequestError = () => new Response(null, { status: 400 }); +const handleFetchError = (e) => new Response(null, { status: e instanceof Error && (e.name === "TimeoutError" || e.constructor.name === "TimeoutError") ? 504 : 500 }); +const handleResponseError = (e, outgoing) => { + const err = e instanceof Error ? e : new Error("unknown error", { cause: e }); + if (err.code === "ERR_STREAM_PREMATURE_CLOSE") console.info("The user aborted a request."); + else { + console.error(e); + if (!outgoing.headersSent) { + if (outgoing instanceof node_http.ServerResponse) outgoing._contentLength = null; + outgoing.writeHead(500, { "Content-Type": "text/plain" }); + } + outgoing.end(`Error: ${err.message}`); + outgoing.destroy(err); + } +}; +const flushHeaders = (outgoing) => { + if ("flushHeaders" in outgoing && outgoing.writable) outgoing.flushHeaders(); +}; +const trySetContentLength = (outgoing, status, length) => { + const http1 = outgoing; + if (http1._contentLength === null && http1._hasBody && http1.useChunkedEncodingByDefault && !http1._removedContLen && status >= 200 && status !== 204 && status !== 304 && !outgoing.hasHeader("content-length") && !outgoing.hasHeader("transfer-encoding") && !outgoing.hasHeader("trailer")) { + http1._contentLength = length; + return true; + } + return false; +}; +const writeDefaultHeaders = (outgoing, status, length) => { + if (trySetContentLength(outgoing, status, length)) outgoing.writeHead(status, { "Content-Type": defaultContentType }); + else outgoing.writeHead(status, { + "Content-Type": defaultContentType, + "Content-Length": length + }); +}; +const responseViaCache = async (res, outgoing) => { + let [status, body, header] = res[cacheKey]; + if (!header) { + if (body === null) { + outgoing.writeHead(status); + outgoing.end(); + } else if (typeof body === "string") { + writeDefaultHeaders(outgoing, status, Buffer.byteLength(body)); + outgoing.end(body); + } else if (body instanceof Uint8Array) { + writeDefaultHeaders(outgoing, status, body.byteLength); + outgoing.end(body); + } else if (body instanceof Blob) { + writeDefaultHeaders(outgoing, status, body.size); + outgoing.end(new Uint8Array(await body.arrayBuffer())); + } else { + outgoing.writeHead(status, { "Content-Type": defaultContentType }); + flushHeaders(outgoing); + await writeFromReadableStream(body, outgoing)?.catch((e) => handleResponseError(e, outgoing)); + } + outgoing[outgoingEnded]?.(); + return; + } + let hasContentLength = false; + let plainHeaders = false; + let canAutoLength = true; + if (header instanceof Headers) { + hasContentLength = header.has("content-length"); + header = buildOutgoingHttpHeaders(header, body === null ? void 0 : defaultContentType); + } else if (Array.isArray(header)) { + const headerObj = new Headers(header); + hasContentLength = headerObj.has("content-length"); + header = buildOutgoingHttpHeaders(headerObj, body === null ? void 0 : defaultContentType); + } else { + plainHeaders = true; + for (const key in header) { + if (key.length === 14 && key.toLowerCase() === "content-length") { + hasContentLength = true; + break; + } + if (key.length === 17 && key.toLowerCase() === "transfer-encoding" || key.length === 7 && key.toLowerCase() === "trailer") canAutoLength = false; + } + } + if (!hasContentLength) { + let length; + if (typeof body === "string") length = Buffer.byteLength(body); + else if (body instanceof Uint8Array) length = body.byteLength; + else if (body instanceof Blob) length = body.size; + if (length !== void 0 && (!plainHeaders || !canAutoLength || !trySetContentLength(outgoing, status, length))) { + if (plainHeaders) header = { ...header }; + header["Content-Length"] = length; + } + } + outgoing.writeHead(status, header); + if (body == null) outgoing.end(); + else if (typeof body === "string" || body instanceof Uint8Array) outgoing.end(body); + else if (body instanceof Blob) outgoing.end(new Uint8Array(await body.arrayBuffer())); + else { + flushHeaders(outgoing); + await writeFromReadableStream(body, outgoing)?.catch((e) => handleResponseError(e, outgoing)); + } + outgoing[outgoingEnded]?.(); +}; +const isPromise = (res) => typeof res.then === "function"; +const responseViaResponseObject = async (res, outgoing, options = {}) => { + if (isPromise(res)) if (options.errorHandler) try { + res = await res; + } catch (err) { + const errRes = await options.errorHandler(err); + if (!errRes) return; + res = errRes; + } + else res = await res.catch(handleFetchError); + if (cacheKey in res) return responseViaCache(res, outgoing); + const resHeaderRecord = buildOutgoingHttpHeaders(res.headers, res.body === null ? void 0 : defaultContentType); + if (res.body) { + const reader = res.body.getReader(); + const values = []; + let done = false; + let currentReadPromise = void 0; + if (resHeaderRecord["transfer-encoding"] !== "chunked") { + let maxReadCount = 2; + for (let i = 0; i < maxReadCount; i++) { + currentReadPromise ||= reader.read(); + const chunk = await readWithoutBlocking(currentReadPromise).catch((e) => { + console.error(e); + done = true; + }); + if (!chunk) { + if (i === 1) { + await new Promise((resolve) => setTimeout(resolve)); + maxReadCount = 3; + continue; + } + break; + } + currentReadPromise = void 0; + if (chunk.value) values.push(chunk.value); + if (chunk.done) { + done = true; + break; + } + } + if (done && !("content-length" in resHeaderRecord)) resHeaderRecord["content-length"] = values.reduce((acc, value) => acc + value.length, 0); + } + outgoing.writeHead(res.status, resHeaderRecord); + values.forEach((value) => { + outgoing.write(value); + }); + if (done) outgoing.end(); + else { + if (values.length === 0) flushHeaders(outgoing); + await writeFromReadableStreamDefaultReader(reader, outgoing, currentReadPromise); + } + } else if (resHeaderRecord[require_constants.X_ALREADY_SENT]) {} else { + outgoing.writeHead(res.status, resHeaderRecord); + outgoing.end(); + } + outgoing[outgoingEnded]?.(); +}; +const getRequestListener = (fetchCallback, options = {}) => { + const autoCleanupIncoming = options.autoCleanupIncoming ?? true; + if (options.overrideGlobalObjects !== false && global.Request !== Request$1) { + Object.defineProperty(global, "Request", { value: Request$1 }); + Object.defineProperty(global, "Response", { value: Response$1 }); + } + return async (incoming, outgoing) => { + let res, req; + let needsBodyCleanup = false; + let closeHandlerAttached = false; + const ensureCloseHandler = () => { + if (!req || closeHandlerAttached) return; + closeHandlerAttached = true; + outgoing.on("close", makeCloseHandler(req, incoming, outgoing, needsBodyCleanup)); + }; + try { + req = newRequest(incoming, options.hostname); + needsBodyCleanup = autoCleanupIncoming && !(incoming.method === "GET" || incoming.method === "HEAD"); + if (needsBodyCleanup) { + incoming[wrapBodyStream] = true; + if (incoming instanceof node_http2.Http2ServerRequest) outgoing[outgoingEnded] = () => { + if (!incoming.readableEnded) setTimeout(() => { + if (!incoming.readableEnded) setTimeout(() => { + incoming.destroy(); + outgoing.destroy(); + }); + }); + }; + } + res = fetchCallback(req, { + incoming, + outgoing + }); + if (!isPromise(res) && isImmediateCacheableResponse(res)) { + if (needsBodyCleanup && !incoming.readableEnded) outgoing.once("finish", () => { + if (!incoming.readableEnded) drainIncoming(incoming); + }); + return responseViaCache(res, outgoing); + } + ensureCloseHandler(); + } catch (e) { + if (!res) if (options.errorHandler) { + ensureCloseHandler(); + res = await options.errorHandler(req ? e : toRequestError(e)); + if (!res) return; + } else if (!req) res = handleRequestError(); + else res = handleFetchError(e); + else return handleResponseError(e, outgoing); + } + try { + return await responseViaResponseObject(res, outgoing, options); + } catch (e) { + return handleResponseError(e, outgoing); + } + }; +}; + +//#endregion +//#region src/websocket.ts +/** +* @link https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent +*/ +const CloseEvent = globalThis.CloseEvent ?? class extends Event { + #eventInitDict; + constructor(type, eventInitDict = {}) { + super(type, eventInitDict); + this.#eventInitDict = eventInitDict; + } + get wasClean() { + return this.#eventInitDict.wasClean ?? false; + } + get code() { + return this.#eventInitDict.code ?? 0; + } + get reason() { + return this.#eventInitDict.reason ?? ""; + } +}; +/** +* Node.js has no global `ErrorEvent`, unlike `Event`/`MessageEvent`/`CloseEvent`. +* @link https://developer.mozilla.org/en-US/docs/Web/API/ErrorEvent +*/ +const ErrorEvent = globalThis.ErrorEvent ?? class extends Event { + #eventInitDict; + constructor(type, eventInitDict = {}) { + super(type, eventInitDict); + this.#eventInitDict = eventInitDict; + } + get message() { + return this.#eventInitDict.message ?? ""; + } + get filename() { + return this.#eventInitDict.filename ?? ""; + } + get lineno() { + return this.#eventInitDict.lineno ?? 0; + } + get colno() { + return this.#eventInitDict.colno ?? 0; + } + get error() { + return this.#eventInitDict.error ?? null; + } +}; +const generateConnectionSymbol = () => Symbol("connection"); +const CONNECTION_SYMBOL_KEY = Symbol("CONNECTION_SYMBOL_KEY"); +const WAIT_FOR_WEBSOCKET_SYMBOL = Symbol("WAIT_FOR_WEBSOCKET_SYMBOL"); +const responseHeadersToSkip = new Set([ + "connection", + "content-length", + "keep-alive", + "proxy-authenticate", + "proxy-authorization", + "te", + "trailer", + "transfer-encoding", + "upgrade", + "sec-websocket-accept", + "sec-websocket-extensions", + "sec-websocket-protocol" +]); +const appendResponseHeaders = (headers, responseHeaders) => { + if (!responseHeaders) return; + responseHeaders.forEach((value, key) => { + if (responseHeadersToSkip.has(key.toLowerCase())) return; + headers.push(`${key}: ${value}`); + }); +}; +const rejectUpgradeRequest = (socket, status, responseHeaders) => { + const responseLines = ["Connection: close", "Content-Length: 0"]; + appendResponseHeaders(responseLines, responseHeaders); + socket.end(`HTTP/1.1 ${status.toString()} ${node_http.STATUS_CODES[status] ?? ""}\r\n${responseLines.join("\r\n")}\r\n\r +`); +}; +const createUpgradeRequest = (request) => { + const protocol = request.socket.encrypted ? "https" : "http"; + const url = new URL(request.url ?? "/", `${protocol}://${request.headers.host ?? "localhost"}`); + const headers = new Headers(); + for (const key in request.headers) { + const value = request.headers[key]; + if (!value) continue; + headers.append(key, Array.isArray(value) ? value[0] : value); + } + return new Request(url, { headers }); +}; +const setupWebSocket = (options) => { + const { server, fetchCallback, wss } = options; + const waiterMap = /* @__PURE__ */ new Map(); + wss.on("connection", (ws, request) => { + const waiter = waiterMap.get(request); + if (waiter) { + waiter.resolve(ws); + waiterMap.delete(request); + } + }); + const rejectWaiter = (request) => { + const waiter = waiterMap.get(request); + if (waiter) { + waiterMap.delete(request); + waiter.reject(/* @__PURE__ */ new Error("WebSocket handshake aborted")); + } + }; + const waitForWebSocket = (request, connectionSymbol) => { + return new Promise((resolve, reject) => { + waiterMap.set(request, { + resolve, + reject, + connectionSymbol + }); + }); + }; + server.on("upgrade", async (request, socket, head) => { + if (request.headers.upgrade?.toLowerCase() !== "websocket") return; + const env = { + incoming: request, + outgoing: void 0, + wss, + [WAIT_FOR_WEBSOCKET_SYMBOL]: waitForWebSocket + }; + let status = 400; + let responseHeaders; + try { + const response = await fetchCallback(createUpgradeRequest(request), env); + if (response instanceof Response) { + status = response.status; + responseHeaders = response.headers; + } + } catch { + if (server.listenerCount("upgrade") === 1) rejectUpgradeRequest(socket, 500); + return; + } + const waiter = waiterMap.get(request); + if (!waiter || waiter.connectionSymbol !== env[CONNECTION_SYMBOL_KEY]) { + rejectWaiter(request); + if (server.listenerCount("upgrade") === 1) rejectUpgradeRequest(socket, status, responseHeaders); + return; + } + const addResponseHeaders = (headers) => { + appendResponseHeaders(headers, responseHeaders); + }; + const reclaimWaiterOnClose = () => rejectWaiter(request); + socket.once("close", reclaimWaiterOnClose); + wss.on("headers", addResponseHeaders); + try { + wss.handleUpgrade(request, socket, head, (ws) => { + socket.off("close", reclaimWaiterOnClose); + wss.emit("connection", ws, request); + }); + } finally { + wss.off("headers", addResponseHeaders); + } + }); + server.on("close", () => { + wss.close(); + }); +}; +const upgradeWebSocket = (0, hono_ws.defineWebSocketHelper)(async (c, events, options) => { + if (c.req.header("upgrade")?.toLowerCase() !== "websocket") return; + const env = c.env; + const waitForWebSocket = env[WAIT_FOR_WEBSOCKET_SYMBOL]; + if (!waitForWebSocket || !env.incoming) return new Response(null, { status: 500 }); + const connectionSymbol = generateConnectionSymbol(); + env[CONNECTION_SYMBOL_KEY] = connectionSymbol; + (async () => { + let ws; + try { + ws = await waitForWebSocket(env.incoming, connectionSymbol); + } catch { + return; + } + const messagesReceivedInStarting = []; + const bufferMessage = (data, isBinary) => { + messagesReceivedInStarting.push([data, isBinary]); + }; + ws.on("message", bufferMessage); + const ctx = { + binaryType: "arraybuffer", + close(code, reason) { + ws.close(code, reason); + }, + protocol: ws.protocol, + raw: ws, + get readyState() { + return ws.readyState; + }, + send(source, opts) { + ws.send(source, { compress: opts?.compress }); + }, + url: new URL(c.req.url) + }; + try { + events?.onOpen?.(new Event("open"), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + const handleMessage = (data, isBinary) => { + const datas = Array.isArray(data) ? data : [data]; + for (const data of datas) try { + events?.onMessage?.(new MessageEvent("message", { data: isBinary ? data instanceof ArrayBuffer ? data : data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength) : typeof data === "string" ? data : Buffer.from(data).toString("utf-8") }), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + }; + ws.off("message", bufferMessage); + for (const message of messagesReceivedInStarting) handleMessage(...message); + ws.on("message", (data, isBinary) => { + handleMessage(data, isBinary); + }); + ws.on("close", (code, reason) => { + try { + events?.onClose?.(new CloseEvent("close", { + code, + reason: reason.toString() + }), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + }); + ws.on("error", (error) => { + try { + events?.onError?.(new ErrorEvent("error", { error }), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + }); + })(); + return new Response(); +}); + +//#endregion +//#region src/server.ts +const createAdaptorServer = (options) => { + const fetchCallback = options.fetch; + const requestListener = getRequestListener(fetchCallback, { + hostname: options.hostname, + overrideGlobalObjects: options.overrideGlobalObjects, + autoCleanupIncoming: options.autoCleanupIncoming + }); + const server = (options.createServer || node_http.createServer)(options.serverOptions || {}, requestListener); + if (options.websocket && options.websocket.server) { + if (options.websocket.server.options.noServer !== true) throw new Error("WebSocket server must be created with { noServer: true } option"); + setupWebSocket({ + server, + fetchCallback, + wss: options.websocket.server + }); + } + return server; +}; +const serve = (options, listeningListener) => { + const server = createAdaptorServer(options); + server.listen(options?.port ?? 3e3, options.hostname, () => { + const serverInfo = server.address(); + listeningListener && listeningListener(serverInfo); + }); + return server; +}; + +//#endregion +exports.RequestError = RequestError; +exports.createAdaptorServer = createAdaptorServer; +exports.getRequestListener = getRequestListener; +exports.serve = serve; +exports.upgradeWebSocket = upgradeWebSocket; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.d.cts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.d.cts new file mode 100644 index 000000000..d3c55ce99 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.d.cts @@ -0,0 +1,101 @@ +import { AddressInfo } from "node:net"; +import { IncomingMessage, Server, ServerOptions, ServerResponse, createServer } from "node:http"; +import { Http2SecureServer, Http2Server, Http2ServerRequest, Http2ServerResponse, SecureServerOptions, ServerOptions as ServerOptions$1, createSecureServer, createServer as createServer$1 } from "node:http2"; +import { ServerOptions as ServerOptions$2, createServer as createServer$2 } from "node:https"; +import { Duplex } from "node:stream"; +import { UpgradeWebSocket } from "hono/ws"; + +//#region src/websocket-types.d.ts +type WSReadyState = 0 | 1 | 2 | 3; +type WebSocketData = string | ArrayBuffer | Uint8Array | readonly Uint8Array[]; +type WebSocketSendOptions = { + compress?: boolean; +}; +interface WebSocketLike { + protocol: string; + readyState: WSReadyState; + close(code?: number, reason?: string): void; + send(data: string | ArrayBuffer | ArrayBufferView, options?: WebSocketSendOptions): void; + on(event: 'message', listener: (data: WebSocketData, isBinary: boolean) => void): this; + on(event: 'close', listener: (code: number, reason: Uint8Array) => void): this; + on(event: 'error', listener: (error: unknown) => void): this; + off(event: 'message', listener: (data: WebSocketData, isBinary: boolean) => void): this; +} +interface WebSocketServerLike { + options: { + noServer?: boolean; + }; + on(event: 'connection', listener: (ws: WebSocketLike, request: IncomingMessage) => void): this; + on(event: 'headers', listener: (headers: string[]) => void): this; + off(event: 'headers', listener: (headers: string[]) => void): this; + emit(event: 'connection', ws: WebSocketLike, request: IncomingMessage): boolean; + handleUpgrade(request: IncomingMessage, socket: Duplex, head: Buffer, callback: (ws: WebSocketLike) => void): void; + close(): void; +} +//#endregion +//#region src/types.d.ts +type HttpBindings = { + incoming: IncomingMessage; + outgoing: ServerResponse; +}; +type Http2Bindings = { + incoming: Http2ServerRequest; + outgoing: Http2ServerResponse; +}; +type FetchCallback = (request: Request, env: HttpBindings | Http2Bindings) => Promise | unknown; +type ServerType = Server | Http2Server | Http2SecureServer; +type createHttpOptions = { + serverOptions?: ServerOptions; + createServer?: typeof createServer; +}; +type createHttpsOptions = { + serverOptions?: ServerOptions$2; + createServer?: typeof createServer$2; +}; +type createHttp2Options = { + serverOptions?: ServerOptions$1; + createServer?: typeof createServer$1; +}; +type createSecureHttp2Options = { + serverOptions?: SecureServerOptions; + createServer?: typeof createSecureServer; +}; +type ServerOptions$3 = createHttpOptions | createHttpsOptions | createHttp2Options | createSecureHttp2Options; +type Options = { + fetch: FetchCallback; + overrideGlobalObjects?: boolean; + autoCleanupIncoming?: boolean; + port?: number; + hostname?: string; + websocket?: { + server: WebSocketServerLike; + }; +} & ServerOptions$3; +type CustomErrorHandler = (err: unknown) => void | Response | Promise; +//#endregion +//#region src/server.d.ts +declare const createAdaptorServer: (options: Options) => ServerType; +declare const serve: (options: Options, listeningListener?: (info: AddressInfo) => void) => ServerType; +//#endregion +//#region src/websocket.d.ts +type UpgradeWebSocketOptions = { + onError: (err: unknown) => void; +}; +declare const upgradeWebSocket: UpgradeWebSocket; +//#endregion +//#region src/listener.d.ts +declare const getRequestListener: (fetchCallback: FetchCallback, options?: { + hostname?: string; + errorHandler?: CustomErrorHandler; + overrideGlobalObjects?: boolean; + autoCleanupIncoming?: boolean; +}) => (incoming: IncomingMessage | Http2ServerRequest, outgoing: ServerResponse | Http2ServerResponse) => Promise; +//#endregion +//#region src/error.d.ts +declare class RequestError extends Error { + constructor(message: string, options?: { + cause?: unknown; + }); +} +//#endregion +export { type Http2Bindings, type HttpBindings, RequestError, type ServerType, type WebSocketData, type WebSocketLike, type WebSocketServerLike, createAdaptorServer, getRequestListener, serve, upgradeWebSocket }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.d.mts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.d.mts new file mode 100644 index 000000000..a815b86f0 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.d.mts @@ -0,0 +1,101 @@ +import { IncomingMessage, Server, ServerOptions, ServerResponse, createServer } from "node:http"; +import { Http2SecureServer, Http2Server, Http2ServerRequest, Http2ServerResponse, SecureServerOptions, ServerOptions as ServerOptions$1, createSecureServer, createServer as createServer$1 } from "node:http2"; +import { Duplex } from "node:stream"; +import { UpgradeWebSocket } from "hono/ws"; +import { AddressInfo } from "node:net"; +import { ServerOptions as ServerOptions$2, createServer as createServer$2 } from "node:https"; + +//#region src/websocket-types.d.ts +type WSReadyState = 0 | 1 | 2 | 3; +type WebSocketData = string | ArrayBuffer | Uint8Array | readonly Uint8Array[]; +type WebSocketSendOptions = { + compress?: boolean; +}; +interface WebSocketLike { + protocol: string; + readyState: WSReadyState; + close(code?: number, reason?: string): void; + send(data: string | ArrayBuffer | ArrayBufferView, options?: WebSocketSendOptions): void; + on(event: 'message', listener: (data: WebSocketData, isBinary: boolean) => void): this; + on(event: 'close', listener: (code: number, reason: Uint8Array) => void): this; + on(event: 'error', listener: (error: unknown) => void): this; + off(event: 'message', listener: (data: WebSocketData, isBinary: boolean) => void): this; +} +interface WebSocketServerLike { + options: { + noServer?: boolean; + }; + on(event: 'connection', listener: (ws: WebSocketLike, request: IncomingMessage) => void): this; + on(event: 'headers', listener: (headers: string[]) => void): this; + off(event: 'headers', listener: (headers: string[]) => void): this; + emit(event: 'connection', ws: WebSocketLike, request: IncomingMessage): boolean; + handleUpgrade(request: IncomingMessage, socket: Duplex, head: Buffer, callback: (ws: WebSocketLike) => void): void; + close(): void; +} +//#endregion +//#region src/types.d.ts +type HttpBindings = { + incoming: IncomingMessage; + outgoing: ServerResponse; +}; +type Http2Bindings = { + incoming: Http2ServerRequest; + outgoing: Http2ServerResponse; +}; +type FetchCallback = (request: Request, env: HttpBindings | Http2Bindings) => Promise | unknown; +type ServerType = Server | Http2Server | Http2SecureServer; +type createHttpOptions = { + serverOptions?: ServerOptions; + createServer?: typeof createServer; +}; +type createHttpsOptions = { + serverOptions?: ServerOptions$2; + createServer?: typeof createServer$2; +}; +type createHttp2Options = { + serverOptions?: ServerOptions$1; + createServer?: typeof createServer$1; +}; +type createSecureHttp2Options = { + serverOptions?: SecureServerOptions; + createServer?: typeof createSecureServer; +}; +type ServerOptions$3 = createHttpOptions | createHttpsOptions | createHttp2Options | createSecureHttp2Options; +type Options = { + fetch: FetchCallback; + overrideGlobalObjects?: boolean; + autoCleanupIncoming?: boolean; + port?: number; + hostname?: string; + websocket?: { + server: WebSocketServerLike; + }; +} & ServerOptions$3; +type CustomErrorHandler = (err: unknown) => void | Response | Promise; +//#endregion +//#region src/server.d.ts +declare const createAdaptorServer: (options: Options) => ServerType; +declare const serve: (options: Options, listeningListener?: (info: AddressInfo) => void) => ServerType; +//#endregion +//#region src/websocket.d.ts +type UpgradeWebSocketOptions = { + onError: (err: unknown) => void; +}; +declare const upgradeWebSocket: UpgradeWebSocket; +//#endregion +//#region src/listener.d.ts +declare const getRequestListener: (fetchCallback: FetchCallback, options?: { + hostname?: string; + errorHandler?: CustomErrorHandler; + overrideGlobalObjects?: boolean; + autoCleanupIncoming?: boolean; +}) => (incoming: IncomingMessage | Http2ServerRequest, outgoing: ServerResponse | Http2ServerResponse) => Promise; +//#endregion +//#region src/error.d.ts +declare class RequestError extends Error { + constructor(message: string, options?: { + cause?: unknown; + }); +} +//#endregion +export { type Http2Bindings, type HttpBindings, RequestError, type ServerType, type WebSocketData, type WebSocketLike, type WebSocketServerLike, createAdaptorServer, getRequestListener, serve, upgradeWebSocket }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.mjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.mjs new file mode 100644 index 000000000..4f5954d70 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/index.mjs @@ -0,0 +1,1335 @@ +import { t as X_ALREADY_SENT } from "./constants-BLSFu_RU.mjs"; +import { STATUS_CODES, ServerResponse, createServer } from "node:http"; +import { Http2ServerRequest, constants } from "node:http2"; +import { Readable } from "node:stream"; +import { defineWebSocketHelper } from "hono/ws"; + +//#region src/error.ts +var RequestError = class extends Error { + constructor(message, options) { + super(message, options); + this.name = "RequestError"; + } +}; + +//#endregion +//#region src/headers.ts +const nonJoinedHeaders = new Set([ + "age", + "authorization", + "content-length", + "content-type", + "etag", + "expires", + "from", + "host", + "if-modified-since", + "if-unmodified-since", + "last-modified", + "location", + "max-forwards", + "proxy-authorization", + "referer", + "retry-after", + "server", + "user-agent" +]); +const validHeaderName = /^[!#$%&'*+\-.^_`|~\dA-Za-z]+$/; +const isHttpWhitespace = (code) => code === 9 || code === 10 || code === 13 || code === 32; +const normalizeHeaderValue = (value) => { + if (!isHttpWhitespace(value.charCodeAt(0)) && !isHttpWhitespace(value.charCodeAt(value.length - 1))) return value; + let start = 0; + let end = value.length; + while (start < end && isHttpWhitespace(value.charCodeAt(start))) start++; + while (end > start && isHttpWhitespace(value.charCodeAt(end - 1))) end--; + return value.slice(start, end); +}; +const forbiddenHeaderValue = /[\0\r\n]/; +const GlobalHeaders = globalThis.Headers; +const materializeHeaders = (rawHeaders, HeadersCtor = GlobalHeaders) => { + const headers = new HeadersCtor(); + for (let i = 0; i < rawHeaders.length; i += 2) { + const name = rawHeaders[i]; + if (!name.startsWith(":")) headers.append(name, rawHeaders[i + 1]); + } + return headers; +}; +var RequestHeaders = class { + #incoming; + #rawHeaders; + #headers; + #invalidValue; + constructor(incoming) { + this.#incoming = incoming; + if (incoming instanceof Http2ServerRequest) this.#rawHeaders = incoming.rawHeaders.slice(); + } + get #lazyRawHeaders() { + return this.#rawHeaders ??= this.#incoming.rawHeaders.slice(); + } + get #native() { + if (!this.#headers) { + this.#headers = materializeHeaders(this.#lazyRawHeaders); + this.#rawHeaders = void 0; + } + return this.#headers; + } + #normalizedName(name) { + if (typeof name !== "string") return; + if (!validHeaderName.test(name)) throw new TypeError(`Invalid header name: ${name}`); + return name.toLowerCase(); + } + #lookupHttp1(lowerName) { + const headers = this.#incoming instanceof Http2ServerRequest ? void 0 : this.#incoming.headers; + if (!headers || nonJoinedHeaders.has(lowerName) || lowerName === "set-cookie" || lowerName === "__proto__") return; + if (!Object.hasOwn(headers, lowerName)) return null; + const rawValue = headers[lowerName]; + if (typeof rawValue === "string") { + const value = normalizeHeaderValue(rawValue); + return forbiddenHeaderValue.test(value) ? void 0 : value; + } + } + #lookup(rawHeaders, lowerName) { + const separator = lowerName === "cookie" ? "; " : ", "; + let value = null; + for (let i = 0; i < rawHeaders.length; i += 2) { + const rawName = rawHeaders[i]; + if (rawName.length === lowerName.length && rawName.toLowerCase() === lowerName) { + const rawValue = normalizeHeaderValue(rawHeaders[i + 1]); + if (forbiddenHeaderValue.test(rawValue)) { + this.#invalidValue = true; + return; + } + value = value === null ? rawValue : value + separator + rawValue; + } + } + return value; + } + append(name, value) { + this.#native.append(name, value); + } + delete(name) { + this.#native.delete(name); + } + get(name) { + const lowerName = this.#normalizedName(name); + if (lowerName && !this.#headers && !this.#invalidValue) { + const http1Value = this.#lookupHttp1(lowerName); + if (http1Value !== void 0) return http1Value; + const value = this.#lookup(this.#lazyRawHeaders, lowerName); + if (value !== void 0) return value; + } + return this.#native.get(name); + } + has(name) { + const lowerName = this.#normalizedName(name); + if (lowerName && !this.#headers && !this.#invalidValue) { + const http1Value = this.#lookupHttp1(lowerName); + if (http1Value !== void 0) return http1Value !== null; + const value = this.#lookup(this.#lazyRawHeaders, lowerName); + if (value !== void 0) return value !== null; + } + return this.#native.has(name); + } + set(name, value) { + this.#native.set(name, value); + } + getSetCookie() { + return this.#native.getSetCookie(); + } + keys() { + return this.#native.keys(); + } + values() { + return this.#native.values(); + } + entries() { + return this.#native.entries(); + } + forEach(callback, thisArg) { + this.#native.forEach((value, key) => { + callback.call(thisArg, value, key, this); + }); + } + [Symbol.iterator]() { + return this.entries(); + } +}; +Object.defineProperty(RequestHeaders.prototype, Symbol.for("nodejs.util.inspect.custom"), { value: function(depth, options, inspectFn) { + return `Headers (lightweight) ${inspectFn(Object.fromEntries(this), { + ...options, + depth: depth == null ? null : depth - 1 + })}`; +} }); +Object.setPrototypeOf(RequestHeaders.prototype, GlobalHeaders.prototype); +const newHeadersFromIncoming = (incoming) => globalThis.Headers === GlobalHeaders ? new RequestHeaders(incoming) : materializeHeaders(incoming.rawHeaders, globalThis.Headers); + +//#endregion +//#region src/url.ts +const reValidRequestUrl = /^\/[!#$&-;=?-\[\]_a-z~]*$/; +const reDotSegment = /\/\.\.?(?:[/?#]|$)/; +const reValidHost = /^[a-z0-9._-]+(?::(?:[1-5]\d{3,4}|[6-9]\d{3}))?$/; +const buildUrl = (scheme, host, incomingUrl) => { + const url = `${scheme}://${host}${incomingUrl}`; + if (!reValidHost.test(host)) { + const urlObj = new URL(url); + if (urlObj.hostname.length !== host.length && urlObj.hostname !== (host.includes(":") ? host.replace(/:\d+$/, "") : host).toLowerCase()) throw new RequestError("Invalid host header"); + return urlObj.href; + } else if (incomingUrl.length === 0) return url + "/"; + else { + if (incomingUrl.charCodeAt(0) !== 47) throw new RequestError("Invalid URL"); + if (!reValidRequestUrl.test(incomingUrl) || reDotSegment.test(incomingUrl)) return new URL(url).href; + return url; + } +}; + +//#endregion +//#region src/request.ts +const toRequestError = (e) => { + if (e instanceof RequestError) return e; + return new RequestError(e.message, { cause: e }); +}; +const GlobalRequest = global.Request; +var Request$1 = class extends GlobalRequest { + constructor(input, options) { + if (typeof input === "object" && getRequestCache in input) { + const hasReplacementBody = options !== void 0 && "body" in options && options.body != null; + if (input[bodyConsumedDirectlyKey] && !hasReplacementBody) throw new TypeError("Cannot construct a Request with a Request object that has already been used."); + input = input[getRequestCache](); + } + if (typeof (options?.body)?.getReader !== "undefined") options.duplex ??= "half"; + super(input, options); + } +}; +const wrapBodyStream = Symbol("wrapBodyStream"); +const byteExactEncodings = new Set([ + "latin1", + "binary", + "hex", + "base64", + "base64url" +]); +const isByteExactEncoding = (encoding) => encoding === null || byteExactEncodings.has(encoding); +const bodyBufferedBeforeDisconnectKey = Symbol("bodyBufferedBeforeDisconnect"); +const bodyBufferedLengthBeforeDisconnectKey = Symbol("bodyBufferedLengthBeforeDisconnect"); +const toBufferChunk = (chunk, encoding) => Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk, encoding ?? "utf8"); +const isRecoverableDisconnectedIncoming = (incoming) => !(incoming instanceof Http2ServerRequest) && !!incoming.complete && !!incoming.readableAborted && typeof incoming.read === "function" && isByteExactEncoding(incoming.readableEncoding); +const recordBodyBufferedBeforeDisconnect = (incoming) => { + if (incoming.readableDidRead || !isRecoverableDisconnectedIncoming(incoming)) return; + const incomingWithRecovery = incoming; + incomingWithRecovery[bodyBufferedLengthBeforeDisconnectKey] ??= incoming.readableLength; +}; +const readBodyBufferedBeforeDisconnect = (incoming, chunks) => { + if (incoming.readableDidRead && !chunks || !isRecoverableDisconnectedIncoming(incoming)) return; + const incomingWithRecovery = incoming; + if (incomingWithRecovery[bodyBufferedBeforeDisconnectKey] !== void 0) return incomingWithRecovery[bodyBufferedBeforeDisconnectKey]; + let result; + const errored = incoming.errored; + if (errored && errored.code !== "ECONNRESET") result = errored; + else if (incomingWithRecovery[bodyBufferedLengthBeforeDisconnectKey] !== void 0 && incoming.readableLength !== incomingWithRecovery[bodyBufferedLengthBeforeDisconnectKey]) result = newBodyUnusableError(); + else { + const bodyChunks = chunks ?? []; + const chunk = incoming.read(); + if (chunk !== null) bodyChunks.push(toBufferChunk(chunk, incoming.readableEncoding)); + const buffer = bodyChunks.length === 1 ? bodyChunks[0] : Buffer.concat(bodyChunks); + result = buffer; + const contentLength = incoming.headers["content-length"]; + if (typeof contentLength === "string" && /^\d+$/.test(contentLength)) { + const expectedLength = Number(contentLength); + if (Number.isSafeInteger(expectedLength) && buffer.length !== expectedLength) result = newBodyUnusableError(); + } + } + incomingWithRecovery[bodyBufferedBeforeDisconnectKey] = result; + return result; +}; +const enqueueBufferedBody = (controller, buffered) => { + if (buffered instanceof Error) { + controller.error(buffered); + return; + } + if (buffered.length > 0) controller.enqueue(buffered); + controller.close(); +}; +const newRequestFromIncoming = (method, url, headers, incoming, abortController) => { + const init = { + method, + headers, + signal: abortController.signal + }; + if (method === "TRACE") { + init.method = "GET"; + const req = new Request$1(url, init); + Object.defineProperty(req, "method", { get() { + return "TRACE"; + } }); + return req; + } + if (!(method === "GET" || method === "HEAD")) if ("rawBody" in incoming && incoming.rawBody instanceof Buffer) init.body = new ReadableStream({ start(controller) { + controller.enqueue(incoming.rawBody); + controller.close(); + } }); + else if (incoming[wrapBodyStream]) { + let reader; + init.body = new ReadableStream({ async pull(controller) { + try { + if (!reader) { + const buffered = readBodyBufferedBeforeDisconnect(incoming); + if (buffered !== void 0) { + enqueueBufferedBody(controller, buffered); + return; + } + } + reader ||= Readable.toWeb(incoming).getReader(); + const { done, value } = await reader.read(); + if (done) controller.close(); + else controller.enqueue(value); + } catch (error) { + controller.error(error); + } + } }); + } else { + const buffered = readBodyBufferedBeforeDisconnect(incoming); + if (buffered !== void 0) init.body = new ReadableStream({ start(controller) { + enqueueBufferedBody(controller, buffered); + } }); + else init.body = Readable.toWeb(incoming); + } + return new Request$1(url, init); +}; +const getRequestCache = Symbol("getRequestCache"); +const requestCache = Symbol("requestCache"); +const incomingKey = Symbol("incomingKey"); +const urlKey = Symbol("urlKey"); +const methodKey = Symbol("methodKey"); +const headersKey = Symbol("headersKey"); +const abortControllerKey = Symbol("abortControllerKey"); +const getAbortController = Symbol("getAbortController"); +const abortRequest = Symbol("abortRequest"); +const bodyBufferKey = Symbol("bodyBuffer"); +const bodyReadPromiseKey = Symbol("bodyReadPromise"); +const bodyConsumedDirectlyKey = Symbol("bodyConsumedDirectly"); +const bodyLockReaderKey = Symbol("bodyLockReader"); +const abortReasonKey = Symbol("abortReason"); +const newBodyUnusableError = () => { + return /* @__PURE__ */ new TypeError("Body is unusable"); +}; +const rejectBodyUnusable = () => { + return Promise.reject(newBodyUnusableError()); +}; +const textDecoder = new TextDecoder(); +const consumeBodyDirectOnce = (request) => { + if (request[bodyConsumedDirectlyKey]) return rejectBodyUnusable(); + request[bodyConsumedDirectlyKey] = true; +}; +const toArrayBuffer = (buf) => { + return buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength); +}; +const contentType = (request) => { + return (request[headersKey] ||= newHeadersFromIncoming(request[incomingKey])).get("content-type") || ""; +}; +const methodTokenRegExp = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/; +const normalizeIncomingMethod = (method) => { + if (typeof method !== "string" || method.length === 0) return "GET"; + switch (method) { + case "DELETE": + case "GET": + case "HEAD": + case "OPTIONS": + case "PATCH": + case "POST": + case "PUT": + case "QUERY": return method; + } + const upper = method.toUpperCase(); + switch (upper) { + case "DELETE": + case "GET": + case "HEAD": + case "OPTIONS": + case "POST": + case "PUT": return upper; + default: return method; + } +}; +const validateDirectReadMethod = (method) => { + if (!methodTokenRegExp.test(method)) return /* @__PURE__ */ new TypeError(`'${method}' is not a valid HTTP method.`); + const normalized = method.toUpperCase(); + if (normalized === "CONNECT" || normalized === "TRACK" || normalized === "TRACE" && method !== "TRACE") return /* @__PURE__ */ new TypeError(`'${method}' HTTP method is unsupported.`); +}; +const readBodyWithFastPath = (request, method, fromBuffer) => { + if (request[bodyConsumedDirectlyKey]) return rejectBodyUnusable(); + const methodName = request.method; + if (methodName === "GET" || methodName === "HEAD") return request[getRequestCache]()[method](); + const methodValidationError = validateDirectReadMethod(methodName); + if (methodValidationError) return Promise.reject(methodValidationError); + if (request[requestCache]) { + if (methodName !== "TRACE") return request[requestCache][method](); + } + const alreadyUsedError = consumeBodyDirectOnce(request); + if (alreadyUsedError) return alreadyUsedError; + const raw = readRawBodyIfAvailable(request); + if (raw) { + const result = Promise.resolve(fromBuffer(raw, request)); + request[bodyBufferKey] = void 0; + return result; + } + return readBodyDirect(request).then((buf) => { + const result = fromBuffer(buf, request); + request[bodyBufferKey] = void 0; + return result; + }); +}; +const readRawBodyIfAvailable = (request) => { + const incoming = request[incomingKey]; + if ("rawBody" in incoming && incoming.rawBody instanceof Buffer) return incoming.rawBody; +}; +const normalizeAbortError = (request, incoming) => { + if (incoming.errored) return incoming.errored; + const reason = request[abortReasonKey]; + if (reason !== void 0) return reason instanceof Error ? reason : new Error(String(reason)); + return /* @__PURE__ */ new Error("Client connection prematurely closed."); +}; +const readBodyDirect = (request) => { + if (request[bodyBufferKey]) return Promise.resolve(request[bodyBufferKey]); + if (request[bodyReadPromiseKey]) return request[bodyReadPromiseKey]; + const incoming = request[incomingKey]; + if (incoming.readableDidRead) return rejectBodyUnusable(); + const buffered = readBodyBufferedBeforeDisconnect(incoming); + if (buffered !== void 0) { + if (buffered instanceof Error) return Promise.reject(buffered); + request[bodyBufferKey] = buffered; + return Promise.resolve(buffered); + } + const promise = new Promise((resolve, reject) => { + const chunks = []; + let settled = false; + const finish = (callback) => { + if (settled) return; + settled = true; + cleanup(); + callback(); + }; + const recoverCompleteBodyAfterDisconnect = (error) => { + const streamError = incoming.errored ?? error; + if (!isRecoverableDisconnectedIncoming(incoming) || streamError && streamError.code !== "ECONNRESET") return false; + finish(() => { + const recovered = readBodyBufferedBeforeDisconnect(incoming, chunks); + if (recovered instanceof Error) reject(recovered); + else if (recovered === void 0) reject(error ?? normalizeAbortError(request, incoming)); + else { + request[bodyBufferKey] = recovered; + resolve(recovered); + } + }); + return true; + }; + const onData = (chunk) => { + chunks.push(toBufferChunk(chunk, incoming.readableEncoding)); + }; + const onEnd = () => { + finish(() => { + const buffer = chunks.length === 1 ? chunks[0] : Buffer.concat(chunks); + request[bodyBufferKey] = buffer; + resolve(buffer); + }); + }; + const onError = (error) => { + if (recoverCompleteBodyAfterDisconnect(error)) return; + finish(() => { + reject(error); + }); + }; + const onClose = () => { + if (incoming.readableEnded) { + onEnd(); + return; + } + if (recoverCompleteBodyAfterDisconnect()) return; + finish(() => { + reject(normalizeAbortError(request, incoming)); + }); + }; + const cleanup = () => { + incoming.off("data", onData); + incoming.off("end", onEnd); + incoming.off("error", onError); + incoming.off("close", onClose); + request[bodyReadPromiseKey] = void 0; + }; + incoming.on("data", onData); + incoming.on("end", onEnd); + incoming.on("error", onError); + incoming.on("close", onClose); + queueMicrotask(() => { + if (settled) return; + if (incoming.readableEnded) onEnd(); + else if (incoming.errored) onError(incoming.errored); + else if (incoming.destroyed) onClose(); + }); + }); + request[bodyReadPromiseKey] = promise; + return promise; +}; +const requestPrototype = { + get method() { + return this[methodKey]; + }, + get url() { + return this[urlKey]; + }, + get headers() { + return this[headersKey] ||= newHeadersFromIncoming(this[incomingKey]); + }, + [abortRequest](reason) { + if (this[abortReasonKey] === void 0) this[abortReasonKey] = reason; + const abortController = this[abortControllerKey]; + if (abortController && !abortController.signal.aborted) abortController.abort(reason); + }, + [getAbortController]() { + this[abortControllerKey] ||= new AbortController(); + if (this[abortReasonKey] !== void 0 && !this[abortControllerKey].signal.aborted) this[abortControllerKey].abort(this[abortReasonKey]); + return this[abortControllerKey]; + }, + [getRequestCache]() { + const abortController = this[getAbortController](); + if (this[requestCache]) return this[requestCache]; + const method = this.method; + if (this[bodyConsumedDirectlyKey] && !(method === "GET" || method === "HEAD")) { + this[bodyBufferKey] = void 0; + const init = { + method: method === "TRACE" ? "GET" : method, + headers: this.headers, + signal: abortController.signal + }; + if (method !== "TRACE") { + init.body = new ReadableStream({ start(c) { + c.close(); + } }); + init.duplex = "half"; + } + const req = new Request$1(this[urlKey], init); + if (method === "TRACE") Object.defineProperty(req, "method", { get() { + return "TRACE"; + } }); + return this[requestCache] = req; + } + return this[requestCache] = newRequestFromIncoming(this.method, this[urlKey], this.headers, this[incomingKey], abortController); + }, + get body() { + if (!this[bodyConsumedDirectlyKey]) return this[getRequestCache]().body; + const request = this[getRequestCache](); + if (!this[bodyLockReaderKey] && request.body) this[bodyLockReaderKey] = request.body.getReader(); + return request.body; + }, + get bodyUsed() { + if (this[bodyConsumedDirectlyKey]) return true; + if (this[requestCache]) return this[requestCache].bodyUsed; + return false; + } +}; +Object.defineProperty(requestPrototype, "signal", { get() { + return this[getAbortController]().signal; +} }); +[ + "cache", + "credentials", + "destination", + "integrity", + "mode", + "redirect", + "referrer", + "referrerPolicy", + "keepalive" +].forEach((k) => { + Object.defineProperty(requestPrototype, k, { get() { + return this[getRequestCache]()[k]; + } }); +}); +["clone", "formData"].forEach((k) => { + Object.defineProperty(requestPrototype, k, { value: function() { + if (this[bodyConsumedDirectlyKey]) { + if (k === "clone") throw newBodyUnusableError(); + return rejectBodyUnusable(); + } + return this[getRequestCache]()[k](); + } }); +}); +Object.defineProperty(requestPrototype, "text", { value: function() { + return readBodyWithFastPath(this, "text", (buf) => textDecoder.decode(buf)); +} }); +Object.defineProperty(requestPrototype, "arrayBuffer", { value: function() { + return readBodyWithFastPath(this, "arrayBuffer", (buf) => toArrayBuffer(buf)); +} }); +Object.defineProperty(requestPrototype, "blob", { value: function() { + return readBodyWithFastPath(this, "blob", (buf, request) => { + const type = contentType(request); + const init = type ? { headers: { "content-type": type } } : void 0; + return new Response(buf, init).blob(); + }); +} }); +Object.defineProperty(requestPrototype, "json", { value: function() { + if (this[bodyConsumedDirectlyKey]) return rejectBodyUnusable(); + return this.text().then(JSON.parse); +} }); +Object.defineProperty(requestPrototype, Symbol.for("nodejs.util.inspect.custom"), { value: function(depth, options, inspectFn) { + return `Request (lightweight) ${inspectFn({ + method: this.method, + url: this.url, + headers: this.headers, + nativeRequest: this[requestCache] + }, { + ...options, + depth: depth == null ? null : depth - 1 + })}`; +} }); +Object.setPrototypeOf(requestPrototype, Request$1.prototype); +const newRequest = (incoming, defaultHostname) => { + const req = Object.create(requestPrototype); + req[incomingKey] = incoming; + req[methodKey] = normalizeIncomingMethod(incoming.method); + const incomingUrl = incoming.url || ""; + if (incomingUrl[0] !== "/" && (incomingUrl.startsWith("http://") || incomingUrl.startsWith("https://"))) { + if (incoming instanceof Http2ServerRequest) throw new RequestError("Absolute URL for :path is not allowed in HTTP/2"); + try { + req[urlKey] = new URL(incomingUrl).href; + } catch (e) { + throw new RequestError("Invalid absolute URL", { cause: e }); + } + return req; + } + const host = (incoming instanceof Http2ServerRequest ? incoming.authority : incoming.headers.host) || defaultHostname; + if (!host) throw new RequestError("Missing host header"); + let scheme; + if (incoming instanceof Http2ServerRequest) { + scheme = incoming.scheme; + if (!(scheme === "http" || scheme === "https")) throw new RequestError("Unsupported scheme"); + } else scheme = incoming.socket && incoming.socket.encrypted ? "https" : "http"; + try { + req[urlKey] = buildUrl(scheme, host, incomingUrl); + } catch (e) { + if (e instanceof RequestError) throw e; + else throw new RequestError("Invalid URL", { cause: e }); + } + return req; +}; + +//#endregion +//#region src/response.ts +const defaultContentType = "text/plain; charset=UTF-8"; +const responseCache = Symbol("responseCache"); +const getResponseCache = Symbol("getResponseCache"); +const cacheKey = Symbol("cache"); +const GlobalResponse = global.Response; +var Response$1 = class Response$1 { + #body; + #init; + [getResponseCache]() { + const cache = this[cacheKey]; + const liveHeaders = cache && cache[2] instanceof Headers ? cache[2] : void 0; + delete this[cacheKey]; + return this[responseCache] ||= new GlobalResponse(this.#body, liveHeaders ? { + status: this.#init?.status, + statusText: this.#init?.statusText, + headers: liveHeaders + } : this.#init); + } + constructor(body, init) { + let headers; + this.#body = body; + if (init instanceof GlobalResponse) { + const cachedGlobalResponse = init[responseCache]; + if (cachedGlobalResponse) { + this.#init = cachedGlobalResponse; + this[getResponseCache](); + return; + } + this.#init = init instanceof Response$1 ? init.#init : init; + headers = new Headers(init.headers); + } else this.#init = init; + if (body == null || typeof body === "string" || typeof body?.getReader !== "undefined" || body instanceof Blob || body instanceof Uint8Array) this[cacheKey] = [ + init?.status || 200, + body ?? null, + headers || init?.headers + ]; + } + get headers() { + const cache = this[cacheKey]; + if (cache) { + if (!(cache[2] instanceof Headers)) cache[2] = new Headers(cache[2] || (cache[1] === null ? void 0 : { "content-type": defaultContentType })); + return cache[2]; + } + return this[getResponseCache]().headers; + } + get status() { + return this[cacheKey]?.[0] ?? this[getResponseCache]().status; + } + get ok() { + const status = this.status; + return status >= 200 && status < 300; + } +}; +[ + "body", + "bodyUsed", + "redirected", + "statusText", + "trailers", + "type", + "url" +].forEach((k) => { + Object.defineProperty(Response$1.prototype, k, { get() { + return this[getResponseCache]()[k]; + } }); +}); +[ + "arrayBuffer", + "blob", + "clone", + "formData", + "json", + "text" +].forEach((k) => { + Object.defineProperty(Response$1.prototype, k, { value: function() { + return this[getResponseCache]()[k](); + } }); +}); +Object.defineProperty(Response$1.prototype, Symbol.for("nodejs.util.inspect.custom"), { value: function(depth, options, inspectFn) { + return `Response (lightweight) ${inspectFn({ + status: this.status, + headers: this.headers, + ok: this.ok, + nativeResponse: this[responseCache] + }, { + ...options, + depth: depth == null ? null : depth - 1 + })}`; +} }); +Object.setPrototypeOf(Response$1, GlobalResponse); +Object.setPrototypeOf(Response$1.prototype, GlobalResponse.prototype); +const validRedirectUrl = /^https?:\/\/[!#-;=?-[\]_a-z~A-Z]+$/; +const parseRedirectUrl = (url) => { + if (url instanceof URL) return url.href; + if (validRedirectUrl.test(url)) return url; + return new URL(url).href; +}; +const validRedirectStatuses = new Set([ + 301, + 302, + 303, + 307, + 308 +]); +Object.defineProperty(Response$1, "redirect", { + value: function redirect(url, status = 302) { + if (!validRedirectStatuses.has(status)) throw new RangeError("Invalid status code"); + return new Response$1(null, { + status, + headers: { location: parseRedirectUrl(url) } + }); + }, + writable: true, + configurable: true +}); +Object.defineProperty(Response$1, "json", { + value: function json(data, init) { + const body = JSON.stringify(data); + if (body === void 0) throw new TypeError("The data is not JSON serializable"); + const initHeaders = init?.headers; + let headers; + if (initHeaders) { + headers = new Headers(initHeaders); + if (!headers.has("content-type")) headers.set("content-type", "application/json"); + } else headers = { "content-type": "application/json" }; + return new Response$1(body, { + status: init?.status ?? 200, + statusText: init?.statusText, + headers + }); + }, + writable: true, + configurable: true +}); + +//#endregion +//#region src/utils.ts +async function readWithoutBlocking(readPromise) { + return Promise.race([readPromise, Promise.resolve().then(() => Promise.resolve(void 0))]); +} +function writeFromReadableStreamDefaultReader(reader, writable, currentReadPromise) { + const cancel = (error) => { + reader.cancel(error).catch(() => {}); + }; + writable.on("close", cancel); + writable.on("error", cancel); + (currentReadPromise ?? reader.read()).then(flow, handleStreamError); + return reader.closed.finally(() => { + writable.off("close", cancel); + writable.off("error", cancel); + }); + function handleStreamError(error) { + if (error) writable.destroy(error); + } + function onDrain() { + reader.read().then(flow, handleStreamError); + } + function flow({ done, value }) { + try { + if (done) writable.end(); + else if (!writable.write(value)) writable.once("drain", onDrain); + else return reader.read().then(flow, handleStreamError); + } catch (e) { + handleStreamError(e); + } + } +} +function writeFromReadableStream(stream, writable) { + if (stream.locked) throw new TypeError("ReadableStream is locked."); + else if (writable.destroyed) return; + return writeFromReadableStreamDefaultReader(stream.getReader(), writable); +} +const buildOutgoingHttpHeaders = (headers, defaultContentType) => { + const res = {}; + if (!(headers instanceof Headers)) headers = new Headers(headers ?? void 0); + if (headers.has("set-cookie")) { + const cookies = []; + for (const [k, v] of headers) if (k === "set-cookie") cookies.push(v); + else res[k] = v; + if (cookies.length > 0) res["set-cookie"] = cookies; + } else for (const [k, v] of headers) res[k] = v; + if (defaultContentType) res["content-type"] ??= defaultContentType; + return res; +}; + +//#endregion +//#region src/listener.ts +const outgoingEnded = Symbol("outgoingEnded"); +const incomingDraining = Symbol("incomingDraining"); +const DRAIN_TIMEOUT_MS = 500; +const MAX_DRAIN_BYTES = 64 * 1024 * 1024; +const drainIncoming = (incoming) => { + const incomingWithDrainState = incoming; + if (incoming.destroyed || incomingWithDrainState[incomingDraining]) return; + incomingWithDrainState[incomingDraining] = true; + if (incoming instanceof Http2ServerRequest) { + try { + incoming.stream?.close?.(constants.NGHTTP2_NO_ERROR); + } catch {} + return; + } + let bytesRead = 0; + const cleanup = () => { + clearTimeout(timer); + incoming.off("data", onData); + incoming.off("end", cleanup); + incoming.off("error", cleanup); + }; + const forceClose = () => { + cleanup(); + const socket = incoming.socket; + if (socket && !socket.destroyed) { + if (typeof socket.destroySoon === "function") socket.destroySoon(); + else if (typeof socket.destroy === "function") socket.destroy(); + } + }; + const timer = setTimeout(forceClose, DRAIN_TIMEOUT_MS); + timer.unref?.(); + const onData = (chunk) => { + bytesRead += chunk.length; + if (bytesRead > MAX_DRAIN_BYTES) forceClose(); + }; + incoming.on("data", onData); + incoming.on("end", cleanup); + incoming.on("error", cleanup); + incoming.resume(); +}; +const makeCloseHandler = (req, incoming, outgoing, needsBodyCleanup) => () => { + if (incoming.errored) { + recordBodyBufferedBeforeDisconnect(incoming); + req[abortRequest](incoming.errored.toString()); + } else if (!outgoing.writableFinished || incoming instanceof Http2ServerRequest && incoming.aborted) { + recordBodyBufferedBeforeDisconnect(incoming); + req[abortRequest]("Client connection prematurely closed."); + } + if (needsBodyCleanup && !incoming.readableEnded) setTimeout(() => { + if (!incoming.readableEnded) setTimeout(() => { + drainIncoming(incoming); + }); + }); +}; +const isImmediateCacheableResponse = (res) => { + if (!(cacheKey in res)) return false; + const body = res[cacheKey][1]; + return body === null || typeof body === "string" || body instanceof Uint8Array; +}; +const handleRequestError = () => new Response(null, { status: 400 }); +const handleFetchError = (e) => new Response(null, { status: e instanceof Error && (e.name === "TimeoutError" || e.constructor.name === "TimeoutError") ? 504 : 500 }); +const handleResponseError = (e, outgoing) => { + const err = e instanceof Error ? e : new Error("unknown error", { cause: e }); + if (err.code === "ERR_STREAM_PREMATURE_CLOSE") console.info("The user aborted a request."); + else { + console.error(e); + if (!outgoing.headersSent) { + if (outgoing instanceof ServerResponse) outgoing._contentLength = null; + outgoing.writeHead(500, { "Content-Type": "text/plain" }); + } + outgoing.end(`Error: ${err.message}`); + outgoing.destroy(err); + } +}; +const flushHeaders = (outgoing) => { + if ("flushHeaders" in outgoing && outgoing.writable) outgoing.flushHeaders(); +}; +const trySetContentLength = (outgoing, status, length) => { + const http1 = outgoing; + if (http1._contentLength === null && http1._hasBody && http1.useChunkedEncodingByDefault && !http1._removedContLen && status >= 200 && status !== 204 && status !== 304 && !outgoing.hasHeader("content-length") && !outgoing.hasHeader("transfer-encoding") && !outgoing.hasHeader("trailer")) { + http1._contentLength = length; + return true; + } + return false; +}; +const writeDefaultHeaders = (outgoing, status, length) => { + if (trySetContentLength(outgoing, status, length)) outgoing.writeHead(status, { "Content-Type": defaultContentType }); + else outgoing.writeHead(status, { + "Content-Type": defaultContentType, + "Content-Length": length + }); +}; +const responseViaCache = async (res, outgoing) => { + let [status, body, header] = res[cacheKey]; + if (!header) { + if (body === null) { + outgoing.writeHead(status); + outgoing.end(); + } else if (typeof body === "string") { + writeDefaultHeaders(outgoing, status, Buffer.byteLength(body)); + outgoing.end(body); + } else if (body instanceof Uint8Array) { + writeDefaultHeaders(outgoing, status, body.byteLength); + outgoing.end(body); + } else if (body instanceof Blob) { + writeDefaultHeaders(outgoing, status, body.size); + outgoing.end(new Uint8Array(await body.arrayBuffer())); + } else { + outgoing.writeHead(status, { "Content-Type": defaultContentType }); + flushHeaders(outgoing); + await writeFromReadableStream(body, outgoing)?.catch((e) => handleResponseError(e, outgoing)); + } + outgoing[outgoingEnded]?.(); + return; + } + let hasContentLength = false; + let plainHeaders = false; + let canAutoLength = true; + if (header instanceof Headers) { + hasContentLength = header.has("content-length"); + header = buildOutgoingHttpHeaders(header, body === null ? void 0 : defaultContentType); + } else if (Array.isArray(header)) { + const headerObj = new Headers(header); + hasContentLength = headerObj.has("content-length"); + header = buildOutgoingHttpHeaders(headerObj, body === null ? void 0 : defaultContentType); + } else { + plainHeaders = true; + for (const key in header) { + if (key.length === 14 && key.toLowerCase() === "content-length") { + hasContentLength = true; + break; + } + if (key.length === 17 && key.toLowerCase() === "transfer-encoding" || key.length === 7 && key.toLowerCase() === "trailer") canAutoLength = false; + } + } + if (!hasContentLength) { + let length; + if (typeof body === "string") length = Buffer.byteLength(body); + else if (body instanceof Uint8Array) length = body.byteLength; + else if (body instanceof Blob) length = body.size; + if (length !== void 0 && (!plainHeaders || !canAutoLength || !trySetContentLength(outgoing, status, length))) { + if (plainHeaders) header = { ...header }; + header["Content-Length"] = length; + } + } + outgoing.writeHead(status, header); + if (body == null) outgoing.end(); + else if (typeof body === "string" || body instanceof Uint8Array) outgoing.end(body); + else if (body instanceof Blob) outgoing.end(new Uint8Array(await body.arrayBuffer())); + else { + flushHeaders(outgoing); + await writeFromReadableStream(body, outgoing)?.catch((e) => handleResponseError(e, outgoing)); + } + outgoing[outgoingEnded]?.(); +}; +const isPromise = (res) => typeof res.then === "function"; +const responseViaResponseObject = async (res, outgoing, options = {}) => { + if (isPromise(res)) if (options.errorHandler) try { + res = await res; + } catch (err) { + const errRes = await options.errorHandler(err); + if (!errRes) return; + res = errRes; + } + else res = await res.catch(handleFetchError); + if (cacheKey in res) return responseViaCache(res, outgoing); + const resHeaderRecord = buildOutgoingHttpHeaders(res.headers, res.body === null ? void 0 : defaultContentType); + if (res.body) { + const reader = res.body.getReader(); + const values = []; + let done = false; + let currentReadPromise = void 0; + if (resHeaderRecord["transfer-encoding"] !== "chunked") { + let maxReadCount = 2; + for (let i = 0; i < maxReadCount; i++) { + currentReadPromise ||= reader.read(); + const chunk = await readWithoutBlocking(currentReadPromise).catch((e) => { + console.error(e); + done = true; + }); + if (!chunk) { + if (i === 1) { + await new Promise((resolve) => setTimeout(resolve)); + maxReadCount = 3; + continue; + } + break; + } + currentReadPromise = void 0; + if (chunk.value) values.push(chunk.value); + if (chunk.done) { + done = true; + break; + } + } + if (done && !("content-length" in resHeaderRecord)) resHeaderRecord["content-length"] = values.reduce((acc, value) => acc + value.length, 0); + } + outgoing.writeHead(res.status, resHeaderRecord); + values.forEach((value) => { + outgoing.write(value); + }); + if (done) outgoing.end(); + else { + if (values.length === 0) flushHeaders(outgoing); + await writeFromReadableStreamDefaultReader(reader, outgoing, currentReadPromise); + } + } else if (resHeaderRecord[X_ALREADY_SENT]) {} else { + outgoing.writeHead(res.status, resHeaderRecord); + outgoing.end(); + } + outgoing[outgoingEnded]?.(); +}; +const getRequestListener = (fetchCallback, options = {}) => { + const autoCleanupIncoming = options.autoCleanupIncoming ?? true; + if (options.overrideGlobalObjects !== false && global.Request !== Request$1) { + Object.defineProperty(global, "Request", { value: Request$1 }); + Object.defineProperty(global, "Response", { value: Response$1 }); + } + return async (incoming, outgoing) => { + let res, req; + let needsBodyCleanup = false; + let closeHandlerAttached = false; + const ensureCloseHandler = () => { + if (!req || closeHandlerAttached) return; + closeHandlerAttached = true; + outgoing.on("close", makeCloseHandler(req, incoming, outgoing, needsBodyCleanup)); + }; + try { + req = newRequest(incoming, options.hostname); + needsBodyCleanup = autoCleanupIncoming && !(incoming.method === "GET" || incoming.method === "HEAD"); + if (needsBodyCleanup) { + incoming[wrapBodyStream] = true; + if (incoming instanceof Http2ServerRequest) outgoing[outgoingEnded] = () => { + if (!incoming.readableEnded) setTimeout(() => { + if (!incoming.readableEnded) setTimeout(() => { + incoming.destroy(); + outgoing.destroy(); + }); + }); + }; + } + res = fetchCallback(req, { + incoming, + outgoing + }); + if (!isPromise(res) && isImmediateCacheableResponse(res)) { + if (needsBodyCleanup && !incoming.readableEnded) outgoing.once("finish", () => { + if (!incoming.readableEnded) drainIncoming(incoming); + }); + return responseViaCache(res, outgoing); + } + ensureCloseHandler(); + } catch (e) { + if (!res) if (options.errorHandler) { + ensureCloseHandler(); + res = await options.errorHandler(req ? e : toRequestError(e)); + if (!res) return; + } else if (!req) res = handleRequestError(); + else res = handleFetchError(e); + else return handleResponseError(e, outgoing); + } + try { + return await responseViaResponseObject(res, outgoing, options); + } catch (e) { + return handleResponseError(e, outgoing); + } + }; +}; + +//#endregion +//#region src/websocket.ts +/** +* @link https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent +*/ +const CloseEvent = globalThis.CloseEvent ?? class extends Event { + #eventInitDict; + constructor(type, eventInitDict = {}) { + super(type, eventInitDict); + this.#eventInitDict = eventInitDict; + } + get wasClean() { + return this.#eventInitDict.wasClean ?? false; + } + get code() { + return this.#eventInitDict.code ?? 0; + } + get reason() { + return this.#eventInitDict.reason ?? ""; + } +}; +/** +* Node.js has no global `ErrorEvent`, unlike `Event`/`MessageEvent`/`CloseEvent`. +* @link https://developer.mozilla.org/en-US/docs/Web/API/ErrorEvent +*/ +const ErrorEvent = globalThis.ErrorEvent ?? class extends Event { + #eventInitDict; + constructor(type, eventInitDict = {}) { + super(type, eventInitDict); + this.#eventInitDict = eventInitDict; + } + get message() { + return this.#eventInitDict.message ?? ""; + } + get filename() { + return this.#eventInitDict.filename ?? ""; + } + get lineno() { + return this.#eventInitDict.lineno ?? 0; + } + get colno() { + return this.#eventInitDict.colno ?? 0; + } + get error() { + return this.#eventInitDict.error ?? null; + } +}; +const generateConnectionSymbol = () => Symbol("connection"); +const CONNECTION_SYMBOL_KEY = Symbol("CONNECTION_SYMBOL_KEY"); +const WAIT_FOR_WEBSOCKET_SYMBOL = Symbol("WAIT_FOR_WEBSOCKET_SYMBOL"); +const responseHeadersToSkip = new Set([ + "connection", + "content-length", + "keep-alive", + "proxy-authenticate", + "proxy-authorization", + "te", + "trailer", + "transfer-encoding", + "upgrade", + "sec-websocket-accept", + "sec-websocket-extensions", + "sec-websocket-protocol" +]); +const appendResponseHeaders = (headers, responseHeaders) => { + if (!responseHeaders) return; + responseHeaders.forEach((value, key) => { + if (responseHeadersToSkip.has(key.toLowerCase())) return; + headers.push(`${key}: ${value}`); + }); +}; +const rejectUpgradeRequest = (socket, status, responseHeaders) => { + const responseLines = ["Connection: close", "Content-Length: 0"]; + appendResponseHeaders(responseLines, responseHeaders); + socket.end(`HTTP/1.1 ${status.toString()} ${STATUS_CODES[status] ?? ""}\r\n${responseLines.join("\r\n")}\r\n\r +`); +}; +const createUpgradeRequest = (request) => { + const protocol = request.socket.encrypted ? "https" : "http"; + const url = new URL(request.url ?? "/", `${protocol}://${request.headers.host ?? "localhost"}`); + const headers = new Headers(); + for (const key in request.headers) { + const value = request.headers[key]; + if (!value) continue; + headers.append(key, Array.isArray(value) ? value[0] : value); + } + return new Request(url, { headers }); +}; +const setupWebSocket = (options) => { + const { server, fetchCallback, wss } = options; + const waiterMap = /* @__PURE__ */ new Map(); + wss.on("connection", (ws, request) => { + const waiter = waiterMap.get(request); + if (waiter) { + waiter.resolve(ws); + waiterMap.delete(request); + } + }); + const rejectWaiter = (request) => { + const waiter = waiterMap.get(request); + if (waiter) { + waiterMap.delete(request); + waiter.reject(/* @__PURE__ */ new Error("WebSocket handshake aborted")); + } + }; + const waitForWebSocket = (request, connectionSymbol) => { + return new Promise((resolve, reject) => { + waiterMap.set(request, { + resolve, + reject, + connectionSymbol + }); + }); + }; + server.on("upgrade", async (request, socket, head) => { + if (request.headers.upgrade?.toLowerCase() !== "websocket") return; + const env = { + incoming: request, + outgoing: void 0, + wss, + [WAIT_FOR_WEBSOCKET_SYMBOL]: waitForWebSocket + }; + let status = 400; + let responseHeaders; + try { + const response = await fetchCallback(createUpgradeRequest(request), env); + if (response instanceof Response) { + status = response.status; + responseHeaders = response.headers; + } + } catch { + if (server.listenerCount("upgrade") === 1) rejectUpgradeRequest(socket, 500); + return; + } + const waiter = waiterMap.get(request); + if (!waiter || waiter.connectionSymbol !== env[CONNECTION_SYMBOL_KEY]) { + rejectWaiter(request); + if (server.listenerCount("upgrade") === 1) rejectUpgradeRequest(socket, status, responseHeaders); + return; + } + const addResponseHeaders = (headers) => { + appendResponseHeaders(headers, responseHeaders); + }; + const reclaimWaiterOnClose = () => rejectWaiter(request); + socket.once("close", reclaimWaiterOnClose); + wss.on("headers", addResponseHeaders); + try { + wss.handleUpgrade(request, socket, head, (ws) => { + socket.off("close", reclaimWaiterOnClose); + wss.emit("connection", ws, request); + }); + } finally { + wss.off("headers", addResponseHeaders); + } + }); + server.on("close", () => { + wss.close(); + }); +}; +const upgradeWebSocket = defineWebSocketHelper(async (c, events, options) => { + if (c.req.header("upgrade")?.toLowerCase() !== "websocket") return; + const env = c.env; + const waitForWebSocket = env[WAIT_FOR_WEBSOCKET_SYMBOL]; + if (!waitForWebSocket || !env.incoming) return new Response(null, { status: 500 }); + const connectionSymbol = generateConnectionSymbol(); + env[CONNECTION_SYMBOL_KEY] = connectionSymbol; + (async () => { + let ws; + try { + ws = await waitForWebSocket(env.incoming, connectionSymbol); + } catch { + return; + } + const messagesReceivedInStarting = []; + const bufferMessage = (data, isBinary) => { + messagesReceivedInStarting.push([data, isBinary]); + }; + ws.on("message", bufferMessage); + const ctx = { + binaryType: "arraybuffer", + close(code, reason) { + ws.close(code, reason); + }, + protocol: ws.protocol, + raw: ws, + get readyState() { + return ws.readyState; + }, + send(source, opts) { + ws.send(source, { compress: opts?.compress }); + }, + url: new URL(c.req.url) + }; + try { + events?.onOpen?.(new Event("open"), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + const handleMessage = (data, isBinary) => { + const datas = Array.isArray(data) ? data : [data]; + for (const data of datas) try { + events?.onMessage?.(new MessageEvent("message", { data: isBinary ? data instanceof ArrayBuffer ? data : data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength) : typeof data === "string" ? data : Buffer.from(data).toString("utf-8") }), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + }; + ws.off("message", bufferMessage); + for (const message of messagesReceivedInStarting) handleMessage(...message); + ws.on("message", (data, isBinary) => { + handleMessage(data, isBinary); + }); + ws.on("close", (code, reason) => { + try { + events?.onClose?.(new CloseEvent("close", { + code, + reason: reason.toString() + }), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + }); + ws.on("error", (error) => { + try { + events?.onError?.(new ErrorEvent("error", { error }), ctx); + } catch (e) { + (options?.onError ?? console.error)(e); + } + }); + })(); + return new Response(); +}); + +//#endregion +//#region src/server.ts +const createAdaptorServer = (options) => { + const fetchCallback = options.fetch; + const requestListener = getRequestListener(fetchCallback, { + hostname: options.hostname, + overrideGlobalObjects: options.overrideGlobalObjects, + autoCleanupIncoming: options.autoCleanupIncoming + }); + const server = (options.createServer || createServer)(options.serverOptions || {}, requestListener); + if (options.websocket && options.websocket.server) { + if (options.websocket.server.options.noServer !== true) throw new Error("WebSocket server must be created with { noServer: true } option"); + setupWebSocket({ + server, + fetchCallback, + wss: options.websocket.server + }); + } + return server; +}; +const serve = (options, listeningListener) => { + const server = createAdaptorServer(options); + server.listen(options?.port ?? 3e3, options.hostname, () => { + const serverInfo = server.address(); + listeningListener && listeningListener(serverInfo); + }); + return server; +}; + +//#endregion +export { RequestError, createAdaptorServer, getRequestListener, serve, upgradeWebSocket }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.cjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.cjs new file mode 100644 index 000000000..bcfb0e52e --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.cjs @@ -0,0 +1,153 @@ +Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' }); +const require_utils_stream = require('./utils/stream.cjs'); +let hono_utils_mime = require("hono/utils/mime"); +let node_fs = require("node:fs"); +let node_path = require("node:path"); + +//#region src/serve-static.ts +const COMPRESSIBLE_CONTENT_TYPE_REGEX = /^\s*(?:text\/[^;\s]+|application\/(?:javascript|json|xml|xml-dtd|ecmascript|dart|postscript|rtf|tar|toml|vnd\.dart|vnd\.ms-fontobject|vnd\.ms-opentype|wasm|x-httpd-php|x-javascript|x-ns-proxy-autoconfig|x-sh|x-tar|x-virtualbox-hdd|x-virtualbox-ova|x-virtualbox-ovf|x-virtualbox-vbox|x-virtualbox-vdi|x-virtualbox-vhd|x-virtualbox-vmdk|x-www-form-urlencoded)|font\/(?:otf|ttf)|image\/(?:bmp|vnd\.adobe\.photoshop|vnd\.microsoft\.icon|vnd\.ms-dds|x-icon|x-ms-bmp)|message\/rfc822|model\/gltf-binary|x-shader\/x-fragment|x-shader\/x-vertex|[^;\s]+?\+(?:json|text|xml|yaml))(?:[;\s]|$)/i; +const ENCODINGS = { + br: ".br", + zstd: ".zst", + gzip: ".gz" +}; +const ENCODINGS_ORDERED_KEYS = Object.keys(ENCODINGS); +const getStats = (path) => { + let stats; + try { + stats = (0, node_fs.statSync)(path); + } catch {} + return stats; +}; +const BYTE_RANGE_PATTERN = /^(?:bytes=)?(?!-$)(\d*)-(\d*)$/; +const parseByteRange = (range) => { + const match = range.match(BYTE_RANGE_PATTERN); + if (!match) return; + const [, start, end] = match; + if (start === "") return { + type: "suffix", + length: Number(end) + }; + if (end === "") return { + type: "open-ended", + start: Number(start) + }; + return { + type: "bounded", + start: Number(start), + end: Number(end) + }; +}; +const resolveByteRange = (spec, size) => { + if (size === 0) return; + if (spec.type === "suffix") { + if (spec.length === 0) return; + return { + start: Math.max(size - spec.length, 0), + end: size - 1 + }; + } + const end = spec.type === "bounded" ? Math.min(spec.end, size - 1) : size - 1; + if (spec.start >= size || spec.start > end) return; + return { + start: spec.start, + end + }; +}; +const tryDecode = (str, decoder) => { + try { + return decoder(str); + } catch { + return str.replace(/(?:%[0-9A-Fa-f]{2})+/g, (match) => { + try { + return decoder(match); + } catch { + return match; + } + }); + } +}; +const tryDecodeURI = (str) => tryDecode(str, decodeURI); +const serveStatic = (options = { root: "" }) => { + const root = options.root || ""; + const optionPath = options.path; + if (root !== "" && !(0, node_fs.existsSync)(root)) console.error(`serveStatic: root path '${root}' is not found, are you sure it's correct?`); + return async (c, next) => { + if (c.finalized) return next(); + if (c.req.method !== "GET" && c.req.method !== "HEAD") return next(); + let filename; + if (optionPath) filename = optionPath; + else try { + if (!options.allowPercentInPath && c.req.path.includes("%")) throw new Error(); + filename = tryDecodeURI(c.req.path); + if (/(?:^|[\/\\])\.{1,2}(?:$|[\/\\])|[\/\\]{2,}|\\/.test(filename)) throw new Error(); + } catch { + await options.onNotFound?.(c.req.path, c); + return next(); + } + let path = (0, node_path.join)(root, !optionPath && options.rewriteRequestPath ? options.rewriteRequestPath(filename, c) : filename); + let stats = getStats(path); + if (stats && stats.isDirectory()) { + const indexFile = options.index ?? "index.html"; + path = (0, node_path.join)(path, indexFile); + stats = getStats(path); + } + if (!stats) { + await options.onNotFound?.(path, c); + return next(); + } + const mimeType = (0, hono_utils_mime.getMimeType)(path); + c.header("Content-Type", mimeType || "application/octet-stream"); + if (options.precompressed && (!mimeType || mimeType === "application/octet-stream" || COMPRESSIBLE_CONTENT_TYPE_REGEX.test(mimeType))) { + const acceptEncodingSet = new Set(c.req.header("Accept-Encoding")?.split(",").map((encoding) => encoding.trim())); + for (const encoding of ENCODINGS_ORDERED_KEYS) { + if (!acceptEncodingSet.has(encoding)) continue; + const precompressedStats = getStats(path + ENCODINGS[encoding]); + if (precompressedStats) { + c.header("Content-Encoding", encoding); + c.header("Vary", "Accept-Encoding", { append: true }); + stats = precompressedStats; + path = path + ENCODINGS[encoding]; + break; + } + } + } + let result; + const size = stats.size; + const range = c.req.header("range") || ""; + c.header("Last-Modified", stats.mtime.toUTCString()); + if (c.req.method === "HEAD") { + c.header("Content-Length", size.toString()); + c.status(200); + result = c.body(null); + } else if (!range) { + c.header("Content-Length", size.toString()); + result = c.body(require_utils_stream.createStreamBody((0, node_fs.createReadStream)(path)), 200); + } else { + c.header("Accept-Ranges", "bytes"); + const resolvedRange = resolveByteRange(parseByteRange(range) ?? { + type: "open-ended", + start: 0 + }, size); + if (!resolvedRange) { + c.header("Content-Range", `bytes */${size}`); + result = c.body(null, 416); + } else { + const { start, end } = resolvedRange; + const chunkSize = end - start + 1; + const stream = (0, node_fs.createReadStream)(path, { + start, + end + }); + c.header("Content-Length", chunkSize.toString()); + c.header("Content-Range", `bytes ${start}-${end}/${size}`); + result = c.body(require_utils_stream.createStreamBody(stream), 206); + } + } + await options.onFound?.(path, c); + return result; + }; +}; + +//#endregion +exports.serveStatic = serveStatic; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.d.cts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.d.cts new file mode 100644 index 000000000..f4dae1201 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.d.cts @@ -0,0 +1,19 @@ +import { Context, Env, MiddlewareHandler } from "hono"; + +//#region src/serve-static.d.ts +type ServeStaticOptions = { + /** + * Root path. Relative path is based on current working directory from which the app was started. + */ + root?: string; + path?: string; + index?: string; + precompressed?: boolean; /** Allow percent signs in the routed request path. Defaults to false. */ + allowPercentInPath?: boolean; + rewriteRequestPath?: (path: string, c: Context) => string; + onFound?: (path: string, c: Context) => void | Promise; + onNotFound?: (path: string, c: Context) => void | Promise; +}; +declare const serveStatic: (options?: ServeStaticOptions) => MiddlewareHandler; +//#endregion +export { ServeStaticOptions, serveStatic }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.d.mts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.d.mts new file mode 100644 index 000000000..f4dae1201 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.d.mts @@ -0,0 +1,19 @@ +import { Context, Env, MiddlewareHandler } from "hono"; + +//#region src/serve-static.d.ts +type ServeStaticOptions = { + /** + * Root path. Relative path is based on current working directory from which the app was started. + */ + root?: string; + path?: string; + index?: string; + precompressed?: boolean; /** Allow percent signs in the routed request path. Defaults to false. */ + allowPercentInPath?: boolean; + rewriteRequestPath?: (path: string, c: Context) => string; + onFound?: (path: string, c: Context) => void | Promise; + onNotFound?: (path: string, c: Context) => void | Promise; +}; +declare const serveStatic: (options?: ServeStaticOptions) => MiddlewareHandler; +//#endregion +export { ServeStaticOptions, serveStatic }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.mjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.mjs new file mode 100644 index 000000000..97505a187 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/serve-static.mjs @@ -0,0 +1,152 @@ +import { createStreamBody } from "./utils/stream.mjs"; +import { getMimeType } from "hono/utils/mime"; +import { createReadStream, existsSync, statSync } from "node:fs"; +import { join } from "node:path"; + +//#region src/serve-static.ts +const COMPRESSIBLE_CONTENT_TYPE_REGEX = /^\s*(?:text\/[^;\s]+|application\/(?:javascript|json|xml|xml-dtd|ecmascript|dart|postscript|rtf|tar|toml|vnd\.dart|vnd\.ms-fontobject|vnd\.ms-opentype|wasm|x-httpd-php|x-javascript|x-ns-proxy-autoconfig|x-sh|x-tar|x-virtualbox-hdd|x-virtualbox-ova|x-virtualbox-ovf|x-virtualbox-vbox|x-virtualbox-vdi|x-virtualbox-vhd|x-virtualbox-vmdk|x-www-form-urlencoded)|font\/(?:otf|ttf)|image\/(?:bmp|vnd\.adobe\.photoshop|vnd\.microsoft\.icon|vnd\.ms-dds|x-icon|x-ms-bmp)|message\/rfc822|model\/gltf-binary|x-shader\/x-fragment|x-shader\/x-vertex|[^;\s]+?\+(?:json|text|xml|yaml))(?:[;\s]|$)/i; +const ENCODINGS = { + br: ".br", + zstd: ".zst", + gzip: ".gz" +}; +const ENCODINGS_ORDERED_KEYS = Object.keys(ENCODINGS); +const getStats = (path) => { + let stats; + try { + stats = statSync(path); + } catch {} + return stats; +}; +const BYTE_RANGE_PATTERN = /^(?:bytes=)?(?!-$)(\d*)-(\d*)$/; +const parseByteRange = (range) => { + const match = range.match(BYTE_RANGE_PATTERN); + if (!match) return; + const [, start, end] = match; + if (start === "") return { + type: "suffix", + length: Number(end) + }; + if (end === "") return { + type: "open-ended", + start: Number(start) + }; + return { + type: "bounded", + start: Number(start), + end: Number(end) + }; +}; +const resolveByteRange = (spec, size) => { + if (size === 0) return; + if (spec.type === "suffix") { + if (spec.length === 0) return; + return { + start: Math.max(size - spec.length, 0), + end: size - 1 + }; + } + const end = spec.type === "bounded" ? Math.min(spec.end, size - 1) : size - 1; + if (spec.start >= size || spec.start > end) return; + return { + start: spec.start, + end + }; +}; +const tryDecode = (str, decoder) => { + try { + return decoder(str); + } catch { + return str.replace(/(?:%[0-9A-Fa-f]{2})+/g, (match) => { + try { + return decoder(match); + } catch { + return match; + } + }); + } +}; +const tryDecodeURI = (str) => tryDecode(str, decodeURI); +const serveStatic = (options = { root: "" }) => { + const root = options.root || ""; + const optionPath = options.path; + if (root !== "" && !existsSync(root)) console.error(`serveStatic: root path '${root}' is not found, are you sure it's correct?`); + return async (c, next) => { + if (c.finalized) return next(); + if (c.req.method !== "GET" && c.req.method !== "HEAD") return next(); + let filename; + if (optionPath) filename = optionPath; + else try { + if (!options.allowPercentInPath && c.req.path.includes("%")) throw new Error(); + filename = tryDecodeURI(c.req.path); + if (/(?:^|[\/\\])\.{1,2}(?:$|[\/\\])|[\/\\]{2,}|\\/.test(filename)) throw new Error(); + } catch { + await options.onNotFound?.(c.req.path, c); + return next(); + } + let path = join(root, !optionPath && options.rewriteRequestPath ? options.rewriteRequestPath(filename, c) : filename); + let stats = getStats(path); + if (stats && stats.isDirectory()) { + const indexFile = options.index ?? "index.html"; + path = join(path, indexFile); + stats = getStats(path); + } + if (!stats) { + await options.onNotFound?.(path, c); + return next(); + } + const mimeType = getMimeType(path); + c.header("Content-Type", mimeType || "application/octet-stream"); + if (options.precompressed && (!mimeType || mimeType === "application/octet-stream" || COMPRESSIBLE_CONTENT_TYPE_REGEX.test(mimeType))) { + const acceptEncodingSet = new Set(c.req.header("Accept-Encoding")?.split(",").map((encoding) => encoding.trim())); + for (const encoding of ENCODINGS_ORDERED_KEYS) { + if (!acceptEncodingSet.has(encoding)) continue; + const precompressedStats = getStats(path + ENCODINGS[encoding]); + if (precompressedStats) { + c.header("Content-Encoding", encoding); + c.header("Vary", "Accept-Encoding", { append: true }); + stats = precompressedStats; + path = path + ENCODINGS[encoding]; + break; + } + } + } + let result; + const size = stats.size; + const range = c.req.header("range") || ""; + c.header("Last-Modified", stats.mtime.toUTCString()); + if (c.req.method === "HEAD") { + c.header("Content-Length", size.toString()); + c.status(200); + result = c.body(null); + } else if (!range) { + c.header("Content-Length", size.toString()); + result = c.body(createStreamBody(createReadStream(path)), 200); + } else { + c.header("Accept-Ranges", "bytes"); + const resolvedRange = resolveByteRange(parseByteRange(range) ?? { + type: "open-ended", + start: 0 + }, size); + if (!resolvedRange) { + c.header("Content-Range", `bytes */${size}`); + result = c.body(null, 416); + } else { + const { start, end } = resolvedRange; + const chunkSize = end - start + 1; + const stream = createReadStream(path, { + start, + end + }); + c.header("Content-Length", chunkSize.toString()); + c.header("Content-Range", `bytes ${start}-${end}/${size}`); + result = c.body(createStreamBody(stream), 206); + } + } + await options.onFound?.(path, c); + return result; + }; +}; + +//#endregion +export { serveStatic }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.cjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.cjs new file mode 100644 index 000000000..87b8e95ab --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.cjs @@ -0,0 +1,8 @@ +Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' }); +const require_constants = require('../constants-BXAKTxRC.cjs'); + +//#region src/utils/response.ts +const RESPONSE_ALREADY_SENT = new Response(null, { headers: { [require_constants.X_ALREADY_SENT]: "true" } }); + +//#endregion +exports.RESPONSE_ALREADY_SENT = RESPONSE_ALREADY_SENT; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.d.cts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.d.cts new file mode 100644 index 000000000..169a3b551 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.d.cts @@ -0,0 +1,4 @@ +//#region src/utils/response.d.ts +declare const RESPONSE_ALREADY_SENT: Response; +//#endregion +export { RESPONSE_ALREADY_SENT }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.d.mts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.d.mts new file mode 100644 index 000000000..169a3b551 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.d.mts @@ -0,0 +1,4 @@ +//#region src/utils/response.d.ts +declare const RESPONSE_ALREADY_SENT: Response; +//#endregion +export { RESPONSE_ALREADY_SENT }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.mjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.mjs new file mode 100644 index 000000000..92495c1f5 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/response.mjs @@ -0,0 +1,7 @@ +import { t as X_ALREADY_SENT } from "../constants-BLSFu_RU.mjs"; + +//#region src/utils/response.ts +const RESPONSE_ALREADY_SENT = new Response(null, { headers: { [X_ALREADY_SENT]: "true" } }); + +//#endregion +export { RESPONSE_ALREADY_SENT }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.cjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.cjs new file mode 100644 index 000000000..b4f4f7d42 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.cjs @@ -0,0 +1,65 @@ +Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' }); +let node_stream = require("node:stream"); +let node_process = require("node:process"); + +//#region src/utils/stream.ts +const pr54206Applied = () => { + const [major, minor] = node_process.versions.node.split(".").map((component) => parseInt(component)); + return major >= 23 || major === 22 && minor >= 7 || major === 20 && minor >= 18; +}; +const useReadableToWeb = pr54206Applied(); +const createStreamBody = (stream, useNativeReadableToWeb = useReadableToWeb) => { + if (useNativeReadableToWeb) return node_stream.Readable.toWeb(stream); + let controller; + let settled = false; + const cleanup = () => { + stream.off("data", onData); + stream.off("error", onError); + stream.off("end", onTerminate); + stream.off("close", onTerminate); + }; + const settle = (callback) => { + if (settled) return; + settled = true; + cleanup(); + callback?.(); + }; + const onData = (chunk) => { + if (settled || !controller) return; + controller.enqueue(chunk); + if ((controller.desiredSize ?? 0) <= 0) stream.pause(); + }; + const onError = (error) => { + settle(() => { + controller?.error(error); + }); + }; + const onTerminate = () => { + settle(() => { + controller?.close(); + }); + }; + return new ReadableStream({ + start(streamController) { + controller = streamController; + stream.on("data", onData); + stream.on("error", onError); + stream.on("end", onTerminate); + stream.on("close", onTerminate); + stream.pause(); + }, + pull() { + if (!settled) stream.resume(); + }, + cancel() { + settle(); + const ignoreError = () => {}; + stream.on("error", ignoreError); + stream.once("close", () => stream.off("error", ignoreError)); + stream.destroy(); + } + }); +}; + +//#endregion +exports.createStreamBody = createStreamBody; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.d.cts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.d.cts new file mode 100644 index 000000000..5f92c4f79 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.d.cts @@ -0,0 +1,6 @@ +import { ReadStream } from "node:fs"; + +//#region src/utils/stream.d.ts +declare const createStreamBody: (stream: ReadStream, useNativeReadableToWeb?: boolean) => ReadableStream; +//#endregion +export { createStreamBody }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.d.mts b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.d.mts new file mode 100644 index 000000000..5f92c4f79 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.d.mts @@ -0,0 +1,6 @@ +import { ReadStream } from "node:fs"; + +//#region src/utils/stream.d.ts +declare const createStreamBody: (stream: ReadStream, useNativeReadableToWeb?: boolean) => ReadableStream; +//#endregion +export { createStreamBody }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.mjs b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.mjs new file mode 100644 index 000000000..8d18a9a2e --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/dist/utils/stream.mjs @@ -0,0 +1,64 @@ +import { Readable } from "node:stream"; +import { versions } from "node:process"; + +//#region src/utils/stream.ts +const pr54206Applied = () => { + const [major, minor] = versions.node.split(".").map((component) => parseInt(component)); + return major >= 23 || major === 22 && minor >= 7 || major === 20 && minor >= 18; +}; +const useReadableToWeb = pr54206Applied(); +const createStreamBody = (stream, useNativeReadableToWeb = useReadableToWeb) => { + if (useNativeReadableToWeb) return Readable.toWeb(stream); + let controller; + let settled = false; + const cleanup = () => { + stream.off("data", onData); + stream.off("error", onError); + stream.off("end", onTerminate); + stream.off("close", onTerminate); + }; + const settle = (callback) => { + if (settled) return; + settled = true; + cleanup(); + callback?.(); + }; + const onData = (chunk) => { + if (settled || !controller) return; + controller.enqueue(chunk); + if ((controller.desiredSize ?? 0) <= 0) stream.pause(); + }; + const onError = (error) => { + settle(() => { + controller?.error(error); + }); + }; + const onTerminate = () => { + settle(() => { + controller?.close(); + }); + }; + return new ReadableStream({ + start(streamController) { + controller = streamController; + stream.on("data", onData); + stream.on("error", onError); + stream.on("end", onTerminate); + stream.on("close", onTerminate); + stream.pause(); + }, + pull() { + if (!settled) stream.resume(); + }, + cancel() { + settle(); + const ignoreError = () => {}; + stream.on("error", ignoreError); + stream.once("close", () => stream.off("error", ignoreError)); + stream.destroy(); + } + }); +}; + +//#endregion +export { createStreamBody }; \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/@hono/node-server/package.json b/tests/fixtures/npm/node_modules/@hono/node-server/package.json new file mode 100644 index 000000000..b3ab47d33 --- /dev/null +++ b/tests/fixtures/npm/node_modules/@hono/node-server/package.json @@ -0,0 +1,128 @@ +{ + "name": "@hono/node-server", + "version": "2.1.5", + "description": "Node.js Adapter for Hono", + "main": "dist/index.mjs", + "type": "module", + "types": "dist/index.d.mts", + "files": [ + "dist" + ], + "exports": { + ".": { + "import": { + "types": "./dist/index.d.mts", + "default": "./dist/index.mjs" + }, + "require": { + "types": "./dist/index.d.cts", + "default": "./dist/index.cjs" + } + }, + "./serve-static": { + "import": { + "types": "./dist/serve-static.d.mts", + "default": "./dist/serve-static.mjs" + }, + "require": { + "types": "./dist/serve-static.d.cts", + "default": "./dist/serve-static.cjs" + } + }, + "./utils/*": { + "import": { + "types": "./dist/utils/*.d.mts", + "default": "./dist/utils/*.mjs" + }, + "require": { + "types": "./dist/utils/*.d.cts", + "default": "./dist/utils/*.cjs" + } + }, + "./conninfo": { + "import": { + "types": "./dist/conninfo.d.mts", + "default": "./dist/conninfo.mjs" + }, + "require": { + "types": "./dist/conninfo.d.cts", + "default": "./dist/conninfo.cjs" + } + }, + "./early-hints": { + "import": { + "types": "./dist/early-hints.d.mts", + "default": "./dist/early-hints.mjs" + }, + "require": { + "types": "./dist/early-hints.d.cts", + "default": "./dist/early-hints.cjs" + } + } + }, + "typesVersions": { + "*": { + ".": [ + "./dist/index.d.mts" + ], + "serve-static": [ + "./dist/serve-static.d.mts" + ], + "utils/*": [ + "./dist/utils/*.d.mts" + ], + "conninfo": [ + "./dist/conninfo.d.mts" + ], + "early-hints": [ + "./dist/early-hints.d.mts" + ] + } + }, + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/honojs/node-server.git" + }, + "homepage": "https://github.com/honojs/node-server", + "author": "Yusuke Wada (https://github.com/yusukebe)", + "publishConfig": { + "registry": "https://registry.npmjs.org", + "access": "public", + "provenance": true + }, + "engines": { + "node": ">=20" + }, + "devDependencies": { + "@hono/eslint-config": "^1.0.1", + "@types/node": "^20.10.0", + "@types/ws": "^8.18.1", + "@whatwg-node/fetch": "^0.9.14", + "eslint": "^9.10.0", + "hono": "^4.13.9", + "np": "^11.2.1", + "prettier": "^3.2.4", + "publint": "^0.3.18", + "tsdown": "^0.20.3", + "typescript": "^5.3.2", + "vitest": "^4.0.18", + "ws": "^8.19.0" + }, + "peerDependencies": { + "hono": "^4 || ^5.0.0-0" + }, + "scripts": { + "test": "vitest", + "build": "tsdown --external hono", + "watch": "tsdown --watch", + "postbuild": "publint", + "prerelease": "pnpm run build && pnpm run test --run", + "release": "np --no-publish", + "typecheck": "tsc --noEmit", + "lint": "eslint src test", + "lint:fix": "eslint src test --fix", + "format": "prettier --check \"src/**/*.{js,ts}\" \"test/**/*.{js,ts}\"", + "format:fix": "prettier --write \"src/**/*.{js,ts}\" \"test/**/*.{js,ts}\"" + } +} \ No newline at end of file diff --git a/tests/fixtures/npm/node_modules/gtminicache/dist/index.d.ts b/tests/fixtures/npm/node_modules/gtminicache/dist/index.d.ts new file mode 100644 index 000000000..5d12cad7e --- /dev/null +++ b/tests/fixtures/npm/node_modules/gtminicache/dist/index.d.ts @@ -0,0 +1,7 @@ +export declare class Cache { + constructor(max: number); + set(key: K, value: V): this; + get(key: K): V | undefined; + keys(): K[]; + describe(label: string): string; +} diff --git a/tests/fixtures/npm/node_modules/gtminicache/dist/index.js b/tests/fixtures/npm/node_modules/gtminicache/dist/index.js new file mode 100644 index 000000000..31c6deef9 --- /dev/null +++ b/tests/fixtures/npm/node_modules/gtminicache/dist/index.js @@ -0,0 +1 @@ +"use strict";var n=(s,e)=>()=>(e||s((e={exports:{}}).exports,e),e.exports);var u=n(i=>{"use strict";Object.defineProperty(i,"__esModule",{value:!0});i.metrics=void 0;var o=require("node:diagnostics_channel");i.metrics=(0,o.channel)("gtminicache:metrics")});var d=n(r=>{"use strict";Object.defineProperty(r,"__esModule",{value:!0});r.describe=a;var l=require("node:util");function a(s,e){return(0,l.format)("%s holds %d",s,e)}});Object.defineProperty(exports,"__esModule",{value:!0});exports.Cache=void 0;var h=u(),_=d(),c=class{#e=new Map;#t;constructor(e){this.#t=e}set(e,t){for(this.#e.delete(e),this.#e.set(e,t);this.#e.size>this.#t;)this.#e.delete(this.#e.keys().next().value);return h.metrics.hasSubscribers&&h.metrics.publish({key:e}),this}get(e){let t=this.#e.get(e);return t!==void 0&&(this.#e.delete(e),this.#e.set(e,t)),t}keys(){return[...this.#e.keys()]}describe(e){return(0,_.describe)(e,this.#e.size)}};exports.Cache=c; diff --git a/tests/fixtures/npm/node_modules/gtminicache/package.json b/tests/fixtures/npm/node_modules/gtminicache/package.json new file mode 100644 index 000000000..3f57e1ae3 --- /dev/null +++ b/tests/fixtures/npm/node_modules/gtminicache/package.json @@ -0,0 +1,7 @@ +{ + "name": "gtminicache", + "version": "1.0.0", + "description": "tsc-style CommonJS modules bundled by esbuild 0.27.0 with --bundle --platform=node --format=cjs --minify: the __commonJS helper and the chunk parameters lose their names, the __esModule stamps read { value: !0 }, and calls into node:util and node:diagnostics_channel use the (0, ns.member)(...) form", + "main": "./dist/index.js", + "types": "./dist/index.d.ts" +} diff --git a/tests/fixtures/npm/node_modules/gtminimark/dist/index.d.ts b/tests/fixtures/npm/node_modules/gtminimark/dist/index.d.ts new file mode 100644 index 000000000..295430b86 --- /dev/null +++ b/tests/fixtures/npm/node_modules/gtminimark/dist/index.d.ts @@ -0,0 +1 @@ +export declare function render(src: string, options?: { pedantic?: boolean }): string; diff --git a/tests/fixtures/npm/node_modules/gtminimark/dist/index.js b/tests/fixtures/npm/node_modules/gtminimark/dist/index.js new file mode 100644 index 000000000..c5e1daa31 --- /dev/null +++ b/tests/fixtures/npm/node_modules/gtminimark/dist/index.js @@ -0,0 +1 @@ +var m = { heading: /^(#{1,6}) +(.*)$/, link: /^\[([^\]]*)\]\(([^ )]*)(?: "([^"]*)")?\)/, ref: /^\[([^\]]*)\]$/, lead: /^( {1,4}| {0,3}\t)/, nonSpace: /[^ ]/, punct: /\\([\\*_\[\]])/g, word: /\w+/g, def: /^\[([^\]]*)\]: (\S+)(?: "([^"]*)")?$/ },B = { ...m, emph: /\*([^*]+)\*/g },P = { ...B, word: /[a-z]+/g },K = { normal: { other: m, inline: B }, pedantic: { other: m, inline: P, strict: !0 } };function X(a, e) { let t = 0, n = ""; for (let i of a) if (i === "\t") { let r = 4 - t % 4; n += " ".repeat(r), t += r } else n += i, t++; return n + e; }function Se(a, e, t, n) { let r = e.href, o = e.title || null; n.links++; return { type: "link", raw: t, href: r, title: o, text: a[1] }; }class T {constructor() { this.state = { links: 0 }; }heading(s) { let c = s.match(this.rules.other.heading); return c ? "h" + c[1].length + ":" + c[2].replace(this.rules.other.punct, "$1") : null; }indent(s, w) { return s.replace(this.rules.other.lead, W => X(W, w)); }first(s) { return s.search(this.rules.other.nonSpace); }emph(s) { return s.replace(this.rules.inline.emph, (W, x) => "" + x + ""); }words(s) { let c = s.match(this.rules.inline.word); return c ? c.length : 0; }link(s) { let c = this.rules.other.link.exec(s); if (!c) return null; let r = c[2], o = ""; o = c[3] ? c[3] : ""; return Se(c, { href: r && r.replace(this.rules.other.punct, "$1"), title: o && o.replace(this.rules.other.punct, "$1") }, c[0], this.state); }ref(s, l) { let c = this.rules.other.ref.exec(s); if (!c) return null; let o = l[c[1].toLowerCase()]; return o ? Se(c, o, c[0], this.state) : { type: "text", raw: c[0] }; }}class L {constructor(e) { this.options = e || {}; this.links = {}; this.tokenizer = new T(); this.tokenizer.rules = this.options.pedantic ? K.pedantic : K.normal; }define(s) { let c = s.match(this.tokenizer.rules.other.def); if (c) this.links[c[1].toLowerCase()] = { href: c[2], title: c[3] || "" }; return !!c; }line(s) {let t = this.tokenizer, h = t.heading(s);if (h) return h;let k = t.link(s);if (k) return JSON.stringify(k);let f = t.ref(s, this.links);if (f) return JSON.stringify(f);return t.first(s) + "|" + JSON.stringify(t.indent(s, t.first(s))) + "|" + t.emph(s) + "|" + t.words(s);}}export function render(src, options) {let l = new L(options), out = [];for (let s of src.split("\n")) if (!l.define(s)) out.push(l.line(s));return out.join("\n") + "\nlinks " + l.tokenizer.state.links;} diff --git a/tests/fixtures/npm/node_modules/gtminimark/package.json b/tests/fixtures/npm/node_modules/gtminimark/package.json new file mode 100644 index 000000000..ff188c682 --- /dev/null +++ b/tests/fixtures/npm/node_modules/gtminimark/package.json @@ -0,0 +1,8 @@ +{ + "name": "gtminimark", + "version": "1.0.0", + "description": "A one-line minified ES module: a module-level var statement whose later declarators spread earlier ones, regexes read through an untyped rules table, and a function called with typed and untyped link objects", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts" +} diff --git a/tests/fixtures/npm/node_modules/hono/LICENSE b/tests/fixtures/npm/node_modules/hono/LICENSE new file mode 100644 index 000000000..bd174facc --- /dev/null +++ b/tests/fixtures/npm/node_modules/hono/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 - present, Yusuke Wada and Hono contributors + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/tests/fixtures/npm/node_modules/hono/dist/helper/websocket/index.js b/tests/fixtures/npm/node_modules/hono/dist/helper/websocket/index.js new file mode 100644 index 000000000..620de4105 --- /dev/null +++ b/tests/fixtures/npm/node_modules/hono/dist/helper/websocket/index.js @@ -0,0 +1,53 @@ +//#region src/helper/websocket/index.ts +/** +* A context for controlling WebSockets +*/ +var WSContext = class { + #init; + constructor(init) { + this.#init = init; + this.raw = init.raw; + this.url = init.url ? new URL(init.url) : null; + this.protocol = init.protocol ?? null; + } + send(source, options) { + this.#init.send(source, options ?? {}); + } + raw; + binaryType = "arraybuffer"; + get readyState() { + return this.#init.readyState; + } + url; + protocol; + close(code, reason) { + this.#init.close(code, reason); + } +}; +const createWSMessageEvent = (source) => { + return new MessageEvent("message", { data: source }); +}; +/** +* Create a WebSocket adapter/helper +*/ +const defineWebSocketHelper = (handler) => { + return ((...args) => { + if (typeof args[0] === "function") { + const [createEvents, options] = args; + return async function upgradeWebSocket(c, next) { + const result = await handler(c, await createEvents(c), options); + if (result) return result; + await next(); + }; + } else { + const [c, events, options] = args; + return (async () => { + const upgraded = await handler(c, events, options); + if (!upgraded) throw new Error("Failed to upgrade WebSocket"); + return upgraded; + })(); + } + }); +}; +//#endregion +export { WSContext, createWSMessageEvent, defineWebSocketHelper }; diff --git a/tests/fixtures/npm/node_modules/hono/package.json b/tests/fixtures/npm/node_modules/hono/package.json new file mode 100644 index 000000000..6264c824a --- /dev/null +++ b/tests/fixtures/npm/node_modules/hono/package.json @@ -0,0 +1,12 @@ +{ + "name": "hono", + "version": "4.13.13", + "description": "Test fixture: only the hono/ws helper that @hono/node-server's index imports.", + "type": "module", + "license": "MIT", + "exports": { + "./ws": { + "import": "./dist/helper/websocket/index.js" + } + } +} diff --git a/tests/harness/__snapshots__/node-types-fenced.txt b/tests/harness/__snapshots__/node-types-fenced.txt index cabce49c4..562733f53 100644 --- a/tests/harness/__snapshots__/node-types-fenced.txt +++ b/tests/harness/__snapshots__/node-types-fenced.txt @@ -120,22 +120,22 @@ fenced.ts:76:1 - error SC2020: 'crypto.publicEncrypt' is typed by @types/node bu 75 | import { createCipheriv, publicEncrypt, secureHeapUsed, setFips } from "node:crypto"; 76 | publicEncrypt("key", Buffer.alloc(1)); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - 77 | createCipheriv("aes-128-ccm", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); + 77 | createCipheriv("aes-128-ocb", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); hint: RSA encryption has no lowering — the asymmetric crypto surface is RSA and EC KeyObjects (generateKeyPair, createPrivateKey, createPublicKey), sign, verify, createSign, createVerify, createECDH, and createDiffieHellman -fenced.ts:77:16 - error SC2020: 'crypto.createCipheriv with cipher 'aes-128-ccm'' is part of the standard library types but has no scriptc lowering yet +fenced.ts:77:16 - error SC2020: 'crypto.createCipheriv with cipher 'aes-128-ocb'' is part of the standard library types but has no scriptc lowering yet 76 | publicEncrypt("key", Buffer.alloc(1)); - 77 | createCipheriv("aes-128-ccm", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); + 77 | createCipheriv("aes-128-ocb", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); | ^~~~~~~~~~~~~ 78 | secureHeapUsed(); - hint: the supported ciphers are aes-128, aes-192 and aes-256 in cbc, ctr, gcm, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede3 in cbc, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede in cbc, ecb, cfb and ofb modes, chacha20, and chacha20-poly1305 + hint: the supported ciphers are aes-128, aes-192 and aes-256 in cbc, ctr, gcm, ccm, ecb, cfb, cfb8, cfb1 and ofb modes and their wrap, wrap-pad, wrap-inv and wrap-pad-inv key wraps, des-ede3 in cbc, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede in cbc, ecb, cfb and ofb modes, chacha20, and chacha20-poly1305 fenced.ts:78:1 - error SC2020: 'crypto.secureHeapUsed' is typed by @types/node but has no scriptc lowering yet - 77 | createCipheriv("aes-128-ccm", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); + 77 | createCipheriv("aes-128-ocb", Buffer.alloc(16), Buffer.alloc(12), { authTagLength: 16 }); 78 | secureHeapUsed(); | ^~~~~~~~~~~~~~~~ 79 | setFips(false); diff --git a/tests/harness/__snapshots__/stream-runtime-option-order.ts.txt b/tests/harness/__snapshots__/stream-runtime-option-order.ts.txt new file mode 100644 index 000000000..ff7f10be1 --- /dev/null +++ b/tests/harness/__snapshots__/stream-runtime-option-order.ts.txt @@ -0,0 +1,26 @@ +stream-runtime-option-order.ts:16:3 - error SC2020: 'the Writable option 'defaultEncoding' with a runtime value that an earlier option value could observe or change' is part of the standard library types but has no scriptc lowering yet + + 15 | highWaterMark: bump(), + 16 | defaultEncoding: encoding, + | ^~~~~~~~~~~~~~~~~~~~~~~~~ + 17 | write(_chunk, _encoding, done) { + + hint: compute the option values into consts before the options object (the runtime value is read before the other options run) + +stream-runtime-option-order.ts:23:3 - error SC2020: 'the Writable option 'decodeStrings' with a runtime value that an earlier option value could observe or change' is part of the standard library types but has no scriptc lowering yet + + 22 | highWaterMark: bump(), + 23 | decodeStrings: decode, + | ^~~~~~~~~~~~~~~~~~~~~ + 24 | write(_chunk, _encoding, done) { + + hint: compute the option values into consts before the options object (the runtime value is read before the other options run) + +stream-runtime-option-order.ts:35:3 - error SC2020: 'the Writable option 'defaultEncoding' with a runtime value that an earlier option value could observe or change' is part of the standard library types but has no scriptc lowering yet + + 34 | highWaterMark: sizes.hwm, + 35 | defaultEncoding: pickEncoding(), + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + 36 | write(_chunk, _encoding, done) { + + hint: compute the option values into consts before the options object (the runtime value is read before the other options run) \ No newline at end of file diff --git a/tests/harness/corpus-costs/7154-crypto-cipher-info.cjs.json b/tests/harness/corpus-costs/7154-crypto-cipher-info.cjs.json new file mode 100644 index 000000000..4eec93aec --- /dev/null +++ b/tests/harness/corpus-costs/7154-crypto-cipher-info.cjs.json @@ -0,0 +1 @@ +{ "plain": 6115, "san": 62881 } diff --git a/tests/harness/corpus-costs/7155-cipher-aes-ccm.cjs.json b/tests/harness/corpus-costs/7155-cipher-aes-ccm.cjs.json new file mode 100644 index 000000000..4968524bd --- /dev/null +++ b/tests/harness/corpus-costs/7155-cipher-aes-ccm.cjs.json @@ -0,0 +1 @@ +{ "plain": 8462, "san": 75851 } diff --git a/tests/harness/corpus-costs/7156-cipher-key-wrap.cjs.json b/tests/harness/corpus-costs/7156-cipher-key-wrap.cjs.json new file mode 100644 index 000000000..b5e45dae6 --- /dev/null +++ b/tests/harness/corpus-costs/7156-cipher-key-wrap.cjs.json @@ -0,0 +1 @@ +{ "plain": 5548, "san": 52523 } diff --git a/tests/harness/corpus-costs/7157-crypto-cipher-info-typed.ts.json b/tests/harness/corpus-costs/7157-crypto-cipher-info-typed.ts.json new file mode 100644 index 000000000..425ec1dc2 --- /dev/null +++ b/tests/harness/corpus-costs/7157-crypto-cipher-info-typed.ts.json @@ -0,0 +1 @@ +{ "plain": 3930, "san": 6119 } diff --git a/tests/harness/corpus-costs/7260-teardown-segments.ts.json b/tests/harness/corpus-costs/7260-teardown-segments.ts.json new file mode 100644 index 000000000..7f7ce1c2b --- /dev/null +++ b/tests/harness/corpus-costs/7260-teardown-segments.ts.json @@ -0,0 +1 @@ +{ "plain": 5343, "san": 11960 } diff --git a/tests/harness/corpus-costs/7264-array-dense-paired-loops.ts.json b/tests/harness/corpus-costs/7264-array-dense-paired-loops.ts.json new file mode 100644 index 000000000..f591a71f3 --- /dev/null +++ b/tests/harness/corpus-costs/7264-array-dense-paired-loops.ts.json @@ -0,0 +1 @@ +{ "plain": 2900, "san": 4200 } diff --git a/tests/harness/corpus-costs/7281-http-server-connect-tunnel.ts.json b/tests/harness/corpus-costs/7281-http-server-connect-tunnel.ts.json new file mode 100644 index 000000000..c35274f8c --- /dev/null +++ b/tests/harness/corpus-costs/7281-http-server-connect-tunnel.ts.json @@ -0,0 +1 @@ +{ "plain": 3700, "san": 3100 } diff --git a/tests/harness/corpus-costs/7282-http2-once-add-listener.ts.json b/tests/harness/corpus-costs/7282-http2-once-add-listener.ts.json new file mode 100644 index 000000000..9e5407833 --- /dev/null +++ b/tests/harness/corpus-costs/7282-http2-once-add-listener.ts.json @@ -0,0 +1 @@ +{ "plain": 4000, "san": 5100 } diff --git a/tests/harness/corpus-costs/7287-http-held-request-backpressure.ts.json b/tests/harness/corpus-costs/7287-http-held-request-backpressure.ts.json new file mode 100644 index 000000000..9622e52d1 --- /dev/null +++ b/tests/harness/corpus-costs/7287-http-held-request-backpressure.ts.json @@ -0,0 +1 @@ +{ "plain": 5600, "san": 5600 } diff --git a/tests/harness/corpus-costs/7380-cycle-deferred-folded-key/main.ts.json b/tests/harness/corpus-costs/7380-cycle-deferred-folded-key/main.ts.json new file mode 100644 index 000000000..d5d1c5f65 --- /dev/null +++ b/tests/harness/corpus-costs/7380-cycle-deferred-folded-key/main.ts.json @@ -0,0 +1 @@ +{ "plain": 2500, "san": 3500 } diff --git a/tests/harness/corpus-costs/7381-cycle-deferred-reached-tdz/main.ts.json b/tests/harness/corpus-costs/7381-cycle-deferred-reached-tdz/main.ts.json new file mode 100644 index 000000000..b24c0c970 --- /dev/null +++ b/tests/harness/corpus-costs/7381-cycle-deferred-reached-tdz/main.ts.json @@ -0,0 +1 @@ +{ "plain": 1800, "san": 2600 } diff --git a/tests/harness/corpus-costs/7382-cycle-static-this-tdz/main.ts.json b/tests/harness/corpus-costs/7382-cycle-static-this-tdz/main.ts.json new file mode 100644 index 000000000..b24c0c970 --- /dev/null +++ b/tests/harness/corpus-costs/7382-cycle-static-this-tdz/main.ts.json @@ -0,0 +1 @@ +{ "plain": 1800, "san": 2600 } diff --git a/tests/harness/corpus-costs/7383-cycle-class-expression-name-tdz/main.ts.json b/tests/harness/corpus-costs/7383-cycle-class-expression-name-tdz/main.ts.json new file mode 100644 index 000000000..b24c0c970 --- /dev/null +++ b/tests/harness/corpus-costs/7383-cycle-class-expression-name-tdz/main.ts.json @@ -0,0 +1 @@ +{ "plain": 1800, "san": 2600 } diff --git a/tests/harness/corpus-costs/7390-record-key-read-presence.ts.json b/tests/harness/corpus-costs/7390-record-key-read-presence.ts.json new file mode 100644 index 000000000..3f7384f20 --- /dev/null +++ b/tests/harness/corpus-costs/7390-record-key-read-presence.ts.json @@ -0,0 +1 @@ +{ "plain": 3500, "san": 6000 } diff --git a/tests/harness/corpus-costs/7404-frozen-records.ts.json b/tests/harness/corpus-costs/7404-frozen-records.ts.json new file mode 100644 index 000000000..765134581 --- /dev/null +++ b/tests/harness/corpus-costs/7404-frozen-records.ts.json @@ -0,0 +1 @@ +{ "plain": 4725, "san": 55833 } diff --git a/tests/harness/corpus-costs/7405-frozen-records-sloppy.cjs.json b/tests/harness/corpus-costs/7405-frozen-records-sloppy.cjs.json new file mode 100644 index 000000000..c7cd5ec8f --- /dev/null +++ b/tests/harness/corpus-costs/7405-frozen-records-sloppy.cjs.json @@ -0,0 +1 @@ +{ "plain": 3175, "san": 37248 } diff --git a/tests/harness/corpus-costs/7406-frozen-containers.ts.json b/tests/harness/corpus-costs/7406-frozen-containers.ts.json new file mode 100644 index 000000000..30bfa4b73 --- /dev/null +++ b/tests/harness/corpus-costs/7406-frozen-containers.ts.json @@ -0,0 +1 @@ +{ "plain": 3146, "san": 3439 } diff --git a/tests/harness/corpus-costs/7445-crypto-key-pair-options.mjs.json b/tests/harness/corpus-costs/7445-crypto-key-pair-options.mjs.json new file mode 100644 index 000000000..273d0c784 --- /dev/null +++ b/tests/harness/corpus-costs/7445-crypto-key-pair-options.mjs.json @@ -0,0 +1 @@ +{ "plain": 3596, "san": 5499 } diff --git a/tests/harness/corpus-costs/7446-crypto-rsa-key-pair-errors.ts.json b/tests/harness/corpus-costs/7446-crypto-rsa-key-pair-errors.ts.json new file mode 100644 index 000000000..ed5d25bcc --- /dev/null +++ b/tests/harness/corpus-costs/7446-crypto-rsa-key-pair-errors.ts.json @@ -0,0 +1 @@ +{ "plain": 7539, "san": 19613 } diff --git a/tests/harness/corpus-costs/7447-union-record-arm-width.mjs.json b/tests/harness/corpus-costs/7447-union-record-arm-width.mjs.json new file mode 100644 index 000000000..5648bd255 --- /dev/null +++ b/tests/harness/corpus-costs/7447-union-record-arm-width.mjs.json @@ -0,0 +1 @@ +{ "plain": 5785, "san": 5722 } diff --git a/tests/harness/corpus-costs/7448-union-record-arm-width.ts.json b/tests/harness/corpus-costs/7448-union-record-arm-width.ts.json new file mode 100644 index 000000000..c60a813c2 --- /dev/null +++ b/tests/harness/corpus-costs/7448-union-record-arm-width.ts.json @@ -0,0 +1 @@ +{ "plain": 3204, "san": 4052 } diff --git a/tests/harness/corpus-costs/7449-crypto-raw-keys.mjs.json b/tests/harness/corpus-costs/7449-crypto-raw-keys.mjs.json new file mode 100644 index 000000000..c8b39ec8a --- /dev/null +++ b/tests/harness/corpus-costs/7449-crypto-raw-keys.mjs.json @@ -0,0 +1 @@ +{ "plain": 7592, "san": 12217 } diff --git a/tests/harness/corpus-costs/7460-transform-writev.mjs.json b/tests/harness/corpus-costs/7460-transform-writev.mjs.json new file mode 100644 index 000000000..f6c21fd3d --- /dev/null +++ b/tests/harness/corpus-costs/7460-transform-writev.mjs.json @@ -0,0 +1 @@ +{ "plain": 2266, "san": 2991 } diff --git a/tests/harness/corpus-costs/7461-transform-writev.cjs.json b/tests/harness/corpus-costs/7461-transform-writev.cjs.json new file mode 100644 index 000000000..b5868c90b --- /dev/null +++ b/tests/harness/corpus-costs/7461-transform-writev.cjs.json @@ -0,0 +1 @@ +{ "plain": 1828, "san": 2413 } diff --git a/tests/harness/corpus-costs/7462-transform-writev.ts.json b/tests/harness/corpus-costs/7462-transform-writev.ts.json new file mode 100644 index 000000000..f8fd66c2a --- /dev/null +++ b/tests/harness/corpus-costs/7462-transform-writev.ts.json @@ -0,0 +1 @@ +{ "plain": 2178, "san": 2875 } diff --git a/tests/harness/corpus-costs/7463-stream-option-values.ts.json b/tests/harness/corpus-costs/7463-stream-option-values.ts.json new file mode 100644 index 000000000..0f66a4266 --- /dev/null +++ b/tests/harness/corpus-costs/7463-stream-option-values.ts.json @@ -0,0 +1 @@ +{ "plain": 1786, "san": 4192 } diff --git a/tests/harness/corpus-costs/7464-stream-option-values.mjs.json b/tests/harness/corpus-costs/7464-stream-option-values.mjs.json new file mode 100644 index 000000000..b609b128f --- /dev/null +++ b/tests/harness/corpus-costs/7464-stream-option-values.mjs.json @@ -0,0 +1 @@ +{ "plain": 1678, "san": 4502 } diff --git a/tests/harness/corpus-costs/7465-transform-callback-values.ts.json b/tests/harness/corpus-costs/7465-transform-callback-values.ts.json new file mode 100644 index 000000000..1c67bb43f --- /dev/null +++ b/tests/harness/corpus-costs/7465-transform-callback-values.ts.json @@ -0,0 +1 @@ +{ "plain": 1571, "san": 2074 } diff --git a/tests/harness/corpus-costs/7466-stream-construct-assign.mjs.json b/tests/harness/corpus-costs/7466-stream-construct-assign.mjs.json new file mode 100644 index 000000000..1c67bb43f --- /dev/null +++ b/tests/harness/corpus-costs/7466-stream-construct-assign.mjs.json @@ -0,0 +1 @@ +{ "plain": 1571, "san": 2074 } diff --git a/tests/harness/corpus-costs/7500-pipeline-callback-undefined.ts.json b/tests/harness/corpus-costs/7500-pipeline-callback-undefined.ts.json new file mode 100644 index 000000000..122c66b27 --- /dev/null +++ b/tests/harness/corpus-costs/7500-pipeline-callback-undefined.ts.json @@ -0,0 +1 @@ +{ "plain": 3783, "san": 4994 } diff --git a/tests/harness/corpus-costs/7501-pipeline-callback-undefined.mjs.json b/tests/harness/corpus-costs/7501-pipeline-callback-undefined.mjs.json new file mode 100644 index 000000000..d9c00a7de --- /dev/null +++ b/tests/harness/corpus-costs/7501-pipeline-callback-undefined.mjs.json @@ -0,0 +1 @@ +{ "plain": 2457, "san": 3243 } diff --git a/tests/harness/corpus-costs/7502-readline-cursor-undefined.ts.json b/tests/harness/corpus-costs/7502-readline-cursor-undefined.ts.json new file mode 100644 index 000000000..7a36db8f3 --- /dev/null +++ b/tests/harness/corpus-costs/7502-readline-cursor-undefined.ts.json @@ -0,0 +1 @@ +{ "plain": 2592, "san": 3421 } diff --git a/tests/harness/corpus-costs/7543-var-list-declarator-order.js.json b/tests/harness/corpus-costs/7543-var-list-declarator-order.js.json new file mode 100644 index 000000000..cc47657bd --- /dev/null +++ b/tests/harness/corpus-costs/7543-var-list-declarator-order.js.json @@ -0,0 +1 @@ +{ "plain": 4300, "san": 5900 } diff --git a/tests/harness/corpus-costs/7545-string-regex-checked-arguments.js.json b/tests/harness/corpus-costs/7545-string-regex-checked-arguments.js.json new file mode 100644 index 000000000..83b82ee18 --- /dev/null +++ b/tests/harness/corpus-costs/7545-string-regex-checked-arguments.js.json @@ -0,0 +1 @@ +{ "plain": 4900, "san": 4600 } diff --git a/tests/harness/corpus-costs/7548-cjs-receiverless-builtin-calls/main.cjs.json b/tests/harness/corpus-costs/7548-cjs-receiverless-builtin-calls/main.cjs.json new file mode 100644 index 000000000..8a603b2f4 --- /dev/null +++ b/tests/harness/corpus-costs/7548-cjs-receiverless-builtin-calls/main.cjs.json @@ -0,0 +1 @@ +{ "plain": 12300, "san": 13600 } diff --git a/tests/harness/corpus-costs/7549-cjs-minified-esmodule-stamp/main.cjs.json b/tests/harness/corpus-costs/7549-cjs-minified-esmodule-stamp/main.cjs.json new file mode 100644 index 000000000..d00a49ec8 --- /dev/null +++ b/tests/harness/corpus-costs/7549-cjs-minified-esmodule-stamp/main.cjs.json @@ -0,0 +1 @@ +{ "plain": 16000, "san": 21200 } diff --git a/tests/harness/corpus-costs/7553-var-for-in-of-zero-iterations.js.json b/tests/harness/corpus-costs/7553-var-for-in-of-zero-iterations.js.json new file mode 100644 index 000000000..566096d24 --- /dev/null +++ b/tests/harness/corpus-costs/7553-var-for-in-of-zero-iterations.js.json @@ -0,0 +1 @@ +{ "plain": 19400, "san": 25900 } diff --git a/tests/harness/corpus-costs/7580-int32-arrays-across-calls.ts.json b/tests/harness/corpus-costs/7580-int32-arrays-across-calls.ts.json new file mode 100644 index 000000000..7c9d20632 --- /dev/null +++ b/tests/harness/corpus-costs/7580-int32-arrays-across-calls.ts.json @@ -0,0 +1 @@ +{ "plain": 3100, "san": 3500 } diff --git a/tests/harness/corpus-costs/7620-js-class-view-descriptors.js.json b/tests/harness/corpus-costs/7620-js-class-view-descriptors.js.json new file mode 100644 index 000000000..ef439d096 --- /dev/null +++ b/tests/harness/corpus-costs/7620-js-class-view-descriptors.js.json @@ -0,0 +1 @@ +{ "plain": 3000, "san": 4500 } diff --git a/tests/harness/corpus-costs/7621-js-class-field-methods.js.json b/tests/harness/corpus-costs/7621-js-class-field-methods.js.json new file mode 100644 index 000000000..ef439d096 --- /dev/null +++ b/tests/harness/corpus-costs/7621-js-class-field-methods.js.json @@ -0,0 +1 @@ +{ "plain": 3000, "san": 4500 } diff --git a/tests/harness/corpus-costs/7640-child-spawn-before-fast-exit.ts.json b/tests/harness/corpus-costs/7640-child-spawn-before-fast-exit.ts.json new file mode 100644 index 000000000..b56ad904d --- /dev/null +++ b/tests/harness/corpus-costs/7640-child-spawn-before-fast-exit.ts.json @@ -0,0 +1 @@ +{ "plain": 2600, "san": 4000 } diff --git a/tests/harness/corpus-costs/7642-element-assignment-order.ts.json b/tests/harness/corpus-costs/7642-element-assignment-order.ts.json new file mode 100644 index 000000000..9f67659a1 --- /dev/null +++ b/tests/harness/corpus-costs/7642-element-assignment-order.ts.json @@ -0,0 +1 @@ +{ "plain": 1500, "san": 2500 } diff --git a/tests/harness/corpus-costs/7733-property-bag-late-prototype.js.json b/tests/harness/corpus-costs/7733-property-bag-late-prototype.js.json new file mode 100644 index 000000000..706d1a2a5 --- /dev/null +++ b/tests/harness/corpus-costs/7733-property-bag-late-prototype.js.json @@ -0,0 +1 @@ +{ "plain": 4400, "san": 3900 } diff --git a/tests/harness/corpus-costs/7734-caught-instanceof-class-binding.js.json b/tests/harness/corpus-costs/7734-caught-instanceof-class-binding.js.json new file mode 100644 index 000000000..71aa92bd3 --- /dev/null +++ b/tests/harness/corpus-costs/7734-caught-instanceof-class-binding.js.json @@ -0,0 +1 @@ +{ "plain": 3300, "san": 3600 } diff --git a/tests/harness/corpus-costs/7738-private-logical-assignment-untyped.js.json b/tests/harness/corpus-costs/7738-private-logical-assignment-untyped.js.json new file mode 100644 index 000000000..d61b17a5d --- /dev/null +++ b/tests/harness/corpus-costs/7738-private-logical-assignment-untyped.js.json @@ -0,0 +1 @@ +{ "plain": 5500, "san": 8800 } diff --git a/tests/harness/corpus-costs/7740-handle-method-values.js.json b/tests/harness/corpus-costs/7740-handle-method-values.js.json new file mode 100644 index 000000000..d18e4dddd --- /dev/null +++ b/tests/harness/corpus-costs/7740-handle-method-values.js.json @@ -0,0 +1 @@ +{ "plain": 9800, "san": 12000 } diff --git a/tests/harness/corpus-costs/7741-abort-signal-listener-value.js.json b/tests/harness/corpus-costs/7741-abort-signal-listener-value.js.json new file mode 100644 index 000000000..e028a5bb1 --- /dev/null +++ b/tests/harness/corpus-costs/7741-abort-signal-listener-value.js.json @@ -0,0 +1 @@ +{ "plain": 4000, "san": 7500 } diff --git a/tests/harness/corpus-costs/7742-response-clone-tee.ts.json b/tests/harness/corpus-costs/7742-response-clone-tee.ts.json new file mode 100644 index 000000000..0c31a05f7 --- /dev/null +++ b/tests/harness/corpus-costs/7742-response-clone-tee.ts.json @@ -0,0 +1 @@ +{ "plain": 3000, "san": 4000 } diff --git a/tests/harness/corpus-costs/7743-transform-stream-identity.ts.json b/tests/harness/corpus-costs/7743-transform-stream-identity.ts.json new file mode 100644 index 000000000..0c31a05f7 --- /dev/null +++ b/tests/harness/corpus-costs/7743-transform-stream-identity.ts.json @@ -0,0 +1 @@ +{ "plain": 3000, "san": 4000 } diff --git a/tests/harness/corpus-costs/7747-text-codec-untyped-slot.js.json b/tests/harness/corpus-costs/7747-text-codec-untyped-slot.js.json new file mode 100644 index 000000000..a537f9ede --- /dev/null +++ b/tests/harness/corpus-costs/7747-text-codec-untyped-slot.js.json @@ -0,0 +1 @@ +{ "plain": 2400, "san": 6500 } diff --git a/tests/harness/corpus-costs/7748-npm-any-bytes-receiver/main.ts.json b/tests/harness/corpus-costs/7748-npm-any-bytes-receiver/main.ts.json new file mode 100644 index 000000000..197263cad --- /dev/null +++ b/tests/harness/corpus-costs/7748-npm-any-bytes-receiver/main.ts.json @@ -0,0 +1 @@ +{ "plain": 4600, "san": 7300 } diff --git a/tests/harness/corpus-costs/7749-text-codec-unknown-roundtrip.ts.json b/tests/harness/corpus-costs/7749-text-codec-unknown-roundtrip.ts.json new file mode 100644 index 000000000..aa8658007 --- /dev/null +++ b/tests/harness/corpus-costs/7749-text-codec-unknown-roundtrip.ts.json @@ -0,0 +1 @@ +{ "plain": 2200, "san": 3600 } diff --git a/tests/harness/corpus-costs/7790-js-class-integrity-layouts.js.json b/tests/harness/corpus-costs/7790-js-class-integrity-layouts.js.json new file mode 100644 index 000000000..af44b31af --- /dev/null +++ b/tests/harness/corpus-costs/7790-js-class-integrity-layouts.js.json @@ -0,0 +1 @@ +{ "plain": 4400, "san": 5200 } diff --git a/tests/harness/corpus-costs/7791-js-class-accessor-layouts.js.json b/tests/harness/corpus-costs/7791-js-class-accessor-layouts.js.json new file mode 100644 index 000000000..eb36dcefe --- /dev/null +++ b/tests/harness/corpus-costs/7791-js-class-accessor-layouts.js.json @@ -0,0 +1 @@ +{ "plain": 5100, "san": 4700 } diff --git a/tests/harness/corpus-costs/7792-untyped-property-reads-mixed-receivers.js.json b/tests/harness/corpus-costs/7792-untyped-property-reads-mixed-receivers.js.json new file mode 100644 index 000000000..d4cd0dbbb --- /dev/null +++ b/tests/harness/corpus-costs/7792-untyped-property-reads-mixed-receivers.js.json @@ -0,0 +1 @@ +{ "plain": 4500, "san": 5000 } diff --git a/tests/harness/corpus-costs/7793-untyped-key-writes.js.json b/tests/harness/corpus-costs/7793-untyped-key-writes.js.json new file mode 100644 index 000000000..eb1b1f127 --- /dev/null +++ b/tests/harness/corpus-costs/7793-untyped-key-writes.js.json @@ -0,0 +1 @@ +{ "plain": 4300, "san": 4800 } diff --git a/tests/harness/corpus-costs/7794-untyped-array-iteration.js.json b/tests/harness/corpus-costs/7794-untyped-array-iteration.js.json new file mode 100644 index 000000000..6c25b05fd --- /dev/null +++ b/tests/harness/corpus-costs/7794-untyped-array-iteration.js.json @@ -0,0 +1 @@ +{ "plain": 3100, "san": 3700 } diff --git a/tests/harness/corpus-costs/7795-untyped-global-constructors.js.json b/tests/harness/corpus-costs/7795-untyped-global-constructors.js.json new file mode 100644 index 000000000..05dfbf1a6 --- /dev/null +++ b/tests/harness/corpus-costs/7795-untyped-global-constructors.js.json @@ -0,0 +1 @@ +{ "plain": 1200, "san": 1200 } diff --git a/tests/harness/corpus-costs/7981-typed-array-paired-loops.ts.json b/tests/harness/corpus-costs/7981-typed-array-paired-loops.ts.json new file mode 100644 index 000000000..c93369880 --- /dev/null +++ b/tests/harness/corpus-costs/7981-typed-array-paired-loops.ts.json @@ -0,0 +1 @@ +{ "plain": 3500, "san": 4500 } diff --git a/tests/harness/corpus-costs/class-brand-probe-keys.ts.json b/tests/harness/corpus-costs/class-brand-probe-keys.ts.json new file mode 100644 index 000000000..9f464b312 --- /dev/null +++ b/tests/harness/corpus-costs/class-brand-probe-keys.ts.json @@ -0,0 +1 @@ +{ "plain": 8500, "san": 30000 } diff --git a/tests/harness/corpus-costs/fetch-headers-copy-cache.ts.json b/tests/harness/corpus-costs/fetch-headers-copy-cache.ts.json new file mode 100644 index 000000000..6057503d9 --- /dev/null +++ b/tests/harness/corpus-costs/fetch-headers-copy-cache.ts.json @@ -0,0 +1 @@ +{ "plain": 6400, "san": 9200 } diff --git a/tests/harness/corpus-costs/string-last-index-of-ascii.ts.json b/tests/harness/corpus-costs/string-last-index-of-ascii.ts.json new file mode 100644 index 000000000..a970c2f16 --- /dev/null +++ b/tests/harness/corpus-costs/string-last-index-of-ascii.ts.json @@ -0,0 +1 @@ +{ "plain": 6000, "san": 9000 } diff --git a/tests/harness/errors.test.ts b/tests/harness/errors.test.ts index 319e7329f..86cab2247 100644 --- a/tests/harness/errors.test.ts +++ b/tests/harness/errors.test.ts @@ -849,7 +849,7 @@ describe("unimplemented cipher names", () => { const r = await compileAndRun( "cipher-computed-name", `import { createCipheriv } from "node:crypto"; -const mode = ["c", "c", "m"].join(""); +const mode = ["o", "c", "b"].join(""); try { createCipheriv("aes-128-" + mode, Buffer.alloc(16), Buffer.alloc(12)); } catch (error) { @@ -866,16 +866,43 @@ try { ); expect(r.exitCode).toBe(0); expect(r.stdout).toBe( - "Error SC2020 The cipher 'aes-128-ccm' has no native implementation; the supported ciphers are " + - "aes-128, aes-192 and aes-256 in cbc, ctr, gcm, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede3 in " + - "cbc, ecb, cfb, cfb8, cfb1 and ofb modes, des-ede in cbc, ecb, cfb and ofb modes, chacha20 and " + - "chacha20-poly1305\n" + + "Error SC2020 The cipher 'aes-128-ocb' has no native implementation; the supported ciphers are " + + "aes-128, aes-192 and aes-256 in cbc, ctr, gcm, ccm, ecb, cfb, cfb8, cfb1 and ofb modes and their " + + "wrap, wrap-pad, wrap-inv and wrap-pad-inv key wraps, des-ede3 in cbc, ecb, cfb, cfb8, cfb1 and " + + "ofb modes, des-ede in cbc, ecb, cfb and ofb modes, chacha20 and chacha20-poly1305\n" + "Error ERR_CRYPTO_UNKNOWN_CIPHER Unknown cipher\n", ); expect(r.stderr).toBe(""); }); }); +// getCipherInfo's result is Node's info object; CipherInfo declares nid as +// required, and Node reports the SIV ciphers without one. +describe("getCipherInfo through CipherInfo", () => { + test("a typed read of a missing nid throws a TypeError", async () => { + const r = await compileAndRun( + "cipher-info-no-nid", + `import { getCipherInfo } from "node:crypto"; +const siv = getCipherInfo("aes-128-siv")!; +console.log(JSON.stringify(siv), getCipherInfo("aes-128-gcm")?.nid); +try { + const nid: number = siv.nid; + console.log(nid); +} catch (error) { + const e = error as Error; + console.log(e.name, e.message); +} +`, + ); + expect(r.exitCode).toBe(0); + expect(r.stdout).toBe( + '{"mode":"siv","name":"aes-128-siv","blockSize":1,"keyLength":32} 895\n' + + "TypeError expected number at $, got undefined\n", + ); + expect(r.stderr).toBe(""); + }); +}); + // new tls.TLSSocket reads a runtime options record: the client mode, whose // handshake Node never starts, and members without a lowering throw the // fence, as do sockets it cannot take over (a second wrap, and @@ -1124,6 +1151,41 @@ try { }); }); +// getCipherInfo reads keyLength and ivLength through getters, which a +// class instance's accessors are not visible to from the runtime, so a +// class that defines either throws a coded SC2020 error. +describe("getCipherInfo options whose class defines keyLength or ivLength", () => { + test("throw SC2020 after the argument checks", async () => { + const r = await compileAndRun( + "cipher-info-class-options", + `import { getCipherInfo } from "node:crypto"; +class Options { + get keyLength(): number { + return 17; + } +} +const options: any = new Options(); +const names: any[] = ["aes-128-cbc", 1.5]; +for (const name of names) { + try { + getCipherInfo(name, options); + } catch (error) { + const e = error as Error & { code?: string }; + console.log(e.name, e.code, e.message); + } +} +`, + ); + expect(r.exitCode).toBe(0); + expect(r.stdout).toBe( + "Error SC2020 getCipherInfo options whose class defines keyLength or ivLength have no lowering; " + + "pass an object with those properties\n" + + 'RangeError ERR_OUT_OF_RANGE The value of "nameOrNid" is out of range. It must be an integer. Received 1.5\n', + ); + expect(r.stderr).toBe(""); + }); +}); + // After a Hash's stream flushes its digest, Node copies the finished // OpenSSL context, whose later updates and digests fail with OpenSSL // error-queue codes. That copy throws a coded SC2020 error. diff --git a/tests/harness/library-mode.test.ts b/tests/harness/library-mode.test.ts index 0d4052952..b60c3ac54 100644 --- a/tests/harness/library-mode.test.ts +++ b/tests/harness/library-mode.test.ts @@ -50,8 +50,8 @@ * no-teachings default is pinned by K5/K7 over * the teachings-free traps profile * K13 npm-posture bare npm specifiers are static-or-refuse: an - * eligible package (own .d.ts, unminified JS, - * no transform markers) compiles statically + * eligible package (own .d.ts, no transform + * markers) compiles statically * into the library graph, its own npm deps * judged by the same bar; every ineligible one * refuses SC4020 naming the failed bar — never @@ -363,6 +363,17 @@ globals: undefined fenced expect(run.stdout).toBe(SESSION + SESSION + SESSION); }); + test("K4: untyped property caches start empty in every session", async () => { + // The shapes a session's cache entries name are freed when it ends; + // the next session's sites must not find those entries. + const { archive, outDir } = await buildLibrary("js-ic-sessions", emission); + const probe = buildProbe("js-ic-sessions", archive, outDir); + const run = runProbe(probe); + expect(run.signal).toBeNull(); + expect(run.status).toBe(0); + expect(run.stdout).toBe("run: 20000\n".repeat(3)); + }); + /* ── K5/K6/K7: the trap channel ──────────────────────────────────────── */ test("K5: a trap delivers to the sink exactly once, host survives", async () => { @@ -582,6 +593,15 @@ survived, sink_calls=1 expect(run.stdout).toBe(SESSION + SESSION + SESSION); }); + platformTest("K10: property caches across sessions under ASan", async () => { + const { archive, outDir } = await buildLibrary("js-ic-sessions", emission, { sanitize: true }); + const probe = buildProbe("js-ic-sessions", archive, outDir, { sanitize: true }); + const run = runProbe(probe); + expect(run.signal).toBeNull(); + expect(run.status).toBe(0); + expect(run.stdout).toBe("run: 20000\n".repeat(3)); + }); + platformTest("K10: K5/K7 under ASan", async () => { const { archive, outDir } = await buildLibrary("traps", emission, { sanitize: true }); const probe = buildProbe("traps", archive, outDir, { sanitize: true }); @@ -746,11 +766,11 @@ describe("K9: library-mode refusals", () => { * synthesized per test like K9's. Emission never engages — SC4020 is a * frontend verdict — so one lane suffices. */ let npmRefusalCounter = 0; -async function npmRefusal( +async function compileNpmFixture( entryFile: string, exports: unknown[], profilePatch: Record = {}, -): Promise<{ code: string; message: string; hint?: string; note?: string; file: string }[]> { +): Promise>> { const outDir = join(cacheDir, `npm-refusal-${npmRefusalCounter++}`); mkdirSync(outDir, { recursive: true }); const profile = { @@ -770,7 +790,15 @@ async function npmRefusal( }; const profilePath = join(outDir, "profile.json"); writeFileSync(profilePath, JSON.stringify(profile)); - const result = await compileLibrary({ profilePath, outDir }); + return compileLibrary({ profilePath, outDir }); +} + +async function npmRefusal( + entryFile: string, + exports: unknown[], + profilePatch: Record = {}, +): Promise<{ code: string; message: string; hint?: string; note?: string; file: string }[]> { + const result = await compileNpmFixture(entryFile, exports, profilePatch); expect(result.ok).toBe(false); if (result.ok) throw new Error("unreachable"); return result.diagnostics.map((d) => ({ @@ -795,13 +823,11 @@ describe("K13: library-mode npm refusals (SC4020, static-or-refuse)", () => { expect(diags[0]!.file.endsWith("lib-markers.ts")).toBe(true); }); - test("minified shipped JS refuses", async () => { - const diags = await npmRefusal("lib-minified.ts", [ + test("minified shipped JS compiles", async () => { + const result = await compileNpmFixture("lib-minified.ts", [ { export: "f", symbol: "kx_f", params: ["f64", "f64"], returns: "f64" }, ]); - expect(diags.map((d) => d.code)).toEqual(["SC4020"]); - expect(diags[0]!.message).toContain("'slimmed'"); - expect(diags[0]!.message).toContain("looks minified"); + expect(result.ok ? [] : result.diagnostics.map((d) => d.message)).toEqual([]); }); test("no own .d.ts refuses by name — never the generic import fence", async () => { diff --git a/tests/harness/library-static-types.test.ts b/tests/harness/library-static-types.test.ts index dfbd70bf9..36e2befac 100644 --- a/tests/harness/library-static-types.test.ts +++ b/tests/harness/library-static-types.test.ts @@ -81,7 +81,8 @@ test.each(["sample", "@scope/sample"])( expect(actual).toBe(reference); expect(actual).toBe("22\n"); } - // Opting in to one runtime package does not admit its untyped dependency. + // A dependency the package's JavaScript imports compiles without + // declarations of its own: no declaration checks that import. const dependency = join(dir, "node_modules/unchecked-dependency"); await mkdir(dependency, { recursive: true }); await writeFile( @@ -96,14 +97,18 @@ test.each(["sample", "@scope/sample"])( join(pkg, "index.js"), "import { adjust } from 'unchecked-dependency'; export function calculate(value) { return adjust(value) * 3 + 1; }\n", ); - const refused = await compileLibrary({ profilePath, outDir: join(dir, "refused") }); - expect(refused.ok).toBe(false); - if (!refused.ok) - expect( - refused.diagnostics.some( - (d) => d.code === "SC4020" && d.message.includes("unchecked-dependency"), - ), - ).toBe(true); + const withDependency = await compileLibrary({ profilePath, outDir: join(dir, "dependency") }); + if (!withDependency.ok) throw new Error(JSON.stringify(withDependency.diagnostics)); + if (process.env["SCRIPTC_PORTABLE_ONLY"] !== "1") { + const source = join(dir, "dependency-host.c"), + binary = join(dir, "dependency-host"); + await writeFile( + source, + '#include \nextern void st_init(void); extern double st_result(void);\nint main(void) { st_init(); printf("%.0f\\n", st_result()); }\n', + ); + execFileSync("clang", [source, withDependency.archivePath, "-lm", "-o", binary]); + expect(execFileSync(binary, { encoding: "utf8" })).toBe("22\n"); + } expect(await readFile(join(types, "index.d.ts"), "utf8")).toBe(declaration); } finally { await rm(dir, { recursive: true, force: true }); diff --git a/tests/harness/npm-static.test.ts b/tests/harness/npm-static.test.ts index 7ee76307f..388da2138 100644 --- a/tests/harness/npm-static.test.ts +++ b/tests/harness/npm-static.test.ts @@ -1490,6 +1490,24 @@ describe(`npm-static pilots${sanitize ? " (sanitized)" : ""}`, () => { ]); }, 180_000); + // 7546: a one-line minified ES module: a var statement whose later + // declarators spread earlier ones, regexes read through an untyped rules + // table, and a function instantiated with typed and untyped arguments. + test("the minified module 7546-minified-esm-rules compiles statically and matches Node", async () => { + await expectNodeParity(join(fixturesRoot, "npm/cases/7546-minified-esm-rules/main.ts"), [ + "gtminimark", + ]); + }, 180_000); + + // 7547: an identifier-minified esbuild CommonJS bundle: the __commonJS + // helper and chunk parameters renamed, { value: !0 } stamps, and + // (0, ns.member)(...) calls of builtin module functions. + test("the identifier-minified bundle 7547-minified-cjs-chunks compiles statically and matches Node", async () => { + await expectNodeParity(join(fixturesRoot, "npm/cases/7547-minified-cjs-chunks/main.ts"), [ + "gtminicache", + ]); + }, 180_000); + // A whitespace/syntax-minified bundle whose helpers keep their names is // judged by the recognizers, not refused up front as minified. test("--npm-static=auto opts a minified bundle with named helpers in", () => { diff --git a/tests/harness/self-hosting-native-toolchain.test.ts b/tests/harness/self-hosting-native-toolchain.test.ts index 091d3f0b5..c5476328e 100644 --- a/tests/harness/self-hosting-native-toolchain.test.ts +++ b/tests/harness/self-hosting-native-toolchain.test.ts @@ -41,6 +41,7 @@ const features: NativeLinkFeatures = { dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, @@ -108,7 +109,7 @@ for (const backend of ["llvm"] as const) { object_format: target.objectFormat, minimum_os: target.minimumOs, }, - runtime_abi: { version: 36, marker: "scr_runtime_abi_v36" }, + runtime_abi: { version: 37, marker: "scr_runtime_abi_v37" }, compiler: { command: "fixture", identity: "fixture", target: target.llvmTriple }, macros: { executable: [], excluded: [], sanitizer: "external-toolchain-required" }, flavors: { diff --git a/tests/harness/self-hosting-runtime-pack.test.ts b/tests/harness/self-hosting-runtime-pack.test.ts index 57857d642..64ddd0213 100644 --- a/tests/harness/self-hosting-runtime-pack.test.ts +++ b/tests/harness/self-hosting-runtime-pack.test.ts @@ -42,6 +42,7 @@ const base: NativeLinkFeatures = { dgram: false, dnsPromises: false, dgramDyn: false, + textCodecDyn: false, watch: false, foreignFfi: false, nodeTest: false, diff --git a/tests/harness/teardown-segments.test.ts b/tests/harness/teardown-segments.test.ts new file mode 100644 index 000000000..95a2d2795 --- /dev/null +++ b/tests/harness/teardown-segments.test.ts @@ -0,0 +1,97 @@ +/* Teardown segments (llvm/shapes.ts): a typed teardown of a type with + * children of its own type keeps the blocks it frees in a segment and frees + * the segment onto the free list when the chain's first teardown returns. + * Corpus program 7260 pins what such programs print; this suite checks the + * memory side natively, which Node parity cannot: a program that rebuilds + * large trees and lists for a hundred generations reuses every block of the + * last generation, so its resident set stays at about one generation's + * worth. A segment that lost blocks would grow it by a generation each time. + * The sanitized lane frees through the system allocator (no segments) and + * keeps freed memory in quarantine, so the bound is checked in the plain + * lane only. */ +import { execFile } from "node:child_process"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { expect, test } from "vitest"; +import { compile } from "@scriptc/compiler"; + +const exec = promisify(execFile); +const sanitize = process.env["SCRIPTC_SAN"] === "1"; + +const PROGRAM = ` +interface Node { + left: Node | null; + right: Node | null; + value: number; +} +function tree(depth: number): Node { + return depth === 0 + ? { left: null, right: null, value: 1 } + : { left: tree(depth - 1), right: tree(depth - 1), value: depth }; +} +function size(node: Node | null): number { + return node === null ? 0 : 1 + size(node.left) + size(node.right); +} +class Branch { + readonly left: Branch | null; + readonly right: Branch | null; + constructor(left: Branch | null, right: Branch | null) { + this.left = left; + this.right = right; + } +} +function branches(depth: number): Branch { + return depth === 0 ? new Branch(null, null) : new Branch(branches(depth - 1), branches(depth - 1)); +} +function count(branch: Branch | null): number { + return branch === null ? 0 : 1 + count(branch.left) + count(branch.right); +} +interface Cell { + next: Cell | null; + n: number; +} +function cells(n: number): Cell | null { + let head: Cell | null = null; + for (let i = 0; i < n; i++) head = { next: head, n: i }; + return head; +} +let total = 0; +let settled = 0; +for (let generation = 0; generation < 100; generation++) { + total += size(tree(16)) + count(branches(15)); + for (let c = cells(100000); c !== null; c = c.next) total += 1; + if (generation === 2) settled = process.memoryUsage.rss(); +} +const growth = Math.round((process.memoryUsage.rss() - settled) / (1024 * 1024)); +console.log(total, growth); +`; + +test.skipIf(sanitize)( + "rebuilding large trees and lists reuses every block their teardowns free", + { timeout: 300_000 }, + async () => { + const dir = await mkdtemp(join(tmpdir(), "scriptc-segments-")); + try { + const source = join(dir, "churn.ts"); + await writeFile(source, PROGRAM); + const result = await compile(source, { + backend: "llvm", + outDir: dir, + outPath: join(dir, "churn"), + }); + expect(result.ok, JSON.stringify(result.diagnostics)).toBe(true); + if (!result.ok) return; + const { stdout } = await exec(result.binaryPath, [], { timeout: 120_000 }); + const [total, growth] = stdout.trim().split(" ").map(Number); + // 100 generations of 131071 + 65535 nodes and 100000 cells. + expect(total).toBe(100 * (131071 + 65535 + 100000)); + // One generation is about 10 MiB; a lost generation per round would be + // hundreds. + expect(growth).toBeLessThan(8); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }, +); diff --git a/tests/harness/test262-runner.test.ts b/tests/harness/test262-runner.test.ts index 6f6df35d4..3ddf3ca84 100644 --- a/tests/harness/test262-runner.test.ts +++ b/tests/harness/test262-runner.test.ts @@ -101,6 +101,26 @@ test("the property helper only admits direct inline descriptor checks", () => { } }); +test("the typed array helper is admitted without the %TypedArray% intrinsic", () => { + const text = source( + "includes: [testTypedArray.js]", + "testWithTypedArrayConstructors(function(TA) { assert.sameValue(new TA(1).length, 1); });", + ); + expect(exclusion(text, metadata(text), "strict")).toBeUndefined(); + const prepared = prepare(text, false, undefined, "strict", ["testTypedArray.js"]); + expect(prepared).toContain("function testWithTypedArrayConstructors("); + expect(prepared).not.toContain("Object.getPrototypeOf(Int8Array)"); + for (const body of [ + "assert.sameValue(typeof TypedArray, 'function');", + "testWithTypedArrayConstructors(function(TA) { assert.sameValue(Object.getPrototypeOf(TA), TypedArray); });", + ]) { + const named = source("includes: [testTypedArray.js]", body); + expect(exclusion(named, metadata(named), "strict")).toBe("harness:TypedArray-intrinsic"); + } + const other = source("includes: [testTypedArray.js, detachArrayBuffer.js]"); + expect(exclusion(other, metadata(other), "strict")).toBe("harness-includes:detachArrayBuffer.js"); +}); + test("receiver-bound this is admitted without adapting script-level this", () => { for (const [body, expected] of [ ["class C { read() { return this.value; } }", undefined], diff --git a/tests/harness/test262.test.ts b/tests/harness/test262.test.ts index a36fa2cdf..f1b616578 100644 --- a/tests/harness/test262.test.ts +++ b/tests/harness/test262.test.ts @@ -19,6 +19,10 @@ const upstreamHarness = ["assert.js", "sta.js"] .map((name) => readFileSync(join(vendorRoot, "harness", name), "utf8")) .join("\n"); const upstreamPropertyHelper = readFileSync(join(vendorRoot, "harness/propertyHelper.js"), "utf8"); +const upstreamTypedArrayHelper = readFileSync( + join(vendorRoot, "harness/testTypedArray.js"), + "utf8", +); function runUpstream( source: string, @@ -29,6 +33,8 @@ function runUpstream( runInContext(upstreamHarness, context, { timeout: 5000 }); if (includes.includes("propertyHelper.js")) runInContext(upstreamPropertyHelper, context, { timeout: 5000 }); + if (includes.includes("testTypedArray.js")) + runInContext(upstreamTypedArrayHelper, context, { timeout: 5000 }); runInContext(variant === "strict" ? `"use strict";\n${source}` : source, context, { timeout: 5000, }); @@ -52,6 +58,8 @@ async function runUpstreamAsync( runInContext(upstreamHarness, context, { timeout: 5000 }); if (includes.includes("propertyHelper.js")) runInContext(upstreamPropertyHelper, context, { timeout: 5000 }); + if (includes.includes("testTypedArray.js")) + runInContext(upstreamTypedArrayHelper, context, { timeout: 5000 }); runInContext(variant === "strict" ? `"use strict";\n${source}` : source, context, { timeout: 5000, }); @@ -375,6 +383,67 @@ if (propertyControls.length > 0) } }); +const typedArrayControls = shardSelect( + [ + { + name: "every constructor with array, array-like and buffer arguments", + status: "pass", + source: + "testWithTypedArrayConstructors(function(TA, makeCtorArg) { var sample = new TA(makeCtorArg([40, 41, 42])); assert.sameValue(sample.length, 3); assert.sameValue(sample[2], 42); }, null, ['passthrough', 'arraylike', 'arraybuffer']);", + }, + { + name: "failing element assertion", + status: "fail", + source: + "testWithTypedArrayConstructors(function(TA, makeCtorArg) { var sample = new TA(makeCtorArg([40, 41, 42])); assert.sameValue(sample[2], 43); });", + }, + { + name: "failure in one constructor", + status: "fail", + source: + "testWithTypedArrayConstructors(function(TA) { assert(TA !== Int16Array, 'Int16Array'); });", + }, + { + name: "passthrough factories visit each constructor once", + status: "pass", + source: + "var count = 0; testWithTypedArrayConstructors(function(TA, makeCtorArg) { assert.sameValue(new TA(makeCtorArg(4)).length, 4); count++; }, null, ['passthrough']); assert.sameValue(count, typedArrayConstructors.length);", + }, + { + name: "explicit constructor list", + status: "pass", + source: + "var names = []; testWithTypedArrayConstructors(function(TA) { names.push(TA.name); }, [Uint8Array, Float64Array], ['passthrough']); assert.compareArray(names, ['Uint8Array', 'Float64Array']);", + }, + { + name: "float constructor classification", + status: "pass", + source: + "assert(isFloatTypedArrayConstructor(Float32Array)); assert(!isFloatTypedArrayConstructor(Int8Array)); assert.sameValue(floatTypedArrayConstructorPrecision(Float64Array), 'double');", + }, + { + name: "unknown argument factory feature", + status: "fail", + source: "testWithTypedArrayConstructors(function() {}, null, ['unknown']);", + }, + ], + (item) => `typed-array:${item.name}`, +); + +if (typedArrayControls.length > 0) + describe(`Test262 typed array helper contract${shardSuffix()}`, () => { + for (const control of typedArrayControls) { + test(control.name, async () => { + const includes = ["compareArray.js", "testTypedArray.js"]; + if (control.status === "fail") + expect(() => runUpstream(control.source, "strict", includes)).toThrow(); + else expect(() => runUpstream(control.source, "strict", includes)).not.toThrow(); + const result = await runSource(control.source, { sanitize, includes }); + expect(result, JSON.stringify(result, null, 2)).toMatchObject({ status: control.status }); + }); + } + }); + const asyncControls = shardSelect( [ { diff --git a/tests/library-mode/js-ic-sessions/lib.ts b/tests/library-mode/js-ic-sessions/lib.ts new file mode 100644 index 000000000..ebea59a54 --- /dev/null +++ b/tests/library-mode/js-ic-sessions/lib.ts @@ -0,0 +1,6 @@ +// The library's typed export over untyped code (work.mjs). +import { runAll } from "./work.mjs"; + +export function run(): number { + return runAll() as number; +} diff --git a/tests/library-mode/js-ic-sessions/probe.c b/tests/library-mode/js-ic-sessions/probe.c new file mode 100644 index 000000000..7dbcb56bf --- /dev/null +++ b/tests/library-mode/js-ic-sessions/probe.c @@ -0,0 +1,25 @@ +/* Three sessions of a library whose untyped code reads and writes through + * property caches: each prints the same result, and under the sanitize + * flavor no session reads a cache entry the previous session freed. */ +#include +#include + +extern void kj_init(void); +extern void kj_set_panic_sink(void (*fn)(void *, const uint8_t *, size_t, uint64_t), void *ctx); +extern void kj_collect(void); +extern double kj_run(void); + +static void sink(void *ctx, const uint8_t *msg, size_t len, uint64_t addr) { + (void)ctx; (void)addr; + printf("UNEXPECTED SINK: %.*s", (int)len, (const char *)msg); +} + +int main(void) { + kj_set_panic_sink(sink, NULL); + for (int session = 0; session < 3; session++) { + kj_init(); + printf("run: %.0f\n", kj_run()); + kj_collect(); + } + return 0; +} diff --git a/tests/library-mode/js-ic-sessions/profile.json b/tests/library-mode/js-ic-sessions/profile.json new file mode 100644 index 000000000..fd802b42c --- /dev/null +++ b/tests/library-mode/js-ic-sessions/profile.json @@ -0,0 +1,14 @@ +{ + "profile_format": 1, + "name": "conformance-js-ic-sessions", + "entry": "lib.ts", + "emission": "llvm", + "abi": { + "prefix": "kj_", + "init_symbol": "kj_init", + "sink_register_symbol": "kj_set_panic_sink", + "collect_symbol": "kj_collect", + "result_reset_symbol": null + }, + "exports": [{ "export": "run", "symbol": "kj_run", "params": [], "returns": "f64" }] +} diff --git a/tests/library-mode/js-ic-sessions/work.mjs b/tests/library-mode/js-ic-sessions/work.mjs new file mode 100644 index 000000000..5c1afcd18 --- /dev/null +++ b/tests/library-mode/js-ic-sessions/work.mjs @@ -0,0 +1,26 @@ +// Dynamic-tier property caches across library sessions: objects built empty +// and filled through untyped stores, read back through untyped loads, and a +// class instance at the same kind of sites. Each session must start with +// empty caches, since the shapes their entries named are freed when the +// previous session ends. +class Box { + constructor(v) { + this.v = v; + } +} +function fill(o, n) { + o.a = n; + o.b = n + 1; + return o; +} +function sum(o) { + return o.a + o.b; +} +function unbox(b) { + return b.v; +} +export function runAll() { + let total = 0; + for (let i = 0; i < 100; i++) total += sum(fill({}, i)); + return total + unbox(new Box(total)); +} diff --git a/tests/library-mode/npm-refuse/lib-minified.ts b/tests/library-mode/npm-refuse/lib-minified.ts index 9e5c16e75..492cd5369 100644 --- a/tests/library-mode/npm-refuse/lib-minified.ts +++ b/tests/library-mode/npm-refuse/lib-minified.ts @@ -1,5 +1,5 @@ -// SC4020 fixture: slimmed ships a one-long-line dist — minified JS fails -// the eligibility bar. +// slimmed ships a one-long-line dist: minified JS compiles like any other +// eligible package. import { pick } from "slimmed"; export function f(a: number, b: number): number { diff --git a/tests/library-mode/npm-static/lib.ts b/tests/library-mode/npm-static/lib.ts index 27107f29d..15bad17a3 100644 --- a/tests/library-mode/npm-static/lib.ts +++ b/tests/library-mode/npm-static/lib.ts @@ -1,5 +1,5 @@ // Library-mode npm posture fixture (static-or-refuse): a bare specifier -// naming an ELIGIBLE package (own .d.ts, unminified shipped JS, no +// naming an ELIGIBLE package (own .d.ts, no // build-transform markers) compiles statically as part of the library // graph — no flag, no island — and the package's own top-level require // ("mathdep") joins through the same bar. The node:path import alongside diff --git a/tests/library-mode/npm-static/node_modules/mathkit/index.js b/tests/library-mode/npm-static/node_modules/mathkit/index.js index f9f9242c3..eadfefba9 100644 --- a/tests/library-mode/npm-static/node_modules/mathkit/index.js +++ b/tests/library-mode/npm-static/node_modules/mathkit/index.js @@ -1,6 +1,6 @@ "use strict"; // A vendored fake package for the library-mode npm posture: eligible by -// every bar (own .d.ts, unminified CJS, no build-transform markers), with +// every bar (own .d.ts, no build-transform markers), with // its OWN npm dep edge (mathdep) so the transitive fixpoint is exercised. const steps = require("mathdep"); diff --git a/tests/test262/README.md b/tests/test262/README.md index 46465645f..2f8a0ed3e 100644 --- a/tests/test262/README.md +++ b/tests/test262/README.md @@ -28,11 +28,11 @@ Pass the resulting directory as `--root` and use `--filter` to survey an API fam ## Execution profile and limits -The current profile is `static-adapted-v5`. It adapts strict scripts to standalone modules and sloppy scripts to CommonJS entries. Preparation supplies host assertions and a completion marker while leaving the upstream test body unchanged. Each program executes in a fresh native process. Conservative syntax checks reject known global-script dependencies, but cannot establish that every adaptation preserves the original script's semantics. Reports are local investigation artifacts, not published compatibility assessments. +The current profile is `static-adapted-v6`. It adapts strict scripts to standalone modules and sloppy scripts to CommonJS entries. Preparation supplies host assertions and a completion marker while leaving the upstream test body unchanged. Each program executes in a fresh native process. Conservative syntax checks reject known global-script dependencies, but cannot establish that every adaptation preserves the original script's semantics. Reports are local investigation artifacts, not published compatibility assessments. The unmodified upstream assertion helper currently reaches scriptc's JavaScript function-expando refusal. Test262 explicitly permits host implementations of harness functions in its [interpretation rules](https://github.com/tc39/test262/blob/7ab7fafa0003f73fc85c1b95d88094d33f7eb8bd/INTERPRETING.md#host-defined-functions). `harness.ts` supplies `assert`, `assert.sameValue`, `assert.notSameValue`, `assert.compareArray`, `assert.throws` for selected Error constructors, `$DONE`, and `Test262Error`. Scalar SameValue and scalar array-element comparisons use the statically compiled `node:assert/strict` implementation. Contract tests compare successful and failing assertions with the original Test262 helpers under Node, including NaN, signed zero, null/undefined, differing primitive types, and premature termination. -Reference-equality assertions terminate with a distinct harness-refusal result: conversion at a native function boundary can copy references, so this adapter cannot faithfully compare every object, array, or function identity. Tests cannot catch this refusal and accidentally pass. Sparse array comparisons are not yet handled faithfully across that boundary. The adapted `propertyHelper.js` admits direct `verifyProperty` calls with an inline descriptor literal or `undefined`; reference-valued comparisons still refuse, while other property helpers and descriptor expressions remain excluded. Assertion aliases, mutation, reflection, additional assertion methods, and other `includes` helpers remain excluded. +Reference-equality assertions terminate with a distinct harness-refusal result: conversion at a native function boundary can copy references, so this adapter cannot faithfully compare every object, array, or function identity. Tests cannot catch this refusal and accidentally pass. Sparse array comparisons are not yet handled faithfully across that boundary. The adapted `propertyHelper.js` admits direct `verifyProperty` calls with an inline descriptor literal or `undefined`; reference-valued comparisons still refuse, while other property helpers and descriptor expressions remain excluded. The adapted `testTypedArray.js` keeps the upstream constructor lists, argument factories and `testWith*` helpers. It omits the `TypedArray` (%TypedArray%) binding, so tests that name it are excluded. Its resizable and immutable buffer factories run only where `ArrayBuffer.prototype` provides `resize` and `transferToImmutable`, as the upstream feature checks intend. Contract tests compare its successful and failing uses with the upstream helper under Node. Assertion aliases, mutation, reflection, additional assertion methods, and other `includes` helpers remain excluded. Raw tests, module variants, agents, resolution negatives, and runtime negatives remain excluded. Parse negatives pass only when a source-matched syntax or vetted JavaScript early-error diagnostic appears; TypeScript type errors and harness-only failures never satisfy a negative expectation. `$262`, dynamic evaluation, observable script-global state, and other unsupported host requirements are excluded as well. The runtime and compiler continue to enforce their existing refusal boundaries for admitted source. diff --git a/tests/test262/run.ts b/tests/test262/run.ts index b798b3f21..2ad02100b 100644 --- a/tests/test262/run.ts +++ b/tests/test262/run.ts @@ -4,7 +4,7 @@ import { parseArgs } from "node:util"; import { shardSelect } from "../harness/shard.js"; import { runSource, type Outcome } from "./execute.js"; import { - assertThrowsSource, directory, exclusion, harnessSource, propertyHelperSource, matchesExpectation, matchesParseNegative, metadata, pin, sha256, snapshotDigest, summarize, + assertThrowsSource, directory, exclusion, harnessSource, propertyHelperSource, typedArrayHelperSource, matchesExpectation, matchesParseNegative, metadata, pin, sha256, snapshotDigest, summarize, testPaths, variants, vendorRoot, verifyVendor, } from "./support.mjs"; @@ -104,8 +104,8 @@ Other variants remain visible as exclusions.`); const report = { schema: "scriptc.test262.v1", revision: pin.commit, snapshotSha256: pin.snapshotSha256, - profile: "static-adapted-v5", - harnessSha256: sha256(harnessSource + assertThrowsSource + propertyHelperSource), + profile: "static-adapted-v6", + harnessSha256: sha256(harnessSource + assertThrowsSource + propertyHelperSource + typedArrayHelperSource), requestedBackend: backend, sanitize: process.env.SCRIPTC_SAN === "1", host: `${process.platform}-${process.arch}`, node: process.version, compilerVersion: JSON.parse(readFileSync(join(directory, "../../packages/compiler/package.json"), "utf8")).version, diff --git a/tests/test262/support.mjs b/tests/test262/support.mjs index e56612457..52739f91a 100644 --- a/tests/test262/support.mjs +++ b/tests/test262/support.mjs @@ -12,6 +12,8 @@ export const completion = "__scriptc_test262_complete__"; export const harnessSource = readFileSync(join(directory, "harness.ts"), "utf8"); export const assertThrowsSource = readFileSync(join(directory, "assert-throws.js"), "utf8"); export const propertyHelperSource = readFileSync(join(directory, "property-helper.js"), "utf8"); +export const typedArrayHelperSource = readFileSync(join(directory, "test-typed-array.js"), "utf8"); +const adaptedIncludes = new Set(["compareArray.js", "propertyHelper.js", "testTypedArray.js"]); export const expectations = JSON.parse(readFileSync(join(directory, "expectations.json"), "utf8")); export const sha256 = (bytes) => createHash("sha256").update(bytes).digest("hex"); @@ -131,11 +133,12 @@ export function exclusion(source, meta, variant) { } if (variant !== "strict" && variant !== "sloppy") return `execution:${variant}`; if (meta.flags.some((flag) => flag.startsWith("CanBlock"))) return "host:agents"; - const unsupportedIncludes = meta.includes.filter((name) => name !== "compareArray.js" && name !== "propertyHelper.js"); + const unsupportedIncludes = meta.includes.filter((name) => !adaptedIncludes.has(name)); if (unsupportedIncludes.length) return `harness-includes:${unsupportedIncludes.join(",")}`; const sf = ts.createSourceFile("test.js", source, ts.ScriptTarget.Latest, true, ts.ScriptKind.JS); let reason; const propertyHelper = meta.includes.includes("propertyHelper.js"); + const typedArrayHelper = meta.includes.includes("testTypedArray.js"); const forbidden = new Set(["$262", "$DONE", "$DONOTEVALUATE", "globalThis", "eval", "Function", "print", "process", "require"]); if (variant === "sloppy") for (const name of ["module", "exports", "__dirname", "__filename"]) forbidden.add(name); if (meta.flags.includes("async")) forbidden.delete("$DONE"); @@ -170,6 +173,9 @@ export function exclusion(source, meta, variant) { reason = "harness:compareArray-surface"; } else if (ts.isIdentifier(node) && node.text === "Test262Error") { if (!ts.isNewExpression(node.parent) || node.parent.expression !== node) reason = "harness:Test262Error-surface"; + } else if (typedArrayHelper && ts.isIdentifier(node) && node.text === "TypedArray") { + // The adapted testTypedArray.js leaves out %TypedArray%. + reason = "harness:TypedArray-intrinsic"; } else if (propertyHelper && ts.isIdentifier(node)) { if (unsupportedPropertyHelpers.has(node.text)) reason = "harness:propertyHelper-surface"; else if (node.text === "verifyProperty") { @@ -188,7 +194,9 @@ export function exclusion(source, meta, variant) { export function prepare(source, asyncTest = false, marker = completion, variant = "strict", includes = []) { const names = asyncTest ? "assert, Test262Error, $DONE" : "assert, Test262Error"; const end = asyncTest ? "" : `;console.log(${JSON.stringify(marker)});\n`; - const helpers = includes.includes("propertyHelper.js") ? `${propertyHelperSource}\n` : ""; + const helpers = + (includes.includes("propertyHelper.js") ? `${propertyHelperSource}\n` : "") + + (includes.includes("testTypedArray.js") ? `${typedArrayHelperSource}\n` : ""); if (variant === "sloppy") return `const { ${names} } = require("./harness.ts");\n${helpers}${source}\n${end}`; return `"use strict";\nimport { ${names} } from "./harness.ts";\n${helpers}${source}\n${end}`; } diff --git a/tests/test262/test-typed-array.js b/tests/test262/test-typed-array.js new file mode 100644 index 000000000..f7ee73c84 --- /dev/null +++ b/tests/test262/test-typed-array.js @@ -0,0 +1,313 @@ +// Copyright (C) 2015 André Bargull. All rights reserved. +// This adaptation of Test262's testTypedArray.js is governed by the BSD license in vendor/LICENSE. +// It keeps the upstream constructor lists and argument factories. The host +// assert.js is replaced by harness.ts, so isPrimitive is defined here, and +// Test262Error is a class that needs `new`. Factories bind through a +// closure, and feature checks use typeof (and tolerate an ArrayBuffer +// without a reachable prototype, which then has no resizable or immutable +// factories). %TypedArray% (`TypedArray`) is left out: tests that name it +// are excluded. + +function isPrimitive(value) { + return !value || (typeof value !== "object" && typeof value !== "function"); +} + +var floatArrayConstructors = [ + Float64Array, + Float32Array +]; + +var nonClampedIntArrayConstructors = [ + Int32Array, + Int16Array, + Int8Array, + Uint32Array, + Uint16Array, + Uint8Array +]; + +var intArrayConstructors = nonClampedIntArrayConstructors.concat([Uint8ClampedArray]); + +// Float16Array is a newer feature +// adding it to this list unconditionally would cause implementations lacking it to fail every test which uses it +if (typeof Float16Array !== "undefined") { + floatArrayConstructors.push(Float16Array); +} + +var bigIntArrayConstructors = []; +if (typeof BigInt64Array !== "undefined") { + bigIntArrayConstructors.push(BigInt64Array); +} +if (typeof BigUint64Array !== "undefined") { + bigIntArrayConstructors.push(BigUint64Array); +} + +var typedArrayConstructors = floatArrayConstructors.concat(intArrayConstructors); + +var allTypedArrayConstructors = typedArrayConstructors.concat(bigIntArrayConstructors); + +function makePassthrough(TA, primitiveOrIterable) { + return primitiveOrIterable; +} + +function makeArray(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable); + // Only values between 0 and 2**53 - 1 inclusive can get mapped into TA contents. + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + return Array.from({ length: n }, function() { return "0"; }); + } + return Array.from(primitiveOrIterable); +} + +function makeArrayLike(TA, primitiveOrIterable) { + var arr = makeArray(TA, primitiveOrIterable); + if (isPrimitive(arr)) return arr; + var obj = { length: arr.length }; + for (var i = 0; i < obj.length; i++) obj[i] = arr[i]; + return obj; +} + +var makeIterable; +if (typeof Symbol !== "undefined") { + makeIterable = function makeIterable(TA, primitiveOrIterable) { + var src = makeArray(TA, primitiveOrIterable); + if (isPrimitive(src)) return src; + var obj = {}; + obj[Symbol.iterator] = function() { return src[Symbol.iterator](); }; + return obj; + }; +} + +function makeArrayBuffer(TA, primitiveOrIterable) { + var arr = makeArray(TA, primitiveOrIterable); + if (isPrimitive(arr)) return arr; + return new TA(arr).buffer; +} + +var makeResizableArrayBuffer, makeGrownArrayBuffer, makeShrunkArrayBuffer, makeImmutableArrayBuffer; +if (typeof ArrayBuffer.prototype === "object" && typeof ArrayBuffer.prototype.resize === "function") { + var copyIntoArrayBuffer = function(destBuffer, srcBuffer) { + var destView = new Uint8Array(destBuffer); + var srcView = new Uint8Array(srcBuffer); + for (var i = 0; i < srcView.length; i++) destView[i] = srcView[i]; + return destBuffer; + }; + + makeResizableArrayBuffer = function makeResizableArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + return new ArrayBuffer(n, { maxByteLength: n * 2 }); + } + var fixed = makeArrayBuffer(TA, primitiveOrIterable); + var byteLength = fixed.byteLength; + var resizable = new ArrayBuffer(byteLength, { maxByteLength: byteLength * 2 }); + return copyIntoArrayBuffer(resizable, fixed); + }; + + makeGrownArrayBuffer = function makeGrownArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + var grown = new ArrayBuffer(Math.floor(n / 2), { maxByteLength: n }); + grown.resize(n); + } + var fixed = makeArrayBuffer(TA, primitiveOrIterable); + var byteLength = fixed.byteLength; + var grown = new ArrayBuffer(Math.floor(byteLength / 2), { maxByteLength: byteLength }); + grown.resize(byteLength); + return copyIntoArrayBuffer(grown, fixed); + }; + + makeShrunkArrayBuffer = function makeShrunkArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + var shrunk = new ArrayBuffer(n * 2, { maxByteLength: n * 2 }); + shrunk.resize(n); + } + var fixed = makeArrayBuffer(TA, primitiveOrIterable); + var byteLength = fixed.byteLength; + var shrunk = new ArrayBuffer(byteLength * 2, { maxByteLength: byteLength * 2 }); + copyIntoArrayBuffer(shrunk, fixed); + shrunk.resize(byteLength); + return shrunk; + }; +} +if (typeof ArrayBuffer.prototype === "object" && typeof ArrayBuffer.prototype.transferToImmutable === "function") { + makeImmutableArrayBuffer = function makeImmutableArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + return (new ArrayBuffer(n)).transferToImmutable(); + } + var mutable = makeArrayBuffer(TA, primitiveOrIterable); + return mutable.transferToImmutable(); + }; +} + +var typedArrayCtorArgFactories = [makePassthrough, makeArray, makeArrayLike]; +if (makeIterable) typedArrayCtorArgFactories.push(makeIterable); +typedArrayCtorArgFactories.push(makeArrayBuffer); +if (makeResizableArrayBuffer) typedArrayCtorArgFactories.push(makeResizableArrayBuffer); +if (makeGrownArrayBuffer) typedArrayCtorArgFactories.push(makeGrownArrayBuffer); +if (makeShrunkArrayBuffer) typedArrayCtorArgFactories.push(makeShrunkArrayBuffer); +if (makeImmutableArrayBuffer) typedArrayCtorArgFactories.push(makeImmutableArrayBuffer); + + +function ctorArgFactoryMatchesSome(argFactory, features) { + for (var i = 0; i < features.length; ++i) { + switch (features[i]) { + case "passthrough": + if (argFactory === makePassthrough) return true; + break; + case "arraylike": + if (argFactory === makeArray || argFactory === makeArrayLike) return true; + break; + case "iterable": + if (argFactory === makeIterable) return true; + break; + case "arraybuffer": + if ( + argFactory === makeArrayBuffer || + argFactory === makeResizableArrayBuffer || + argFactory === makeGrownArrayBuffer || + argFactory === makeShrunkArrayBuffer || + argFactory === makeImmutableArrayBuffer + ) { + return true; + } + break; + case "resizable": + if ( + argFactory === makeResizableArrayBuffer || + argFactory === makeGrownArrayBuffer || + argFactory === makeShrunkArrayBuffer + ) { + return true; + } + break; + case "immutable": + if (argFactory === makeImmutableArrayBuffer) return true; + break; + default: + throw new Test262Error("unknown feature: " + features[i]); + } + } + return false; +} + + +function bindArgFactory(argFactory, constructor) { + return function(primitiveOrIterable) { + return argFactory(constructor, primitiveOrIterable); + }; +} + +function testWithAllTypedArrayConstructors(f, constructors, includeArgFactories, excludeArgFactories) { + var ctors = constructors || allTypedArrayConstructors; + var ctorArgFactories = typedArrayCtorArgFactories; + if (includeArgFactories) { + ctorArgFactories = []; + for (var i = 0; i < typedArrayCtorArgFactories.length; ++i) { + if (ctorArgFactoryMatchesSome(typedArrayCtorArgFactories[i], includeArgFactories)) { + ctorArgFactories.push(typedArrayCtorArgFactories[i]); + } + } + } + if (excludeArgFactories) { + ctorArgFactories = ctorArgFactories.slice(); + for (var i = ctorArgFactories.length - 1; i >= 0; --i) { + if (ctorArgFactoryMatchesSome(ctorArgFactories[i], excludeArgFactories)) { + ctorArgFactories.splice(i, 1); + } + } + } + if (ctorArgFactories.length === 0) { + throw new Test262Error("no arg factories match include " + includeArgFactories + " and exclude " + excludeArgFactories); + } + for (var k = 0; k < ctorArgFactories.length; ++k) { + var argFactory = ctorArgFactories[k]; + for (var i = 0; i < ctors.length; ++i) { + var constructor = ctors[i]; + var boundArgFactory = bindArgFactory(argFactory, constructor); + try { + f(constructor, boundArgFactory); + } catch (e) { + e.message += " (Testing with " + constructor.name + " and " + argFactory.name + ".)"; + throw e; + } + } + } +} + +function testWithTypedArrayConstructors(f, constructors, includeArgFactories, excludeArgFactories) { + var ctors = constructors || typedArrayConstructors; + testWithAllTypedArrayConstructors(f, ctors, includeArgFactories, excludeArgFactories); +} + +function testWithBigIntTypedArrayConstructors(f, constructors, includeArgFactories, excludeArgFactories) { + var ctors = constructors || [BigInt64Array, BigUint64Array]; + testWithAllTypedArrayConstructors(f, ctors, includeArgFactories, excludeArgFactories); +} + +var nonAtomicsFriendlyTypedArrayConstructors = floatArrayConstructors.concat([Uint8ClampedArray]); +function testWithNonAtomicsFriendlyTypedArrayConstructors(f, includeArgFactories, excludeArgFactories) { + testWithAllTypedArrayConstructors( + f, + nonAtomicsFriendlyTypedArrayConstructors, + includeArgFactories, + excludeArgFactories + ); +} + +function testWithAtomicsFriendlyTypedArrayConstructors(f, includeArgFactories, excludeArgFactories) { + testWithAllTypedArrayConstructors( + f, + [ + Int32Array, + Int16Array, + Int8Array, + Uint32Array, + Uint16Array, + Uint8Array, + ], + includeArgFactories, + excludeArgFactories + ); +} + +function testTypedArrayConversions(byteConversionValues, fn) { + var values = byteConversionValues.values; + var expected = byteConversionValues.expected; + + testWithTypedArrayConstructors(function(TA) { + var name = TA.name.slice(0, -5); + + return values.forEach(function(value, index) { + var exp = expected[name][index]; + var initial = 0; + if (exp === 0) { + initial = 1; + } + fn(TA, value, exp, initial); + }); + }, null, ["passthrough"]); +} + +function isFloatTypedArrayConstructor(arg) { + return floatArrayConstructors.indexOf(arg) !== -1; +} + +function floatTypedArrayConstructorPrecision(FA) { + if (typeof Float16Array !== "undefined" && FA === Float16Array) { + return "half"; + } else if (FA === Float32Array) { + return "single"; + } else if (FA === Float64Array) { + return "double"; + } else { + throw new Error("Malformed test - floatTypedArrayConstructorPrecision called with non-float TypedArray"); + } +} diff --git a/tests/test262/upstream.json b/tests/test262/upstream.json index 1dc97ba20..025576e72 100644 --- a/tests/test262/upstream.json +++ b/tests/test262/upstream.json @@ -169,6 +169,12 @@ "test/built-ins/String/prototype/trim/15.5.4.20-2-47.js", "test/built-ins/String/prototype/trim/15.5.4.20-2-50.js", "test/built-ins/String/prototype/trim/15.5.4.20-2-6.js", + "test/built-ins/TypedArray/prototype/byteOffset/return-byteoffset.js", + "test/built-ins/TypedArray/prototype/subarray/return-abrupt-from-begin-symbol.js", + "test/built-ins/TypedArrayConstructors/ctors/buffer-arg/byteoffset-is-negative-zero-sab.js", + "test/built-ins/TypedArrayConstructors/ctors/length-arg/is-infinity-throws-rangeerror.js", + "test/built-ins/TypedArrayConstructors/ctors/typedarray-arg/undefined-newtarget-throws.js", + "test/built-ins/TypedArrayConstructors/internals/DefineOwnProperty/key-is-numericindex-desc-not-configurable-throws.js", "test/language/arguments-object/func-decl-args-trailing-comma-null.js", "test/language/arguments-object/func-expr-args-trailing-comma-undefined.js", "test/language/expressions/addition/S11.6.1_A4_T1.js", @@ -214,6 +220,7 @@ "harness/assert.js": "206e274ca325eb8a652e3911c3fbd090e2480d11ed7579dc17a5d17a2360ed48", "harness/propertyHelper.js": "08bc29114d804c4f24f906bac950ee4dc9c93c659d93258e7b44a54283ba9bb8", "harness/sta.js": "1930c54af79455c484799f43e9a28e2b2f15c40d0917c9941ca54e26db243f35", + "harness/testTypedArray.js": "51eb50e53e6215f31f14d64d249bea2d08ad943c77835f63ba52ec694de2189a", "test/annexB/language/function-code/function-redeclaration-block.js": "8d4a3f259dabceeee101959aac117a19f5bfe9e374e1178b45f45a1142195d9e", "test/annexB/language/function-code/if-decl-else-decl-b-func-existing-fn-no-init.js": "ccd95922fd6c784012f7299445f0859db04812232a2266dd90070b026244bbf5", "test/built-ins/Array/prototype/at/returns-item-relative-index.js": "71f53cf86f1387101b19b6ae9939d47ced34497d3cee12f954eef454f912ba88", @@ -377,6 +384,12 @@ "test/built-ins/String/prototype/trim/15.5.4.20-2-47.js": "3e1a745af465177a96a29601e56299afc044060dbc2ac3061a59032be451cfde", "test/built-ins/String/prototype/trim/15.5.4.20-2-50.js": "38b84997faa99e422408917eeac7f3575bd6064b429ea390a9d041fd77fce532", "test/built-ins/String/prototype/trim/15.5.4.20-2-6.js": "3d5857f24117f65f07a70edd0d254dfa07f593c76effed94500847ccf8564c06", + "test/built-ins/TypedArray/prototype/byteOffset/return-byteoffset.js": "04cd6e31c8223319bbc6f5e5630ee476708a3897ead70e9cec34d40fb3ebd062", + "test/built-ins/TypedArray/prototype/subarray/return-abrupt-from-begin-symbol.js": "51fd475317d19c863a85c26f1ffa2070b10abf123b49a3243bb3829218b0b7a9", + "test/built-ins/TypedArrayConstructors/ctors/buffer-arg/byteoffset-is-negative-zero-sab.js": "96741e885f333125077884cb59c23c297fad61a4834941a5ac38720227094d7d", + "test/built-ins/TypedArrayConstructors/ctors/length-arg/is-infinity-throws-rangeerror.js": "ea340740e8fb3227b19fff023508c38d3bd6b2ff45f7a623d1e4460fad9b32c1", + "test/built-ins/TypedArrayConstructors/ctors/typedarray-arg/undefined-newtarget-throws.js": "19d2ebe0d3db51eb9f978b9b2462bb38ff5cf4702f32e3163d8be46e851d0f8f", + "test/built-ins/TypedArrayConstructors/internals/DefineOwnProperty/key-is-numericindex-desc-not-configurable-throws.js": "711cf2873e49c9d48b3246f8dd067380ac902d286907b620735f5f4dbd36c943", "test/language/arguments-object/func-decl-args-trailing-comma-null.js": "80aecea8edd0d7bac1ddcfcd6cb97dc9a513b25a9283bb9cedd21fd77dd64d50", "test/language/arguments-object/func-expr-args-trailing-comma-undefined.js": "cf39515a9031db4c81db231a157516bdc97617c392e4ef28890b2ae74777a99e", "test/language/expressions/addition/S11.6.1_A4_T1.js": "12928eb319dcf0cc785e399580effb0b7495709296dce9c0a58e8408616a959f", diff --git a/tests/test262/vendor/harness/testTypedArray.js b/tests/test262/vendor/harness/testTypedArray.js new file mode 100644 index 000000000..a6418c2f7 --- /dev/null +++ b/tests/test262/vendor/harness/testTypedArray.js @@ -0,0 +1,431 @@ +// Copyright (C) 2015 André Bargull. All rights reserved. +// This code is governed by the BSD license found in the LICENSE file. +/*--- +description: | + Collection of functions used to assert the correctness of TypedArray objects. +defines: + - floatArrayConstructors + - nonClampedIntArrayConstructors + - intArrayConstructors + - typedArrayConstructors + - TypedArray + - testWithAllTypedArrayConstructors + - testWithTypedArrayConstructors + - testWithBigIntTypedArrayConstructors + - nonAtomicsFriendlyTypedArrayConstructors + - testWithAtomicsFriendlyTypedArrayConstructors + - testWithNonAtomicsFriendlyTypedArrayConstructors + - testTypedArrayConversions +---*/ + +var floatArrayConstructors = [ + Float64Array, + Float32Array +]; + +var nonClampedIntArrayConstructors = [ + Int32Array, + Int16Array, + Int8Array, + Uint32Array, + Uint16Array, + Uint8Array +]; + +var intArrayConstructors = nonClampedIntArrayConstructors.concat([Uint8ClampedArray]); + +// Float16Array is a newer feature +// adding it to this list unconditionally would cause implementations lacking it to fail every test which uses it +if (typeof Float16Array !== "undefined") { + floatArrayConstructors.push(Float16Array); +} + +var bigIntArrayConstructors = []; +if (typeof BigInt64Array !== "undefined") { + bigIntArrayConstructors.push(BigInt64Array); +} +if (typeof BigUint64Array !== "undefined") { + bigIntArrayConstructors.push(BigUint64Array); +} + +/** + * Array containing every non-bigint typed array constructor. + */ +var typedArrayConstructors = floatArrayConstructors.concat(intArrayConstructors); + +/** + * Array containing every typed array constructor, including those with bigint values. + */ +var allTypedArrayConstructors = typedArrayConstructors.concat(bigIntArrayConstructors); + +/** + * The %TypedArray% intrinsic constructor function. + */ +var TypedArray = Object.getPrototypeOf(Int8Array); + +function makePassthrough(TA, primitiveOrIterable) { + return primitiveOrIterable; +} + +function makeArray(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable); + // Only values between 0 and 2**53 - 1 inclusive can get mapped into TA contents. + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + return Array.from({ length: n }, function() { return "0"; }); + } + return Array.from(primitiveOrIterable); +} + +function makeArrayLike(TA, primitiveOrIterable) { + var arr = makeArray(TA, primitiveOrIterable); + if (isPrimitive(arr)) return arr; + var obj = { length: arr.length }; + for (var i = 0; i < obj.length; i++) obj[i] = arr[i]; + return obj; +} + +var makeIterable; +if (typeof Symbol !== "undefined" && Symbol.iterator) { + makeIterable = function makeIterable(TA, primitiveOrIterable) { + var src = makeArray(TA, primitiveOrIterable); + if (isPrimitive(src)) return src; + var obj = {}; + obj[Symbol.iterator] = function() { return src[Symbol.iterator](); }; + return obj; + }; +} + +function makeArrayBuffer(TA, primitiveOrIterable) { + var arr = makeArray(TA, primitiveOrIterable); + if (isPrimitive(arr)) return arr; + return new TA(arr).buffer; +} + +var makeResizableArrayBuffer, makeGrownArrayBuffer, makeShrunkArrayBuffer, makeImmutableArrayBuffer; +if (ArrayBuffer.prototype.resize) { + var copyIntoArrayBuffer = function(destBuffer, srcBuffer) { + var destView = new Uint8Array(destBuffer); + var srcView = new Uint8Array(srcBuffer); + for (var i = 0; i < srcView.length; i++) destView[i] = srcView[i]; + return destBuffer; + }; + + makeResizableArrayBuffer = function makeResizableArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + return new ArrayBuffer(n, { maxByteLength: n * 2 }); + } + var fixed = makeArrayBuffer(TA, primitiveOrIterable); + var byteLength = fixed.byteLength; + var resizable = new ArrayBuffer(byteLength, { maxByteLength: byteLength * 2 }); + return copyIntoArrayBuffer(resizable, fixed); + }; + + makeGrownArrayBuffer = function makeGrownArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + var grown = new ArrayBuffer(Math.floor(n / 2), { maxByteLength: n }); + grown.resize(n); + } + var fixed = makeArrayBuffer(TA, primitiveOrIterable); + var byteLength = fixed.byteLength; + var grown = new ArrayBuffer(Math.floor(byteLength / 2), { maxByteLength: byteLength }); + grown.resize(byteLength); + return copyIntoArrayBuffer(grown, fixed); + }; + + makeShrunkArrayBuffer = function makeShrunkArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + var shrunk = new ArrayBuffer(n * 2, { maxByteLength: n * 2 }); + shrunk.resize(n); + } + var fixed = makeArrayBuffer(TA, primitiveOrIterable); + var byteLength = fixed.byteLength; + var shrunk = new ArrayBuffer(byteLength * 2, { maxByteLength: byteLength * 2 }); + copyIntoArrayBuffer(shrunk, fixed); + shrunk.resize(byteLength); + return shrunk; + }; +} +if (ArrayBuffer.prototype.transferToImmutable) { + makeImmutableArrayBuffer = function makeImmutableArrayBuffer(TA, primitiveOrIterable) { + if (isPrimitive(primitiveOrIterable)) { + var n = Number(primitiveOrIterable) * TA.BYTES_PER_ELEMENT; + if (!(n >= 0 && n < 9007199254740992)) return primitiveOrIterable; + return (new ArrayBuffer(n)).transferToImmutable(); + } + var mutable = makeArrayBuffer(TA, primitiveOrIterable); + return mutable.transferToImmutable(); + }; +} + +var typedArrayCtorArgFactories = [makePassthrough, makeArray, makeArrayLike]; +if (makeIterable) typedArrayCtorArgFactories.push(makeIterable); +typedArrayCtorArgFactories.push(makeArrayBuffer); +if (makeResizableArrayBuffer) typedArrayCtorArgFactories.push(makeResizableArrayBuffer); +if (makeGrownArrayBuffer) typedArrayCtorArgFactories.push(makeGrownArrayBuffer); +if (makeShrunkArrayBuffer) typedArrayCtorArgFactories.push(makeShrunkArrayBuffer); +if (makeImmutableArrayBuffer) typedArrayCtorArgFactories.push(makeImmutableArrayBuffer); + +/** + * @typedef {"passthrough" | "arraylike" | "iterable" | "arraybuffer" | "resizable" | "immutable"} typedArrayArgFactoryFeature + */ + +/** + * A predicate for testing whether a TypedArray argument factory from this file + * matches any of the provided features. + * + * @param {Function} argFactory + * @param {typedArrayArgFactoryFeature[]} features + * @returns {boolean} + */ +function ctorArgFactoryMatchesSome(argFactory, features) { + for (var i = 0; i < features.length; ++i) { + switch (features[i]) { + case "passthrough": + if (argFactory === makePassthrough) return true; + break; + case "arraylike": + if (argFactory === makeArray || argFactory === makeArrayLike) return true; + break; + case "iterable": + if (argFactory === makeIterable) return true; + break; + case "arraybuffer": + if ( + argFactory === makeArrayBuffer || + argFactory === makeResizableArrayBuffer || + argFactory === makeGrownArrayBuffer || + argFactory === makeShrunkArrayBuffer || + argFactory === makeImmutableArrayBuffer + ) { + return true; + } + break; + case "resizable": + if ( + argFactory === makeResizableArrayBuffer || + argFactory === makeGrownArrayBuffer || + argFactory === makeShrunkArrayBuffer + ) { + return true; + } + break; + case "immutable": + if (argFactory === makeImmutableArrayBuffer) return true; + break; + default: + throw Test262Error("unknown feature: " + features[i]); + } + } + return false; +} + +/** + * Callback for testing a typed array constructor. + * + * @callback typedArrayConstructorCallback + * @param {Function} TypedArrayConstructor the constructor object to test with + * @param {Function} [TypedArrayConstructorArgFactory] a function for making + * a TypedArrayConstructor argument from a primitive (usually a number) or + * iterable (usually an array) + */ + +/** + * Calls the provided function with (typedArrayCtor, typedArrayCtorArgFactory) + * pairs, where typedArrayCtor is Uint8Array/Int8Array/BigInt64Array/etc. and + * typedArrayCtorArgFactory is a function for mapping a primitive (usually a + * number) or iterable (usually an array) into a value suitable as the first + * argument of typedArrayCtor (an Array, arraylike, iterable, or ArrayBuffer). + * + * @param {typedArrayConstructorCallback} f - the function to call + * @param {Array} [constructors] - an explicit list of TypedArray constructors + * @param {typedArrayArgFactoryFeature[]} [includeArgFactories] - for selecting + * initial constructor argument factory functions, rather than starting with + * all argument factories + * @param {typedArrayArgFactoryFeature[]} [excludeArgFactories] - for excluding + * constructor argument factory functions, after an initial selection + */ +function testWithAllTypedArrayConstructors(f, constructors, includeArgFactories, excludeArgFactories) { + var ctors = constructors || allTypedArrayConstructors; + var ctorArgFactories = typedArrayCtorArgFactories; + if (includeArgFactories) { + ctorArgFactories = []; + for (var i = 0; i < typedArrayCtorArgFactories.length; ++i) { + if (ctorArgFactoryMatchesSome(typedArrayCtorArgFactories[i], includeArgFactories)) { + ctorArgFactories.push(typedArrayCtorArgFactories[i]); + } + } + } + if (excludeArgFactories) { + ctorArgFactories = ctorArgFactories.slice(); + for (var i = ctorArgFactories.length - 1; i >= 0; --i) { + if (ctorArgFactoryMatchesSome(ctorArgFactories[i], excludeArgFactories)) { + ctorArgFactories.splice(i, 1); + } + } + } + if (ctorArgFactories.length === 0) { + throw Test262Error("no arg factories match include " + includeArgFactories + " and exclude " + excludeArgFactories); + } + for (var k = 0; k < ctorArgFactories.length; ++k) { + var argFactory = ctorArgFactories[k]; + for (var i = 0; i < ctors.length; ++i) { + var constructor = ctors[i]; + var boundArgFactory = argFactory.bind(undefined, constructor); + try { + f(constructor, boundArgFactory); + } catch (e) { + e.message += " (Testing with " + constructor.name + " and " + argFactory.name + ".)"; + throw e; + } + } + } +} + +/** + * Calls the provided function with (typedArrayCtor, typedArrayCtorArgFactory) + * pairs, where typedArrayCtor is Uint8Array/Int8Array/BigInt64Array/etc. and + * typedArrayCtorArgFactory is a function for mapping a primitive (usually a + * number) or iterable (usually an array) into a value suitable as the first + * argument of typedArrayCtor (an Array, arraylike, iterable, or ArrayBuffer). + * + * typedArrayCtor will not be BigInt64Array or BigUint64Array unless one or both + * of those are explicitly provided. + * + * @param {typedArrayConstructorCallback} f - the function to call + * @param {Array} [constructors] - an explicit list of TypedArray constructors + * @param {typedArrayArgFactoryFeature[]} [includeArgFactories] - for selecting + * initial constructor argument factory functions, rather than starting with + * all argument factories + * @param {typedArrayArgFactoryFeature[]} [excludeArgFactories] - for excluding + * constructor argument factory functions, after an initial selection + */ +function testWithTypedArrayConstructors(f, constructors, includeArgFactories, excludeArgFactories) { + var ctors = constructors || typedArrayConstructors; + testWithAllTypedArrayConstructors(f, ctors, includeArgFactories, excludeArgFactories); +} + +/** + * Calls the provided function for every BigInt typed array constructor. + * + * @param {typedArrayConstructorCallback} f - the function to call + * @param {Array} [constructors] - an explicit list of TypedArray constructors + * @param {typedArrayArgFactoryFeature[]} [includeArgFactories] - for selecting + * initial constructor argument factory functions, rather than starting with + * all argument factories + * @param {typedArrayArgFactoryFeature[]} [excludeArgFactories] - for excluding + * constructor argument factory functions, after an initial selection + */ +function testWithBigIntTypedArrayConstructors(f, constructors, includeArgFactories, excludeArgFactories) { + var ctors = constructors || [BigInt64Array, BigUint64Array]; + testWithAllTypedArrayConstructors(f, ctors, includeArgFactories, excludeArgFactories); +} + +var nonAtomicsFriendlyTypedArrayConstructors = floatArrayConstructors.concat([Uint8ClampedArray]); +/** + * Calls the provided function for every non-"Atomics Friendly" typed array constructor. + * + * @param {typedArrayConstructorCallback} f - the function to call for each typed array constructor. + * @param {Array} selected - An optional Array with filtered typed arrays + * @param {typedArrayArgFactoryFeature[]} [includeArgFactories] - for selecting + * initial constructor argument factory functions, rather than starting with + * all argument factories + * @param {typedArrayArgFactoryFeature[]} [excludeArgFactories] - for excluding + * constructor argument factory functions, after an initial selection + */ +function testWithNonAtomicsFriendlyTypedArrayConstructors(f, includeArgFactories, excludeArgFactories) { + testWithAllTypedArrayConstructors( + f, + nonAtomicsFriendlyTypedArrayConstructors, + includeArgFactories, + excludeArgFactories + ); +} + +/** + * Calls the provided function for every "Atomics Friendly" typed array constructor. + * + * @param {typedArrayConstructorCallback} f - the function to call for each typed array constructor. + * @param {Array} selected - An optional Array with filtered typed arrays + * @param {typedArrayArgFactoryFeature[]} [includeArgFactories] - for selecting + * initial constructor argument factory functions, rather than starting with + * all argument factories + * @param {typedArrayArgFactoryFeature[]} [excludeArgFactories] - for excluding + * constructor argument factory functions, after an initial selection + */ +function testWithAtomicsFriendlyTypedArrayConstructors(f, includeArgFactories, excludeArgFactories) { + testWithAllTypedArrayConstructors( + f, + [ + Int32Array, + Int16Array, + Int8Array, + Uint32Array, + Uint16Array, + Uint8Array, + ], + includeArgFactories, + excludeArgFactories + ); +} + +/** + * Helper for conversion operations on TypedArrays, the expected values + * properties are indexed in order to match the respective value for each + * TypedArray constructor + * @param {Function} fn - the function to call for each constructor and value. + * will be called with the constructor, value, expected + * value, and a initial value that can be used to avoid + * a false positive with an equivalent expected value. + */ +function testTypedArrayConversions(byteConversionValues, fn) { + var values = byteConversionValues.values; + var expected = byteConversionValues.expected; + + testWithTypedArrayConstructors(function(TA) { + var name = TA.name.slice(0, -5); + + return values.forEach(function(value, index) { + var exp = expected[name][index]; + var initial = 0; + if (exp === 0) { + initial = 1; + } + fn(TA, value, exp, initial); + }); + }, null, ["passthrough"]); +} + +/** + * Checks if the given argument is one of the float-based TypedArray constructors. + * + * @param {constructor} ctor - the value to check + * @returns {boolean} + */ +function isFloatTypedArrayConstructor(arg) { + return floatArrayConstructors.indexOf(arg) !== -1; +} + +/** + * Determines the precision of the given float-based TypedArray constructor. + * + * @param {constructor} ctor - the value to check + * @returns {string} "half", "single", or "double" for Float16Array, Float32Array, and Float64Array respectively. + */ +function floatTypedArrayConstructorPrecision(FA) { + if (typeof Float16Array !== "undefined" && FA === Float16Array) { + return "half"; + } else if (FA === Float32Array) { + return "single"; + } else if (FA === Float64Array) { + return "double"; + } else { + throw new Error("Malformed test - floatTypedArrayConstructorPrecision called with non-float TypedArray"); + } +} diff --git a/tests/test262/vendor/test/built-ins/TypedArray/prototype/byteOffset/return-byteoffset.js b/tests/test262/vendor/test/built-ins/TypedArray/prototype/byteOffset/return-byteoffset.js new file mode 100644 index 000000000..cbca8c3ec --- /dev/null +++ b/tests/test262/vendor/test/built-ins/TypedArray/prototype/byteOffset/return-byteoffset.js @@ -0,0 +1,31 @@ +// Copyright (C) 2016 the V8 project authors. All rights reserved. +// This code is governed by the BSD license found in the LICENSE file. +/*--- +esid: sec-get-%typedarray%.prototype.byteoffset +description: > + Return value from [[ByteOffset]] internal slot +info: | + 22.2.3.3 get %TypedArray%.prototype.byteOffset + + ... + 6. Let offset be the value of O's [[ByteOffset]] internal slot. + 7. Return size. +includes: [testTypedArray.js] +features: [TypedArray] +---*/ + +testWithTypedArrayConstructors(function(TA, makeCtorArg) { + var ta1 = new TA(); + assert.sameValue(ta1.byteOffset, 0, "Regular typedArray"); + + var offset = 4 * TA.BYTES_PER_ELEMENT; + + var buffer1 = makeCtorArg(8); + var ta2 = new TA(buffer1, offset); + assert.sameValue(ta2.byteOffset, offset, "TA(buffer, offset)"); + + var buffer2 = makeCtorArg(8); + var sample = new TA(buffer2, offset); + var ta3 = new TA(sample); + assert.sameValue(ta3.byteOffset, 0, "TA(typedArray)"); +}, null, ["arraybuffer"]); diff --git a/tests/test262/vendor/test/built-ins/TypedArray/prototype/subarray/return-abrupt-from-begin-symbol.js b/tests/test262/vendor/test/built-ins/TypedArray/prototype/subarray/return-abrupt-from-begin-symbol.js new file mode 100644 index 000000000..1e54741d3 --- /dev/null +++ b/tests/test262/vendor/test/built-ins/TypedArray/prototype/subarray/return-abrupt-from-begin-symbol.js @@ -0,0 +1,24 @@ +// Copyright (C) 2016 the V8 project authors. All rights reserved. +// This code is governed by the BSD license found in the LICENSE file. +/*--- +esid: sec-%typedarray%.prototype.subarray +description: Return abrupt from ToInteger(begin), begin is symbol +info: | + 22.2.3.27 %TypedArray%.prototype.subarray( begin , end ) + + ... + 7. Let relativeBegin be ? ToInteger(begin). + ... +includes: [testTypedArray.js] +features: [Symbol, TypedArray] +---*/ + +var s = Symbol("1"); + +testWithTypedArrayConstructors(function(TA, makeCtorArg) { + var sample = new TA(makeCtorArg(0)); + + assert.throws(TypeError, function() { + sample.subarray(s); + }); +}); diff --git a/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/buffer-arg/byteoffset-is-negative-zero-sab.js b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/buffer-arg/byteoffset-is-negative-zero-sab.js new file mode 100644 index 000000000..b4bf00149 --- /dev/null +++ b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/buffer-arg/byteoffset-is-negative-zero-sab.js @@ -0,0 +1,23 @@ +// Copyright (C) 2015 André Bargull. All rights reserved. +// This code is governed by the BSD license found in the LICENSE file. + +/*--- +esid: pending +description: > + TypedArray's [[ByteOffset]] internal slot is always a positive number, test with negative zero. +info: | + %TypedArray% ( buffer [ , byteOffset [ , length ] ] ) + + ... + 6. Let offset be ? ToInteger(byteOffset). + 7. If offset < 0, throw a RangeError exception. + 8. If offset is -0, let offset be +0. + ... +includes: [testTypedArray.js] +features: [SharedArrayBuffer, TypedArray] +---*/ + +testWithTypedArrayConstructors(function(TAConstructor) { + var typedArray = new TAConstructor(new SharedArrayBuffer(8), -0); + assert.sameValue(typedArray.byteOffset, +0); +}, null, ["passthrough"]); diff --git a/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/length-arg/is-infinity-throws-rangeerror.js b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/length-arg/is-infinity-throws-rangeerror.js new file mode 100644 index 000000000..c04f79f76 --- /dev/null +++ b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/length-arg/is-infinity-throws-rangeerror.js @@ -0,0 +1,27 @@ +// Copyright (C) 2016 the V8 project authors. All rights reserved. +// This code is governed by the BSD license found in the LICENSE file. +/*--- +esid: sec-typedarray-length +description: > + Throws a RangeError if length is a Infinity value +info: | + 22.2.4.2 TypedArray ( length ) + + This description applies only if the TypedArray function is called with at + least one argument and the Type of the first argument is not Object. + + ... + 4. Let numberLength be ? ToNumber(length). + 5. Let elementLength be ToLength(numberLength). + 6. If SameValueZero(numberLength, elementLength) is false, throw a RangeError + exception. + ... +includes: [testTypedArray.js] +features: [TypedArray] +---*/ + +testWithTypedArrayConstructors(function(TA) { + assert.throws(RangeError, function() { + new TA(Infinity); + }); +}, null, ["passthrough"]); diff --git a/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/typedarray-arg/undefined-newtarget-throws.js b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/typedarray-arg/undefined-newtarget-throws.js new file mode 100644 index 000000000..e96b6d1d4 --- /dev/null +++ b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/ctors/typedarray-arg/undefined-newtarget-throws.js @@ -0,0 +1,27 @@ +// Copyright (C) 2016 the V8 project authors. All rights reserved. +// This code is governed by the BSD license found in the LICENSE file. +/*--- +esid: sec-typedarray-typedarray +description: > + Throws a TypeError if NewTarget is undefined. +info: | + 22.2.4.3 TypedArray ( typedArray ) + + This description applies only if the TypedArray function is called with at + least one argument and the Type of the first argument is Object and that + object has a [[TypedArrayName]] internal slot. + + ... + 2. If NewTarget is undefined, throw a TypeError exception. + ... +includes: [testTypedArray.js] +features: [TypedArray] +---*/ + +testWithTypedArrayConstructors(function(TA, makeCtorArg) { + var typedArray = new TA(makeCtorArg(4)); + + assert.throws(TypeError, function() { + TA(typedArray); + }); +}); diff --git a/tests/test262/vendor/test/built-ins/TypedArrayConstructors/internals/DefineOwnProperty/key-is-numericindex-desc-not-configurable-throws.js b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/internals/DefineOwnProperty/key-is-numericindex-desc-not-configurable-throws.js new file mode 100644 index 000000000..898cf1abc --- /dev/null +++ b/tests/test262/vendor/test/built-ins/TypedArrayConstructors/internals/DefineOwnProperty/key-is-numericindex-desc-not-configurable-throws.js @@ -0,0 +1,39 @@ +// Copyright (C) 2021 Alexey Shvayka. All rights reserved. +// This code is governed by the BSD license found in the LICENSE file. +/*--- +esid: sec-integer-indexed-exotic-objects-defineownproperty-p-desc +description: > + Throws TypeError for valid index & non-configurable descriptor. +info: | + [[DefineOwnProperty]] ( P, Desc ) + + [...] + 3. If Type(P) is String, then + a. Let numericIndex be ! CanonicalNumericIndexString(P). + b. If numericIndex is not undefined, then + [...] + ii. If Desc has a [[Configurable]] field and if Desc.[[Configurable]] is false, return false. +includes: [testTypedArray.js] +features: [align-detached-buffer-semantics-with-web-reality, TypedArray] +---*/ + +testWithTypedArrayConstructors(function(TA, makeCtorArg) { + var sample = new TA(makeCtorArg([0])); + + assert.throws(TypeError, function() { + Object.defineProperty(sample, "0", { + configurable: false, + }); + }, "partial descriptor"); + + assert.throws(TypeError, function() { + Object.defineProperty(sample, "0", { + value: 42, + writable: true, + enumerable: true, + configurable: false, + }); + }, "complete descriptor"); + + assert.sameValue(sample[0], 0, "side effect check"); +}, null, ["passthrough"]);
+
+ {hasChildren ? ( + + ) : } +
+ + {displayName} + +
+ {row.kind}{row.apiSymbol && row.apiSymbol !== row.signature && !isRoot ? ` · ${row.apiSymbol}` : ""} +
+
+
+
-
- {hasChildren ? ( - - ) : } -
- - {displayName} - -
- {row.kind}{row.apiSymbol && row.apiSymbol !== row.signature && !isRoot ? ` · ${row.apiSymbol}` : ""} -
-
-
-