trending/scriptc
1
0
Fork 0
mirror of https://github.com/vercel-labs/scriptc.git synced 2026-10-29 07:04:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge main

Browse Source
This commit is contained in:
Chris Tate
2026-10-20 23:21:29 -05:00
parent 5f737e8d9c 43f1c445e8
commit a1c55207fc
1288 changed files with 120048 additions and 91740 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+5
View File
@@ -12,6 +12,8 @@ node_modules/
!tests/fixtures/npm-static/nested/inner/node_modules/
!tests/fixtures/npm-static/node_modules/*/node_modules/
!tests/fixtures/npm/store/*/node_modules/
!tests/fixtures/npm/node_modules/*/node_modules/
!tests/fixtures/npm/workspace/*/node_modules/
!tests/fixtures/npm/cases/*/node_modules/
!tests/library-mode/*/node_modules/
!tests/fixtures/provenance/node_modules/
@@ -49,3 +51,6 @@ tmp-*/
.env.local
.env*
.env*.local
# the vendored fs-extra corpus package: its lib/output module and jsonfile's own universalify
!tests/corpus/fs-extra-package/node_modules/fs-extra/lib/output/
!tests/corpus/fs-extra-package/node_modules/jsonfile/node_modules/
AGENTS.md
+6 -5
View File
@@ -4,7 +4,7 @@ This is the only agent guide for the repository. Its rules apply everywhere, inc
## Markdown style
Write prose paragraphs and list items as single source lines and let editors and renderers wrap them naturally. Do not hard-wrap Markdown at a fixed column; preserve separate lines only where Markdown syntax or intentional formatting requires them, such as headings, lists, tables, blockquotes, and code blocks.
Write prose paragraphs and list items as single source lines and let editors and renderers wrap them naturally. Do not hard-wrap Markdown at a fixed column; preserve separate lines only where Markdown syntax or intentional formatting requires them, such as headings, lists, tables, blockquotes, and code blocks. The Limitations page partials under `docs/content/limitations/` are the exception: they keep one sentence per line (see Limitations page partials).
## Build and test
@@ -140,10 +140,11 @@ Never hand-edit these files:
- `packages/compiler/surface-manifest.json`
- `internal/compatibility/generated/node-v24-internal.json`
- `internal/compatibility/generated/node-v24-backlog.json`
- `docs/src/generated/node-v24-compatibility.json`
Change compiler decision sources or the implementation-owned compatibility manifests, then regenerate. Use `pnpm manifest` when compiler decision tables change and `pnpm node-compat` whenever the Node pin, compatibility manifests, compiler surface manifest, changes.
Change compiler decision sources or the implementation-owned compatibility manifests, then regenerate. Use `pnpm manifest` when compiler decision tables change and `pnpm node-compat` whenever the Node pin, compatibility manifests, compiler surface manifest, changes. `pnpm node-compat --local` reclassifies the committed ledger without fetching Node's docs; it writes the same files as `pnpm node-compat` while the Node pin is unchanged.
Never resolve a merge conflict in these files by hand. Resolve the source conflicts, then run `node scripts/regen-generated.mjs`: it takes the merged-in branch's version of each conflicted generated file, regenerates all of them from the merged sources, and stages the ones that conflicted.
The public docs artifact must not expose repository test paths, internal source identifiers, engineering evidence bookkeeping, documentation/metadata headings, command-line or configuration-only entries, or rows that are N/A in the native tier. Those remain in the complete internal census under `internal/compatibility/generated/`.
@@ -158,7 +159,7 @@ For each API family:
5. Add differential evidence that executes under Node and as a compiled binary. Cover stdout, stderr, exit status, success behavior, and important error shapes. A load-only export check does not prove behavior.
6. Update the implementation-owned source or manifest. Positive `supported` and `partial` mappings must name existing test evidence.
7. Run `pnpm manifest` if compiler surface tables changed, then run `pnpm node-compat`. Never patch generated JSON or headers.
8. Inspect the internal ledger, public artifact, and backlog diff. Confirm the intended API family changed and unrelated rows did not.
8. Inspect the internal ledger and public artifact diff, and compare `pnpm node-compat:backlog` output before and after. Confirm the intended API family changed and unrelated rows did not.
9. Run focused tests, `pnpm node-compat:check`, and the full validation gate required for the change. If docs output changed, also run the docs gate.
Compatibility commands:
@@ -196,7 +197,7 @@ The docs site is a standalone pnpm workspace under `docs/`. It uses Next.js App
- A labeled bullet (`- **Crypto:** ...`) is `docs/content/limitations/<section>/<label>.mdx`, for example `builtin-apis/crypto.mdx`. Edit that file to change the bullet.
- The other text of a section (paragraphs, code blocks, unlabeled lists) is `docs/content/limitations/<section>.mdx`; text before the first heading is `introduction.mdx`.
- `<section>` and `<label>` are the heading and label text in lowercase with each run of other characters replaced by `-`.
- Write one sentence per line; a bullet's continuation lines are indented two spaces. Markdown joins the lines, and edits to different sentences don't conflict.
- Write one sentence per line; a bullet's continuation lines are indented two spaces. Markdown joins the lines, and edits to different sentences don't conflict. `pnpm lint` fails on a line that holds two sentences, and `node docs/scripts/limitations-partials.mjs fix` splits such lines.
- To add a bullet, add its file and an include line at its position in `limitations.mdx`, with a blank line on each side. Consecutive bullet includes render as one list.
- To add a section, add the heading to `limitations.mdx` and include the section's partial below it.
benchmarks/runtime/README.md
+25 -23
View File
@@ -4,33 +4,35 @@ Application-shaped workloads for measuring the speed of compiled executables. Ea
| Workload | Category | Exercises |
| --- | --- | --- |
| `json-records` | cpu | `JSON.stringify`, typed `JSON.parse`, Map aggregation |
| `regex-logs` | cpu | regex `exec` with groups, `test`, global `replace` |
| `ast-interp` | cpu | tokenizer, recursive-descent parser, class dispatch |
| `records-sort` | cpu | comparator sorts, group-by, `reduce`/`filter` |
| `template-render` | cpu | template literals, escaping, `map`/`join` |
| `functional-pipeline` | cpu | `filter`/`map`/`reduce`/`flatMap` chains, closures, small records |
| `async-pipeline` | cpu | `async`/`await`, `Promise.all` fan-out |
| `word-graph` | cpu | string-keyed Map/Set, BFS |
| `alloc-trees` | cpu | allocation-heavy trees, immutable object-spread updates |
| `cyclic-graphs` | cpu | object graphs with back references built and dropped: trees with parent and sibling links, rings of records sharing a long-lived registry, closures capturing their owner |
| `numeric-kernels` | cpu | sieve, hashing, `number[][]` matmul, `Float64Array` |
| `typed-array-scan`, `typed-array-scan-workers` | cpu | element-by-element scan loops over `Uint8Array`, `Uint16Array`, `Int32Array` and `Float64Array`; the second entry is a worker-enabled executable (it can start a worker), so each access also checks for shared storage |
| `intern-table`, `intern-table-map`, `intern-table-1`, `intern-table-map-1` | cpu | worker threads interning the same type-like objects under string keys and records under number keys, then hits and misses: one shared `InternTable` per kind of key, or a module-level `Map` per thread (the `-map` entries), with 8 threads or 1 (the `-1` entries); under Node.js every thread has its own tables |
| `thread-slots`, `thread-slots-cache` | cpu | worker threads reading one published type graph from an `InternTable` and keeping per-thread state for each type they visit: a `ThreadSlots` family, or a direct-mapped cache by id in front of a `Map` by id (the `-cache` entry); under Node.js both are per thread |
| `channel-messages` | cpu | 200,000 MessageChannel messages in one thread, half numbers and half small objects, all posted before the port delivers any |
| `ast-interp` | cpu | tokenizer, recursive-descent parser, class dispatch |
| `async-pipeline` | cpu | `async`/`await`, `Promise.all` fan-out |
| `atomics-contended` | cpu | four worker threads updating `SharedArrayBuffer` counters with `Atomics`: one shared counter, one counter per worker, a `compareExchange` loop, and `Atomics.wait`/`notify` handoffs |
| `value-objects` | cpu | small immutable records and class instances: vector helpers, method chains, particle updates |
| `value-records` | cpu | records that hold strings, spread state updates and destructured tuples: token helpers, an immutable reducer, tuple returns |
| `region-builds` | cpu | a watch-mode build loop over `LifetimeRegion`: each build binds a program in a new region and releases the previous one, which a cache keeps referenced for two builds |
| `optional-fields` | cpu | records and class instances with `number \| undefined` and other scalar-union fields |
| `record-boundaries` | cpu | records built from unchecked array reads passed to and returned from annotated functions |
| `number-maps` | cpu | number-keyed Map/Set: a dense-id table, sparse ids in many small maps, get/has/set mixes |
| `published-graphs` | cpu | syntax trees with parent links and a symbol table, published with `@scriptc/threads` and then read |
| `validate-errors` | cpu | throw/catch on a fraction of inputs, small call chains |
| `channel-messages` | cpu | 200,000 MessageChannel messages in one thread, half numbers and half small objects, all posted before the port delivers any |
| `csv-numbers` | cpu | `split`, `Number`, `parseInt`, `toFixed` |
| `cyclic-graphs` | cpu | object graphs with back references built and dropped: trees with parent and sibling links, rings of records sharing a long-lived registry, closures capturing their owner |
| `functional-pipeline` | cpu | `filter`/`map`/`reduce`/`flatMap` chains, closures, small records |
| `intern-table`, `intern-table-map`, `intern-table-1`, `intern-table-map-1` | cpu | worker threads interning the same type-like objects under string keys and records under number keys, then hits and misses: one shared `InternTable` per kind of key, or a module-level `Map` per thread (the `-map` entries), with 8 threads or 1 (the `-1` entries); under Node.js every thread has its own tables |
| `json-records` | cpu | `JSON.stringify`, typed `JSON.parse`, Map aggregation |
| `log-lines` | cpu | 200k immediate `console.log`/`process.stdout.write` lines into the runner's pipe |
| `log-summary`, `inventory-report` | cpu | the multi-module applications from `benchmarks/builds`, on 300k-line inputs |
| `number-maps` | cpu | number-keyed Map/Set: a dense-id table, sparse ids in many small maps, get/has/set mixes |
| `numeric-kernels` | cpu | sieve, hashing, `number[][]` matmul, `Float64Array` |
| `optional-fields` | cpu | records and class instances with `number \| undefined` and other scalar-union fields |
| `published-graphs` | cpu | syntax trees with parent links and a symbol table, published with `@scriptc/threads` and then read |
| `record-boundaries` | cpu | records built from unchecked array reads passed to and returned from annotated functions |
| `records-sort` | cpu | comparator sorts, group-by, `reduce`/`filter` |
| `regex-logs` | cpu | regex `exec` with groups, `test`, global `replace` |
| `region-builds` | cpu | a watch-mode build loop over `LifetimeRegion`: each build binds a program in a new region and releases the previous one, which a cache keeps referenced for two builds |
| `template-render` | cpu | template literals, escaping, `map`/`join` |
| `thread-slots`, `thread-slots-cache` | cpu | worker threads reading one published type graph from an `InternTable` and keeping per-thread state for each type they visit: a `ThreadSlots` family, or a direct-mapped cache by id in front of a `Map` by id (the `-cache` entry); under Node.js both are per thread |
| `typed-array-scan`, `typed-array-scan-workers` | cpu | element-by-element scan loops over `Uint8Array`, `Uint16Array`, `Int32Array` and `Float64Array`; the second entry is a worker-enabled executable (it can start a worker), so each access also checks for shared storage |
| `unicode-lexer` | cpu | a hand-written lexer over a 2.7M-unit bundled module whose comments and strings are in several languages: `charCodeAt` scans, line starts |
| `validate-errors` | cpu | throw/catch on a fraction of inputs, small call chains |
| `value-objects` | cpu | small immutable records and class instances: vector helpers, method chains, particle updates |
| `value-records` | cpu | records that hold strings, spread state updates and destructured tuples: token helpers, an immutable reducer, tuple returns |
| `word-graph` | cpu | string-keyed Map/Set, BFS |
| `worker-messages` | cpu | message round trips between the main thread and one worker: each posts a number and the other answers it, so every round trip is two messages and two wakeups of a waiting thread |
| `cli-config` | startup | a `deploy-plan` CLI: read a ~110 KB JSON manifest, validate, interpolate env values, order 80 services into dependency waves, print a summary |
| `http-api` | server | a `node:http` JSON API: routing, query parsing, a seeded catalog, per-cart state, typed JSON bodies, `ETag`/`If-None-Match`, auth and request-id headers |
@@ -46,7 +48,7 @@ Node is timed in the same interleaved loop and every row reports candidate/Node;
## Workload kinds
Entries in `workloads.json` are process workloads unless they set `kind`.
Each workload is a JSON file in `workloads/`, named after the workload, that holds its `entry`, `args` and options; adding a workload adds its file and its row in the table above, kept in category and then name order. Workloads are process workloads unless they set `kind`.
Process workloads time whole launches (wall time, startup included). An optional `launches` count makes each sample the median of that many sequential launches; `cli-config` uses 20 so that a startup-dominated command (a few milliseconds compiled, tens of milliseconds under Node) is measured with low noise. Its table row is labeled `(per launch)`. On a heavily loaded macOS host, whole batches of launches can run several times slower (samples become bimodal), so judge startup changes on the Linux lane or a quiet machine.
benchmarks/runtime/workloads.json
-283
View File
@@ -1,283 +0,0 @@
{
"workloads": [
{
"name": "json-records",
"entry": "workloads/json-records.ts",
"args": [
"4"
]
},
{
"name": "regex-logs",
"entry": "workloads/regex-logs.ts",
"args": [
"1"
]
},
{
"name": "ast-interp",
"entry": "workloads/ast-interp.ts",
"args": [
"5"
]
},
{
"name": "records-sort",
"entry": "workloads/records-sort.ts",
"args": [
"1"
]
},
{
"name": "template-render",
"entry": "workloads/template-render.ts",
"args": [
"1"
]
},
{
"name": "functional-pipeline",
"entry": "workloads/functional-pipeline.ts",
"args": [
"1"
]
},
{
"name": "async-pipeline",
"entry": "workloads/async-pipeline.ts",
"args": [
"1"
]
},
{
"name": "word-graph",
"entry": "workloads/word-graph.ts",
"args": [
"2"
]
},
{
"name": "alloc-trees",
"entry": "workloads/alloc-trees.ts",
"args": [
"1"
]
},
{
"name": "cyclic-graphs",
"entry": "workloads/cyclic-graphs.ts",
"args": [
"3"
]
},
{
"name": "numeric-kernels",
"entry": "workloads/numeric-kernels.ts",
"args": [
"0.5"
]
},
{
"name": "typed-array-scan",
"entry": "workloads/typed-array-scan.ts",
"args": [
"3"
]
},
{
"name": "typed-array-scan-workers",
"entry": "workloads/typed-array-scan-workers.ts",
"args": [
"3"
]
},
{
"name": "intern-table",
"entry": "workloads/intern-table.ts",
"args": [
"table",
"8",
"100000"
]
},
{
"name": "intern-table-map",
"entry": "workloads/intern-table.ts",
"args": [
"map",
"8",
"100000"
]
},
{
"name": "intern-table-1",
"entry": "workloads/intern-table.ts",
"args": [
"table",
"1",
"100000"
]
},
{
"name": "intern-table-map-1",
"entry": "workloads/intern-table.ts",
"args": [
"map",
"1",
"100000"
]
},
{
"name": "thread-slots",
"entry": "workloads/thread-slots.ts",
"args": [
"slots",
"8",
"50000",
"20"
]
},
{
"name": "thread-slots-cache",
"entry": "workloads/thread-slots.ts",
"args": [
"cache",
"8",
"50000",
"20"
]
},
{
"name": "channel-messages",
"entry": "workloads/channel-messages.ts",
"args": [
"200000"
]
},
{
"name": "atomics-contended",
"entry": "workloads/atomics-contended.ts",
"args": [
"2"
]
},
{
"name": "value-objects",
"entry": "workloads/value-objects.ts",
"args": [
"2"
]
},
{
"name": "value-records",
"entry": "workloads/value-records.ts",
"args": [
"1"
]
},
{
"name": "region-builds",
"entry": "workloads/region-builds.ts",
"args": [
"1"
]
},
{
"name": "optional-fields",
"entry": "workloads/optional-fields.ts",
"args": [
"10"
]
},
{
"name": "record-boundaries",
"entry": "workloads/record-boundaries.ts",
"args": [
"8"
]
},
{
"name": "number-maps",
"entry": "workloads/number-maps.ts",
"args": [
"4"
]
},
{
"name": "published-graphs",
"entry": "workloads/published-graphs.ts",
"args": [
"1"
]
},
{
"name": "validate-errors",
"entry": "workloads/validate-errors.ts",
"args": [
"1"
]
},
{
"name": "csv-numbers",
"entry": "workloads/csv-numbers.ts",
"args": [
"1"
]
},
{
"name": "log-lines",
"entry": "workloads/log-lines.ts",
"args": [
"4"
]
},
{
"name": "log-summary",
"entry": "../builds/log-summary/main.ts",
"input": "log-summary",
"args": [
"{input}",
"--min-status",
"200",
"--limit",
"12"
]
},
{
"name": "inventory-report",
"entry": "../builds/inventory-report/main.ts",
"input": "inventory-report",
"args": [
"{input}"
]
},
{
"name": "cli-config",
"entry": "workloads/cli-config.ts",
"input": "cli-config",
"launches": 20,
"args": [
"{input}",
"--env",
"production",
"--max-wave",
"12"
]
},
{
"name": "http-api",
"kind": "server",
"entry": "workloads/http-api.ts",
"args": [],
"client": {
"script": "clients/http-load.mjs",
"args": [
"--connections=64",
"--threads=2",
"--warmup=4000",
"--requests=40000"
]
}
}
]
}
benchmarks/runtime/workloads.mjs
+33
View File
@@ -0,0 +1,33 @@
/* The runtime benchmark workloads: one JSON file per workload in
* workloads/, named after the workload, next to the entry programs. A file
* holds the workload's entry, args and options; changes that add
* different workloads add different files. */
import { existsSync, readdirSync, readFileSync } from "node:fs";
import { join } from "node:path";
const CATEGORIES = ["cpu", "startup", "server"];
/* Server workloads run a listening program; startup-bound ones time
* several sequential launches per sample. */
export const workloadCategory = (w) =>
w.kind === "server" ? "server" : (w.launches ?? 1) > 1 ? "startup" : "cpu";
/* The workloads of the suite at `suite` (a benchmarks/runtime directory),
* by category and then name. A checkout from before the per-workload files
* lists them in workloads.json, in its own order. */
export function readWorkloads(suite) {
const legacy = join(suite, "workloads.json");
if (existsSync(legacy)) return JSON.parse(readFileSync(legacy, "utf8")).workloads;
const dir = join(suite, "workloads");
return readdirSync(dir)
.filter((file) => file.endsWith(".json"))
.map((file) => ({
name: file.slice(0, -".json".length),
...JSON.parse(readFileSync(join(dir, file), "utf8")),
}))
.sort(
(a, b) =>
CATEGORIES.indexOf(workloadCategory(a)) - CATEGORIES.indexOf(workloadCategory(b)) ||
(a.name < b.name ? -1 : a.name > b.name ? 1 : 0),
);
}
benchmarks/runtime/workloads/alloc-trees.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/alloc-trees.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/ast-interp.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/ast-interp.ts",
"args": [
"5"
]
}
benchmarks/runtime/workloads/async-pipeline.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/async-pipeline.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/atomics-contended.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/atomics-contended.ts",
"args": [
"2"
]
}
benchmarks/runtime/workloads/channel-messages.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/channel-messages.ts",
"args": [
"200000"
]
}
benchmarks/runtime/workloads/cli-config.json
+12
View File
@@ -0,0 +1,12 @@
{
"entry": "workloads/cli-config.ts",
"input": "cli-config",
"launches": 20,
"args": [
"{input}",
"--env",
"production",
"--max-wave",
"12"
]
}
benchmarks/runtime/workloads/csv-numbers.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/csv-numbers.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/cyclic-graphs.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/cyclic-graphs.ts",
"args": [
"3"
]
}
benchmarks/runtime/workloads/functional-pipeline.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/functional-pipeline.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/http-api.json
+14
View File
@@ -0,0 +1,14 @@
{
"kind": "server",
"entry": "workloads/http-api.ts",
"args": [],
"client": {
"script": "clients/http-load.mjs",
"args": [
"--connections=64",
"--threads=2",
"--warmup=4000",
"--requests=40000"
]
}
}
benchmarks/runtime/workloads/intern-table-1.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"table",
"1",
"100000"
]
}
benchmarks/runtime/workloads/intern-table-map-1.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"map",
"1",
"100000"
]
}
benchmarks/runtime/workloads/intern-table-map.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"map",
"8",
"100000"
]
}
benchmarks/runtime/workloads/intern-table.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"table",
"8",
"100000"
]
}
benchmarks/runtime/workloads/inventory-report.json
+7
View File
@@ -0,0 +1,7 @@
{
"entry": "../builds/inventory-report/main.ts",
"input": "inventory-report",
"args": [
"{input}"
]
}
benchmarks/runtime/workloads/json-records.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/json-records.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/log-lines.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/log-lines.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/log-summary.json
+11
View File
@@ -0,0 +1,11 @@
{
"entry": "../builds/log-summary/main.ts",
"input": "log-summary",
"args": [
"{input}",
"--min-status",
"200",
"--limit",
"12"
]
}
benchmarks/runtime/workloads/number-maps.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/number-maps.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/numeric-kernels.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/numeric-kernels.ts",
"args": [
"0.5"
]
}
benchmarks/runtime/workloads/optional-fields.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/optional-fields.ts",
"args": [
"10"
]
}
benchmarks/runtime/workloads/published-graphs.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/published-graphs.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/record-boundaries.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/record-boundaries.ts",
"args": [
"8"
]
}
benchmarks/runtime/workloads/records-sort.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/records-sort.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/regex-logs.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/regex-logs.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/region-builds.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/region-builds.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/template-render.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/template-render.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/thread-slots-cache.json
+9
View File
@@ -0,0 +1,9 @@
{
"entry": "workloads/thread-slots.ts",
"args": [
"cache",
"8",
"50000",
"20"
]
}
benchmarks/runtime/workloads/thread-slots.json
+9
View File
@@ -0,0 +1,9 @@
{
"entry": "workloads/thread-slots.ts",
"args": [
"slots",
"8",
"50000",
"20"
]
}
benchmarks/runtime/workloads/typed-array-scan-workers.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/typed-array-scan-workers.ts",
"args": [
"3"
]
}
benchmarks/runtime/workloads/typed-array-scan.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/typed-array-scan.ts",
"args": [
"3"
]
}
benchmarks/runtime/workloads/unicode-lexer.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/unicode-lexer.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/unicode-lexer.ts
+116
View File
@@ -0,0 +1,116 @@
// A lexer over a large bundled module whose comments and strings are in
// several languages: line starts, tokens by kind and comment text, read
// with charCodeAt as hand-written scanners do. Each pass relexes the whole
// text, as a bundler or an editor does after a change.
const comments = [
"// Berechnet die Größe des Puffers für übergroße Einträge",
"// Calcule la taille maximale avant la prochaine itération",
"// バッファのサイズを計算します",
"/* 计算缓冲区大小，然后返回结果 */",
"// Returns the cached value (fast path) — see the note above",
"/** Résumé: the état of the café's ñandú registry */",
];
const strings = ['"Grüße"', '"naïve façade"', '"東京"', '"ok ✓"', '"plain ascii"', "'mañana'"];
const names = ["buffer", "größe", "index", "resultat", "count", "value", "entry", "table"];
let seed = 11;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
function pick(list: readonly string[]): string {
return list[Math.floor(random() * list.length)]!;
}
function makeSource(lines: number): string {
const out: string[] = [];
for (let i = 0; i < lines; i++) {
const r = random();
if (r < 0.2) out.push(pick(comments));
else if (r < 0.6) out.push(` const ${pick(names)}${i} = ${pick(names)}(${Math.floor(random() * 1000)}, ${pick(strings)});`);
else if (r < 0.85) out.push(` if (${pick(names)} >= ${i % 97}) return ${pick(names)}.${pick(names)} + 1;`);
else out.push(`function ${pick(names)}${i}(a, b) { return a * b - ${i}; }`);
}
return out.join("\n");
}
interface Counts {
lines: number;
identifiers: number;
numbers: number;
strings: number;
comments: number;
commentChars: number;
punctuation: number;
}
function isIdentifierStart(ch: number): boolean {
return (ch >= 97 && ch <= 122) || (ch >= 65 && ch <= 90) || ch === 95 || ch === 36 || ch > 127;
}
function isIdentifierPart(ch: number): boolean {
return isIdentifierStart(ch) || (ch >= 48 && ch <= 57);
}
function lex(text: string): Counts {
const counts: Counts = { lines: 1, identifiers: 0, numbers: 0, strings: 0, comments: 0, commentChars: 0, punctuation: 0 };
let pos = 0;
const end = text.length;
while (pos < end) {
const ch = text.charCodeAt(pos);
if (ch === 10) {
counts.lines++;
pos++;
} else if (ch === 32 || ch === 9 || ch === 13) {
pos++;
} else if (ch === 47 && text.charCodeAt(pos + 1) === 47) {
const start = pos;
while (pos < end && text.charCodeAt(pos) !== 10) pos++;
counts.comments++;
counts.commentChars += pos - start;
} else if (ch === 47 && text.charCodeAt(pos + 1) === 42) {
const start = pos;
pos += 2;
while (pos < end && !(text.charCodeAt(pos) === 42 && text.charCodeAt(pos + 1) === 47)) {
if (text.charCodeAt(pos) === 10) counts.lines++;
pos++;
}
pos += 2;
counts.comments++;
counts.commentChars += pos - start;
} else if (ch === 34 || ch === 39) {
pos++;
while (pos < end && text.charCodeAt(pos) !== ch) pos++;
pos++;
counts.strings++;
} else if (ch >= 48 && ch <= 57) {
while (pos < end && text.charCodeAt(pos) >= 48 && text.charCodeAt(pos) <= 57) pos++;
counts.numbers++;
} else if (isIdentifierStart(ch)) {
while (pos < end && isIdentifierPart(text.charCodeAt(pos))) pos++;
counts.identifiers++;
} else {
counts.punctuation++;
pos++;
}
}
return counts;
}
function lineStarts(text: string): number[] {
const starts = [0];
for (let pos = 0; pos < text.length; pos++) if (text.charCodeAt(pos) === 10) starts.push(pos + 1);
return starts;
}
const scale = Number(process.argv[2] ?? "1");
const source = makeSource(60000);
const passes = Math.round(6 * scale);
let last = lex(source);
let total = last.identifiers + lineStarts(source).length;
for (let i = 1; i < passes; i++) {
last = lex(source);
total += last.identifiers + lineStarts(source).length;
}
console.log("units", source.length, "passes", passes, "total", total);
console.log(JSON.stringify(last));
benchmarks/runtime/workloads/validate-errors.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/validate-errors.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/value-objects.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/value-objects.ts",
"args": [
"2"
]
}
benchmarks/runtime/workloads/value-records.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/value-records.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/word-graph.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/word-graph.ts",
"args": [
"2"
]
}
benchmarks/runtime/workloads/worker-messages.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/worker-messages.ts",
"args": [
"100000"
]
}
benchmarks/runtime/workloads/worker-messages.ts
+24
View File
@@ -0,0 +1,24 @@
// Message round trips between the main thread and one worker: the main
// thread posts a number and the worker answers it plus one, the given
// number of times. Each round trip is two messages and two wakeups of a
// waiting thread. Argument: the number of round trips.
import { Worker, isMainThread, parentPort } from "node:worker_threads";
if (isMainThread) {
const rounds = Number(process.argv[2] ?? "100000");
const worker = new Worker(new URL(import.meta.url));
let sent = 0;
let sum = 0;
worker.on("message", (value: number) => {
sum += value;
if (++sent === rounds) {
console.log(`${rounds} round trips, sum ${sum}`);
void worker.terminate();
} else worker.postMessage(sent);
});
worker.postMessage(0);
} else {
parentPort!.on("message", (value: number) => {
parentPort!.postMessage(value + 1);
});
}
benchmarks/scaling/strace-workloads.mjs
+2 -1
View File
@@ -7,6 +7,7 @@ import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { parseArgs } from "node:util";
import { readWorkloads } from "../runtime/workloads.mjs";
const here = dirname(fileURLToPath(import.meta.url));
const suite = resolve(here, "../runtime");
@@ -18,7 +19,7 @@ const { values } = parseArgs({
},
});
mkdirSync(values.out, { recursive: true });
const { workloads } = JSON.parse(readFileSync(join(suite, "workloads.json"), "utf8"));
const workloads = readWorkloads(suite);
const cli = join(values.scriptc, "packages/cli/dist/bootstrap.js");
// Same inputs as scripts/bench-runtime.mjs generates.
function input(kind) {
docs/content/docs/dependencies.mdx
+52 -1
View File
@@ -73,7 +73,58 @@ Some packages your program imports directly ship no declarations, so their types
$ scriptc build app.ts --npm-static=three -o app >/dev/null
```
Naming a package skips the declaration requirement above, but the package still has to compile. Add a trailing `...` to select a package together with every package it depends on, for example `--npm-static=fs-extra...`. `--npm-static=auto` uses the default automatic selection. The option accepts a comma-separated list and can be repeated.
Naming a package skips the declaration requirement above, but the package still has to compile. Add a trailing `...` to select a package together with every package it depends on, for example `--npm-static=semver...`. `--npm-static=auto` uses the default automatic selection. The option accepts a comma-separated list and can be repeated.
## Built-in packages
scriptc implements a few packages itself and compiles that implementation in place of the package's published JavaScript. Which implementation compiles depends on the version your program has installed, so an import resolves the same way it does under Node.js: a package that isn't installed is still a missing package, and a version scriptc doesn't implement compiles from its own files as usual.
`undici` 5.x and 7.x are built in. The implementation covers `request()`, `Dispatcher` (including subclasses that override `dispatch`), `Client`, `Pool` with its `stats`, `Agent` with per-origin `factory` functions and `connect` options, `buildConnector`, the `errors` classes and their `UND_ERR_*` codes, `setGlobalDispatcher` and `getGlobalDispatcher`, `fetch` and `Headers`, and the request events undici publishes on `node:diagnostics_channel`. Requests use HTTP/1.1 with keep-alive connection reuse and undici's headers, body and idle timeouts, and each version keeps its own behavior where 5.x and 7.x differ, such as the error an aborted request rejects with. The package's other exports aren't declared, so using one is a type error at build time. Known differences:
- A response body supports the readable-stream members undici's body mixin uses: `on`, `once`, `pause`, `resume`, `destroy`, `text()`, `json()`, `arrayBuffer()`, `bytes()`, `blob()` and `dump()`. It can't be piped or iterated with `for await`.
- Diagnostics channel messages carry the request but not the socket or connector.
- `fetch` with a dispatcher returns a response whose `url` is empty and whose `redirected` is `false`, offers only `gzip, deflate` encodings, and sends string and byte bodies. Without a dispatcher it is the built-in `fetch`.
- Request bodies are strings and bytes; stream, iterable and form-data bodies, `pipelining` above 1, `CONNECT` requests and Unix socket paths aren't supported.
`fs-extra` 8.x through 11.x is built in, and its declarations replace `@types/fs-extra`. The default import, a namespace import and named imports all read the same exports. fs's members are re-exported, and fs-extra's wrapped methods follow its rule: with a callback as the last argument they call back, otherwise they return a promise. The extra methods cover `copy`, `copySync`, `move`, `moveSync`, `remove`, `removeSync`, `emptyDir`, `ensureDir`, `mkdirs`, `mkdirp`, `ensureFile`, `pathExists`, `outputFile`, `readJson`, `readJsonSync`, `writeJson`, `writeJsonSync`, `outputJson` and their aliases, with fs-extra's error messages. Known differences:
- `ensureLink`, `ensureSymlink`, `createLink`, `createSymlink`, `gracefulify`, the promise forms of the descriptor-based methods, and the callback forms of `stat`, `lstat`, `readdir`, `mkdtemp`, `realpath` and `readJson` aren't declared, so using one is a type error at build time. Paths are strings, text is UTF-8, and write flags, string modes and `copyFile` mode flags aren't accepted.
- graceful-fs's retry queue, which fs-extra uses under Node.js, isn't part of the implementation, so a program that runs out of file descriptors fails at the first `EMFILE` error instead of retrying.
- An ES module that imports fs-extra can use any of its exports by name. Node.js only exposes the names its CommonJS lexer finds in fs-extra's source, so a name like `readFile` throws a `SyntaxError` there.
Small polyfill packages whose module is exactly a Node.js built-in on Node.js 24 compile as that built-in, at the `require()` that loads them, from your program's CommonJS files, from compiled packages, and from esbuild bundles whose module paths name the package and its version (as pnpm installs do). The package's own files aren't compiled.
<table>
<thead>
<tr><th>Package</th><th>Versions</th><th>Compiles as</th></tr>
</thead>
<tbody>
<tr><td><code>isarray</code></td><td>0.x, 1.x, 2.x</td><td><code>Array.isArray</code></td></tr>
<tr><td><code>safe-buffer</code></td><td>5.x</td><td><code>require("buffer")</code></td></tr>
<tr><td><code>process-nextick-args</code></td><td>2.x</td><td><code>process</code></td></tr>
<tr><td><code>inherits</code></td><td>2.x</td><td><code>require("util").inherits</code></td></tr>
<tr><td><code>util-deprecate</code></td><td>1.x</td><td><code>require("util").deprecate</code></td></tr>
<tr><td><code>readable-stream</code></td><td>2.x, 3.x, 4.x</td><td><code>require("stream")</code> and its classes</td></tr>
</tbody>
</table>
`readable-stream` compiles as Node.js's built-in streams, which is its own `READABLE_STREAM=disable` mode for 2.x and 4.x. That can differ from its default behavior under Node.js; see [limitations](/docs/limitations#readable-stream).
## Optional dependencies that never load
Some packages load an optional dependency inside `try`/`catch` and work without it, such as `node-fetch` with `encoding`. To keep such a dependency out of the native build even when it is installed, mark it unloaded in your entry's `package.json`:
```json:package.json
{
"scriptc": {
"packages": {
"encoding": "unloaded"
}
}
}
```
A `require()` of an unloaded package throws the same `MODULE_NOT_FOUND` error Node.js throws when the package isn't installed, and the package isn't compiled.
## CommonJS packages
docs/content/docs/limitations.mdx
+4
View File
@@ -178,6 +178,10 @@ title: "Limitations"
<include>../limitations/buffer-construction.mdx</include>
### readable-stream
<include>../limitations/readable-stream.mdx</include>
### HTTP headers
<include>../limitations/http-headers.mdx</include>
docs/content/limitations/builtin-apis/crypto.mdx
+4 -1
View File
@@ -12,5 +12,8 @@
Decrypting chacha20-poly1305 without an authentication tag fails in `final()`; Node 24.22 returns unverified plaintext.
After `update()` fails, for example past the AES-GCM limit of 2^36 - 32 bytes per message, every later call on that cipher throws; Node still lets `final()` run.
An output encoding that is a function or an instance of a class defining `toString` throws a coded SC2020 error.
Public-key operations remain unsupported.
`createECDH` and `ECDH.convertKey` support prime192v1, secp224r1, prime256v1, secp384r1, secp521r1, secp192k1, secp224k1, secp256k1, brainpoolP256r1, brainpoolP384r1, and brainpoolP512r1, on native targets, through `generateKeys()`, `computeSecret()`, `getPublicKey()`, `getPrivateKey()`, `setPrivateKey()`, and `setPublicKey()`, in compressed, uncompressed, and hybrid forms; the other curves `crypto.getCurves()` lists and wasm32-wasi builds are refused.
A call on an ECDH value whose arguments no @types/node overload accepts (`getPublicKey("HEX")`) is refused at compile time, as is `computeSecret(view, encoding)`: Node reads that encoding as the input encoding and answers a Buffer where @types/node declares a string.
ECDH property reads and writes throw a coded SC2020 error.
Other public-key operations remain unsupported.
Native derivations finish before their deferred callbacks, so computation blocks the caller.
docs/content/limitations/builtin-function-values.mdx
+5
View File
@@ -8,10 +8,15 @@ Some builtins can be stored and passed as functions:
In JavaScript, `setTimeout`, `setInterval`, `setImmediate`, `clearTimeout`, `clearInterval`, `clearImmediate`, `queueMicrotask`, and `process.nextTick` are function values wherever they are stored, including conditional and logical expressions and assignments: `typeof` answers `"function"`, identity holds across spellings, and calls schedule as the direct calls do, returning the same numeric timer handles.
Calls through these values bypass `node:test` mock timers.
Other builtin globals stored in JavaScript remain opaque identity values.
In JavaScript, `require("events")` and the `EventEmitter` class assigned to an untyped binding or stored in an object (`ReleaseEmitter = EventEmitter`) are the untyped `EventEmitter` constructor that constructor functions inherit from (see below): `new` through it builds such an emitter object, and identity and `instanceof` agree with the class.
A class can extend `require("events")` directly.
Other immutable aliases of table-backed Node builtins support direct calls, but cannot be passed elsewhere.
`util.promisify` supports compile-time conversions of `child_process.execFile`, `child_process.exec`, `setTimeout` and `setImmediate` (to the `timers/promises` functions, as in Node), `stream.finished` and `stream.pipeline` (to the `stream/promises` functions, as in Node), the `zlib` callback one-shots `deflate`, `inflate`, `deflateRaw`, `inflateRaw`, `gzip`, `gunzip` and `unzip` (with or without options), the `crypto` callback functions `randomBytes`, `pbkdf2`, `scrypt`, `hkdf`, `randomFill`, `randomInt` and `generateKey`, `dns.lookup` (settling with `{ address, family }`, or the address array with `all: true`) and `dns.lookupService` (settling with `{ hostname, service }`), and the `fs` callback functions that have an `fs/promises` counterpart (`access`, `appendFile`, `chmod`, `chown`, `copyFile`, `lchown`, `link`, `lstat`, `lutimes`, `mkdir`, `mkdtemp`, `readFile`, `readdir`, `readlink`, `realpath`, `rename`, `rm`, `rmdir`, `stat`, `statfs`, `symlink`, `truncate`, `unlink`, `utimes`, and `writeFile`); calls through the converted function lower as the `fs/promises` call.
`util.promisify` over an untyped (`any`) function returns a native wrapper that follows Node's callback rules and carries the original's `name` and `length`.
Such a function with other own properties, `util.promisify.custom` among them, throws `SC2020` where `util.promisify` runs, and the wrapper does not report Node's DEP0174 warning when the original returns a promise.
`util.styleText` supports standard formats and aliases, `#RGB` and `#RRGGBB` hex colors, arrays of formats, nested ANSI resets, and explicit color enabling with `{ validateStream: false }`.
Native calls accept standard process streams; custom native streams and mutations to `util.inspect.colors` remain unsupported.
docs/content/limitations/language-features/function-values.mdx
+3
View File
@@ -2,3 +2,6 @@
A function stored in a slot with a different signature remains the same function for equality, array searches, `length`, and own properties.
Typed rest-parameter functions can be stored and called indirectly, including through a slot with fixed parameters, where the slot's arguments arrive in the rest array.
Builtin function values have API-specific restrictions; see [Builtin function values](#builtin-function-values).
- **JavaScript parameters:** In a JavaScript file TypeScript does not check, a prototype method (`C.prototype.m = function (...) {}`) takes whatever values its callers pass; its JSDoc types document the usual values without constraining them.
A sloppy-mode function with named parameters may read `arguments` only where the aliasing of its parameters cannot show: it reads `arguments.length`, or reads elements and never writes a parameter or an element.
Other uses of `arguments` in such a function are refused with `SC1090`; ES modules, `'use strict'` code, classes, and functions with default, rest, or destructured parameters have no such restriction.
docs/content/limitations/readable-stream.mdx
+6
View File
@@ -0,0 +1,6 @@
`readable-stream` 2.x, 3.x and 4.x compile as Node.js's built-in `stream` module.
For 2.x and 4.x, that is the package's own behavior when the `READABLE_STREAM` environment variable is `disable`.
Without that variable, Node.js runs the package's JavaScript copy of the stream classes, so a native build can differ from Node.js wherever a program depends on that difference: `instanceof` checks against the built-in classes, class identity, error messages, and timing details.
The package's `lib/_stream_*.js` files also compile as the matching built-in classes, although Node.js loads the package's own implementations through them in every environment.
`readable-stream` 3.x is a copy of Node.js 10's streams.
Its own `disable` mode throws on Node.js 24, so no setting makes Node.js run it against the built-in classes; a native build maps the members its entry exports (the stream classes, `Stream`, `finished` and `pipeline`) to the built-in ones.
docs/content/limitations/udp-socket-options.mdx
+7
View File
@@ -1,2 +1,9 @@
`socket.addMembership()` and `socket.dropMembership()` on a `dgram` socket that is not bound yet throw `SC2020`; Node binds such a socket to an ephemeral port first, so bind the socket before joining a group.
The `SystemError` that the buffer size methods throw (`ERR_SOCKET_BUFFER_SIZE`, for example from `getRecvBufferSize()` before `bind()`) has Node's `message`, `code`, `errno` and `syscall`, but no `info` object.
In JavaScript, `socket.send()` also accepts lists of buffers, sends on a connected socket, and takes a completion callback, which receives `null` and the byte count, or the send's failure instead of an `'error'` event.
The callback runs after the socket's pending events, where Node runs it on the next tick.
In TypeScript, `send()` with a callback, a list, or on a connected socket fails to compile with `SC2020`.
JavaScript can read a socket's `on`, `once`, `addListener`, `removeListener`, `off`, `emit`, `send`, `close`, `bind`, `connect`, `disconnect`, `address`, `ref`, and `unref` as function values, for example `socket.emit.bind(socket)`; reading other members as values is refused with `SC1090`.
Listeners registered that way, or with untyped parameters, take only the `message`, `error`, `close`, `listening`, and `connect` events.
When a socket closes, listeners registered from TypeScript are released, so an `emit()` from JavaScript after `'close'` reaches only listeners registered from JavaScript.
docs/content/limitations/unix-domain-sockets.mdx
+7 -2
View File
@@ -1,2 +1,7 @@
`server.listen(path)` and the `socketPath` option of `http.request()` and `https.request()` use Unix domain sockets on Linux and macOS. On Windows, where Node maps these paths to named pipes, they fail with `ENOTSUP`. `server.address()` on a server listening on a Unix socket returns the socket path, as in Node, when the result is typed `string | AddressInfo | null` (the @types/node declaration). Where it is typed as the `{ address, family, port }` record alone, which cannot hold the path, the call throws an error for such a server. `listen(path, backlog, callback)` is refused at compile time; pass `listen(path, callback)`.
When the options of `http.request()` come from a variable, or from an object literal that spreads one, they are read at run time with Node's validation errors. The `auth`, `createConnection`, `family`, `hints`, `insecureHTTPParser`, `joinDuplicateHeaders`, `localAddress`, `localPort`, `lookup` and `uniqueHeaders` options then throw a runtime error when they hold a value, and so do `setHost: false` and `setDefaultHeaders: false`.
`server.listen(path)` and the `socketPath` option of `http.request()` and `https.request()` use Unix domain sockets on Linux and macOS.
On Windows, where Node maps these paths to named pipes, they fail with `ENOTSUP`.
`server.address()` on a server listening on a Unix socket returns the socket path, as in Node, when the result is typed `string | AddressInfo | null` (the @types/node declaration).
Where it is typed as the `{ address, family, port }` record alone, which cannot hold the path, the call throws an error for such a server.
`listen(path, backlog, callback)` is refused at compile time; pass `listen(path, callback)`.
When the options of `http.request()` come from a variable, or from an object literal that spreads one, they are read at run time with Node's validation errors.
The `auth`, `createConnection`, `family`, `hints`, `insecureHTTPParser`, `joinDuplicateHeaders`, `localAddress`, `localPort`, `lookup` and `uniqueHeaders` options then throw a runtime error when they hold a value, and so do `setHost: false` and `setDefaultHeaders: false`.
docs/scripts/limitations-partials.mjs
+54 -2
View File
@@ -21,7 +21,13 @@
* node scripts/limitations-partials.mjs split <file> [page] write the page and partials from one file
* node scripts/limitations-partials.mjs merge <base> <theirs>
* three-way merge a branch's single-file page (<theirs>, forked from
* <base>) into the current partials, then split the result */
* <base>) into the current partials, then split the result
* node scripts/limitations-partials.mjs check fail on a partial line that holds two sentences
* node scripts/limitations-partials.mjs fix put each such sentence on its own line
*
* One sentence per line keeps changes to different sentences on different
* lines, so concurrent edits merge. Markdown joins the lines with a space,
* so the rendered page is the same. */
import { execFileSync } from "node:child_process";
import { mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
@@ -139,6 +145,41 @@ function write(text, page = defaultPage) {
console.error(`${relative(docsRoot, page)}: ${partials.size} partials in ${relative(docsRoot, partialsDir)}/`);
}
// A sentence ends at `.`, `!` or `?` (after any closing `)`, `"` or `*`)
// followed by a space and a capital letter, a code span, a link or bold
// text. Abbreviations don't end one.
const sentenceEnd = /(?<!\b(?:e\.g|i\.e|etc|vs|cf))([.!?][)"*]*) (?=[A-Z`[*])/g;
/** The partial text with each sentence of a prose line on its own line;
* a bullet's sentences continue with the two-space indent. Code fences,
* headings, tables, HTML and include lines are left alone, as are code
* spans and link targets. */
export function splitSentences(text) {
let inFence = false;
const out = [];
for (const line of text.split("\n")) {
if (/^\s*(```|~~~)/.test(line)) inFence = !inFence;
if (inFence || /^\s*(```|~~~|#|<|\||$)/.test(line)) {
out.push(line);
continue;
}
const indent = /^\s*(?:[-*]|\d+\.) /.test(line) ? " " : line.match(/^\s*/)[0];
// Mask code spans (all but the opening backtick, which can start a
// sentence) and link targets, so their periods never split.
const masked = line.replace(/`[^`]*`|\]\([^)]*\)/g, (m) => m[0] + "\0".repeat(m.length - 1));
const pieces = [];
let start = 0;
for (const match of masked.matchAll(sentenceEnd)) {
const end = match.index + match[1].length;
pieces.push(line.slice(start, end));
start = end + 1;
}
pieces.push(line.slice(start));
out.push(pieces[0], ...pieces.slice(1).map((piece) => indent + piece));
}
return out.join("\n");
}
function listPartials(dir = partialsDir) {
return readdirSync(dir, { withFileTypes: true }).flatMap((entry) =>
entry.isDirectory() ? listPartials(join(dir, entry.name)) : [join(dir, entry.name)],
@@ -175,9 +216,20 @@ if (command === "join") {
} finally {
rmSync(work, { recursive: true, force: true });
}
} else if (command === "check" || command === "fix") {
let unsplit = 0;
for (const file of listPartials().sort()) {
const text = readFileSync(file, "utf8");
const split = splitSentences(text);
if (split === text) continue;
unsplit++;
if (command === "fix") writeFileSync(file, split);
else console.error(`${relative(docsRoot, file)}: put each sentence on its own line (node docs/scripts/limitations-partials.mjs fix)`);
}
if (command === "check" && unsplit > 0) process.exitCode = 1;
} else if (command === "list") {
for (const file of listPartials().sort()) console.log(relative(docsRoot, file));
} else {
console.error("usage: limitations-partials.mjs join [page] | split <file> [page] | merge <base> <theirs> | list");
console.error("usage: limitations-partials.mjs join [page] | split <file> [page] | merge <base> <theirs> | check | fix | list");
process.exitCode = 2;
}
docs/src/generated/node-v24-compatibility-meta.json
-7
View File
@@ -1,7 +0,0 @@
{
"schemaVersion": 3,
"nodeVersion": "24.22.0",
"nodeCommit": "f14cd3aa7bf9ca8df2cc5b8bb14beb0f335b1d93",
"artifactVersion": "ee72820d45a79aa154f0",
"rowCount": 3880
}
docs/src/generated/node-v24-compatibility.json
+86 -590
View File
File diff suppressed because it is too large Load Diff
internal/compatibility/README.md
+6 -2
View File
@@ -5,11 +5,15 @@ This workspace package owns scriptc's Node.js parity system.
- `node-v24.json` pins the Node release tag, commit, and canonical docs/source inputs.
- `static-support.json` overlays dedicated compiler paths and their internal test evidence on the compiler's generated surface manifest.
- `generated/node-v24-internal.json` is the engineering ledger, including implementation evidence and repository test paths.
- `generated/node-v24-backlog.json` is the internal tier-specific work queue. It separates verification, implementation, explicit-refusal replacement, and partial-surface audit work, and prioritizes stable Node APIs above experimental, deprecated, and legacy APIs.
- `pnpm node-compat:backlog` derives the internal tier-specific work queue from the ledger; it is not committed. It separates verification, implementation, explicit-refusal replacement, and partial-surface audit work, and prioritizes stable Node APIs above experimental, deprecated, and legacy APIs.
- `docs/src/generated/node-v24-compatibility.json` is the stripped public artifact consumed by the docs website. It contains compiler-relevant runtime APIs, statuses, verification bases, and user-facing details; documentation, metadata, command-line/configuration entries, and rows that are N/A for native execution remain internal.
Statuses are evidence-aware: `supported` and `partial` require test evidence; `refused` requires an explicit compiler diagnostic or native throwing stub; `not-implemented` is the actionable backlog for an owned API family with no implementation match; `by-design` records an architectural exclusion; `unreviewed` still needs classification; and `not-applicable` is not a runtime API for that execution tier.
Run `pnpm node-compat:backlog` from the repository root for a summary. The filters `--tier`, `--action`, `--priority`, `--chapter`, and `--status` compose; use `--format=json` or `--format=tsv` for tooling.
Run `pnpm node-compat` from the repository root after changing the Node pin or the support manifest. `pnpm node-compat:check` is the offline drift/evidence gate; `pnpm --filter @internal/compatibility check:upstream` additionally verifies the pinned upstream inputs over the network.
Run `pnpm node-compat` from the repository root after changing the Node pin or the support manifest. `pnpm node-compat --local` does the same without network access while the pin is unchanged, by reclassifying the committed ledger rows; after a merge, `node scripts/regen-generated.mjs` regenerates every generated file instead of a hand merge. `pnpm node-compat:check` is the offline drift/evidence gate; `pnpm --filter @internal/compatibility check:upstream` additionally verifies the pinned upstream inputs over the network.
## npm packages that compile as builtins
The ledger counts Node APIs, so npm packages that compile as a Node builtin have no rows. `packages/compiler/src/frontend/builtin-package-aliases.ts` lists them, each with the checked major versions and the reason its module is that builtin on Node 24. One entry is a documented divergence rather than an identity: `readable-stream` 2.x, 3.x and 4.x compile as `node:stream`; for 2.x and 4.x that is the package's `READABLE_STREAM=disable` behavior. Without that variable Node runs the package's own JavaScript streams. The `tests/corpus/readable-stream-disable` program pins the disabled behavior. 3.x, a copy of Node 10's streams, cannot run disabled on Node 24 (its disable branch throws), so `tests/corpus/readable-stream-3` pins only behavior its own streams share with the builtin.
internal/compatibility/generated/node-v24-backlog.json
-88685
View File
File diff suppressed because it is too large Load Diff
internal/compatibility/generated/node-v24-internal.json
+180 -652
View File
File diff suppressed because it is too large Load Diff
internal/compatibility/src/backlog.mjs
+4 -1
View File
@@ -2,9 +2,12 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { buildBacklog } from "./generate.mjs";
const packageRoot = fileURLToPath(new URL("../", import.meta.url));
const backlog = JSON.parse(readFileSync(`${packageRoot}generated/node-v24-backlog.json`, "utf8"));
const backlog = buildBacklog(
JSON.parse(readFileSync(`${packageRoot}generated/node-v24-internal.json`, "utf8")),
);
const args = process.argv.slice(2);
function option(name) {
internal/compatibility/src/generate.mjs
+62 -20
View File
@@ -13,17 +13,26 @@
*
* pnpm node-compat fetch latest-v24.x and rewrite the snapshot
* pnpm node-compat --check fetch and fail if the snapshot would change
* pnpm node-compat --local rewrite the snapshot without fetching: reclassify
* the committed rows from the current sources
* pnpm node-compat:check offline implementation/profile drift check
*/
import { createHash } from "node:crypto";
import { readFileSync, writeFileSync } from "node:fs";
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
const packageRoot = fileURLToPath(new URL("../", import.meta.url));
const root = fileURLToPath(new URL("../../../", import.meta.url));
const internalOutputPath = `${packageRoot}generated/node-v24-internal.json`;
const backlogOutputPath = `${packageRoot}generated/node-v24-backlog.json`;
const publicOutputPath = `${root}docs/src/generated/node-v24-compatibility.json`;
// Files derived from the ledger elsewhere: the docs build computes the
// metadata from the public artifact, and `pnpm node-compat:backlog` builds
// the backlog in memory. A merge that keeps a branch's copy would bring
// back a file every compatibility change rewrites.
const retiredOutputPaths = [
`${root}docs/src/generated/node-v24-compatibility-meta.json`,
`${packageRoot}generated/node-v24-backlog.json`,
];
const compilerManifestPath = `${root}packages/compiler/surface-manifest.json`;
const compilerCompatPath = `${packageRoot}static-support.json`;
const pinPath = `${packageRoot}node-v24.json`;
@@ -443,12 +452,6 @@ function summarizeTier(rows, key, chapterPolicy) {
return tier("unreviewed", "", { source: "derived:descendants" });
}
function statusCounts(rows, key) {
const counts = Object.fromEntries(STATUSES.map((status) => [status, 0]));
for (const row of rows) if (row.scope === "api") counts[row[key].status] += 1;
return counts;
}
function verifiedAnchorIds(html) {
return new Set(
[...html.matchAll(/<a class="mark" href="#[^"]+" id="([^"]+)">#/g)].map((match) => match[1]),
@@ -520,7 +523,6 @@ function buildSnapshot({ allJson, indexMarkdown, allSha256, indexSha256 }) {
title: link.title,
entries: chapterRows.length,
apiEntries: chapterRows.filter((row) => row.scope === "api").length,
static: statusCounts(chapterRows, "static"),
});
rows.push(...chapterRows);
}
@@ -585,6 +587,12 @@ function publicDetail(tier) {
if (/^compiler-dedicated:assert\.doesNot(?:Throw|Reject)$/.test(source)) {
return "Supports callbacks or native promises with messages, Error classes, regular expressions, and validators accepting unknown values; object expectations and complete AssertionError metadata remain unsupported.";
}
if (/^surface-manifest:node-builtin\.crypto\.createECDH$/.test(source)) {
return "Supports prime192v1, secp224r1, prime256v1, secp384r1, secp521r1, secp192k1, secp224k1, secp256k1, brainpoolP256r1, brainpoolP384r1 and brainpoolP512r1 with Node's curve, key, point, format and encoding errors; other curves Node knows are refused.";
}
if (/^compiler-dedicated:(?:ECDH|ecdh\.)/.test(source)) {
return "Matches Node's key formats, point encodings, private-key range checks, key-pair checks and errors on the supported curves; ECDH.convertKey is supported, and property reads and writes are refused.";
}
if (/^compiler-dedicated:(?:Cipheriv|Decipheriv|cipher\.|decipher\.)/.test(source)) {
return "Matches Node's update() output lengths, padding, authentication tags, string encodings, and state errors for the supported ciphers; the stream.Transform interface is refused.";
}
@@ -863,7 +871,7 @@ function backlogAction(tier) {
return null;
}
function buildBacklog(snapshot) {
export function buildBacklog(snapshot) {
const tasks = [];
const tierItems = { static: {} };
const priorities = { high: 0, normal: 0, low: 0 };
@@ -959,7 +967,6 @@ function projectPublicMatrix(snapshot) {
title: chapter.title,
entries: chapterRows.length,
apiEntries: chapterRows.length,
static: statusCounts(chapterRows, "static"),
});
rows.push(...chapterRows);
}
@@ -1008,7 +1015,39 @@ function publicSnapshot(snapshot) {
};
}
/** The committed ledger with every row's classification recomputed from
* the current sources. The rows themselves come from the pinned Node docs,
* so while the pin is unchanged this is what a fetching run writes. */
function reclassify(snapshot) {
if (snapshot.nodeVersion !== pin.version || snapshot.nodeCommit !== pin.commit) {
throw new Error(
`the ledger targets Node ${snapshot.nodeVersion}, the pin ${pin.version}; run 'pnpm node-compat' to fetch the new docs`,
);
}
const ctx = classificationContext();
const ancestors = [];
for (const row of snapshot.rows) {
ancestors.length = row.depth;
const parentSignature = ancestors[row.depth - 1]?.signature;
row.static = classifyStatic(row, null, ctx, parentSignature);
ancestors[row.depth] = row;
}
for (const chapter of snapshot.chapters) {
const chapterRows = snapshot.rows.filter((row) => row.chapter === chapter.slug);
chapterRows[0].static = summarizeTier(
chapterRows,
"static",
ctx.compilerCompat.chapterPolicies[chapter.slug],
);
}
return snapshot;
}
function checkLocal() {
for (const path of retiredOutputPaths) {
if (existsSync(path))
throw new Error(`${path.slice(root.length)} is no longer generated; delete it`);
}
const snapshot = JSON.parse(readFileSync(internalOutputPath, "utf8"));
if (snapshot.nodeVersion !== pin.version || snapshot.nodeCommit !== pin.commit) {
throw new Error(
@@ -1056,9 +1095,6 @@ function checkLocal() {
throw new Error(`${row.id} exposes a row that is not applicable to native compilation`);
}
}
if (readFileSync(backlogOutputPath, "utf8") !== render(buildBacklog(snapshot))) {
throw new Error("the Node compatibility backlog is stale; run 'pnpm node-compat'");
}
if (readFileSync(publicOutputPath, "utf8") !== render(publicSnapshot(snapshot))) {
throw new Error("the docs compatibility artifact is stale; run 'pnpm node-compat'");
}
@@ -1103,6 +1139,15 @@ async function fetchText(url) {
async function main() {
const args = new Set(process.argv.slice(2));
if (args.has("--check-local")) return checkLocal();
if (args.has("--local")) {
const snapshot = reclassify(JSON.parse(readFileSync(internalOutputPath, "utf8")));
writeFileSync(internalOutputPath, render(snapshot));
writeFileSync(publicOutputPath, render(publicSnapshot(snapshot)));
console.log(
`reclassified the compatibility ledger and public artifact (${snapshot.rows.length} source rows, ${projectPublicMatrix(snapshot).rows.length} public matrix rows, ${pin.tag})`,
);
return;
}
const [allText, indexMarkdown] = await Promise.all([
fetchText(pin.allJsonUrl),
@@ -1124,14 +1169,11 @@ async function main() {
);
verifyAnchors(snapshot, htmlByChapter);
const internalRendered = render(snapshot);
const backlogRendered = render(buildBacklog(snapshot));
const publicRendered = render(publicSnapshot(snapshot));
if (args.has("--check")) {
if (readFileSync(internalOutputPath, "utf8") !== internalRendered)
throw new Error("internal Node compatibility snapshot is stale; run 'pnpm node-compat'");
if (readFileSync(backlogOutputPath, "utf8") !== backlogRendered)
throw new Error("Node compatibility backlog is stale; run 'pnpm node-compat'");
if (readFileSync(publicOutputPath, "utf8") !== publicRendered)
throw new Error("public Node compatibility snapshot is stale; run 'pnpm node-compat'");
console.log(
@@ -1140,11 +1182,11 @@ async function main() {
return;
}
writeFileSync(internalOutputPath, internalRendered);
writeFileSync(backlogOutputPath, backlogRendered);
writeFileSync(publicOutputPath, publicRendered);
console.log(
`wrote compatibility ledger, backlog, and public artifact (${snapshot.rows.length} source rows, ${projectPublicMatrix(snapshot).rows.length} public matrix rows, ${pin.tag})`,
`wrote compatibility ledger and public artifact (${snapshot.rows.length} source rows, ${projectPublicMatrix(snapshot).rows.length} public matrix rows, ${pin.tag})`,
);
}
await main();
// backlog.mjs imports buildBacklog; only running this file generates.
if (process.argv[1] === fileURLToPath(import.meta.url)) await main();
internal/compatibility/src/update.mjs
+1 -2
View File
@@ -2,8 +2,7 @@
import { spawnSync } from "node:child_process";
const check = process.argv.slice(2).includes("--check");
const args = check ? ["--check"] : [];
const args = process.argv.slice(2).filter((arg) => arg === "--check" || arg === "--local");
for (const script of ["generate.mjs"]) {
const result = spawnSync(process.execPath, [`src/${script}`, ...args], { stdio: "inherit" });
internal/compatibility/static-support.json
+3
View File
@@ -44,6 +44,7 @@
"node-builtin.console": ["tests/corpus/console-imports.cjs", "tests/corpus/2440-console-inspect-args.ts", "tests/corpus/console-native-value.js", "tests/harness/npm-static.test.ts"],
"node-builtin.crypto.createCipheriv": ["tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/fixtures/node-types/cipher.ts"],
"node-builtin.crypto.createDecipheriv": ["tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/fixtures/node-types/cipher.ts"],
"node-builtin.crypto.createECDH": ["tests/corpus/6700-ecdh-shared-secret.ts", "tests/corpus/6701-ecdh-formats.ts", "tests/corpus/6702-ecdh-errors.js", "tests/corpus/6703-ecdh-private-keys.ts", "tests/corpus/6704-ecdh-set-public-key.cjs", "tests/corpus/6705-ecdh-handles.ts", "tests/fixtures/node-types/ecdh.ts"],
"node-builtin.crypto": ["tests/corpus/1358-crypto-random.ts", "tests/corpus/1534-crypto-hash-chain.ts", "tests/corpus/2556-crypto-introspection.ts", "tests/corpus/2890-crypto-static-utility.ts", "tests/harness/npm-static.test.ts", "tests/corpus/node24-crypto-sha2.ts", "tests/corpus/node24-crypto-kdf.ts", "tests/corpus/node24-crypto-kdf-errors.ts", "tests/corpus/node24-crypto-kdf-options.ts"],
"node-builtin.dgram": ["tests/corpus/2597-dgram-send-ladders.cjs"],
"node-builtin.diagnostics_channel": ["tests/corpus/2214-dc-bind-store.cjs"],
@@ -410,6 +411,8 @@
{ "symbols": ["assert.deepEqual", "assert.notDeepEqual"], "status": "partial", "evidence": ["tests/corpus/5513-assert-deep-equal-loose.js", "tests/corpus/5514-assert-deep-equal-typed.ts", "tests/corpus/5516-assert-deep-equal-edges.js"] },
{ "symbols": ["assert", "assert.ok", "assert.equal", "assert.notEqual", "assert.strictEqual", "assert.notStrictEqual", "assert.deepStrictEqual", "assert.notDeepStrictEqual", "assert.fail", "assert.match", "assert.doesNotMatch", "assert.throws", "assert.rejects", "assert.ifError"], "status": "partial", "evidence": ["tests/corpus/1600-assert-passing.ts", "tests/corpus/1604-assert-deep-structures.ts", "tests/corpus/1609-assert-async.ts", "tests/corpus/1721-assert-throws-regex-class.ts", "tests/corpus/2936-static-loose-equality.ts", "tests/corpus/5515-assert-deep-strict-typed-messages.ts"] },
{ "chapter": "crypto", "symbols": ["Cipheriv", "cipher.update", "cipher.final", "cipher.setAAD", "cipher.getAuthTag", "cipher.setAutoPadding", "Decipheriv", "decipher.update", "decipher.final", "decipher.setAAD", "decipher.setAuthTag", "decipher.setAutoPadding"], "status": "partial", "evidence": ["tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/fixtures/node-types/cipher.ts"] },
{ "chapter": "crypto", "symbols": ["ECDH", "ecdh.computeSecret", "ecdh.generateKeys", "ecdh.getPrivateKey", "ecdh.getPublicKey", "ecdh.setPrivateKey", "ecdh.setPublicKey"], "status": "partial", "evidence": ["tests/corpus/6700-ecdh-shared-secret.ts", "tests/corpus/6701-ecdh-formats.ts", "tests/corpus/6702-ecdh-errors.js", "tests/corpus/6703-ecdh-private-keys.ts", "tests/corpus/6704-ecdh-set-public-key.cjs", "tests/corpus/6705-ecdh-handles.ts", "tests/fixtures/node-types/ecdh.ts"] },
{ "chapter": "errors", "symbols": ["ERR_CRYPTO_ECDH_INVALID_FORMAT", "ERR_CRYPTO_ECDH_INVALID_PUBLIC_KEY"], "status": "partial", "evidence": ["tests/corpus/6702-ecdh-errors.js"] },
{ "symbols": ["crypto.createHash", "Hash", "hash.update", "hash.digest", "hash.copy", "crypto.createHmac", "Hmac", "hmac.update", "hmac.digest", "crypto.hash", "crypto.pbkdf2", "crypto.pbkdf2Sync"], "status": "partial", "evidence": ["tests/corpus/5760-crypto-hash-stream.ts", "tests/corpus/5768-crypto-hash-unknown.ts", "tests/corpus/5769-crypto-hash-unknown.cjs", "tests/corpus/5761-crypto-hash-stream-states.ts", "tests/corpus/5762-crypto-hash-stream.cjs", "tests/corpus/2890-crypto-static-utility.ts", "tests/corpus/node24-crypto-sha2.ts", "tests/corpus/5435-crypto-hash-algorithms.ts", "tests/corpus/5436-crypto-hash-names.js", "tests/corpus/5437-crypto-hmac-algorithms.ts", "tests/corpus/5438-crypto-kdf-digests.ts", "tests/corpus/5440-crypto-hash-encodings.js", "tests/corpus/5441-crypto-hash-output-length.ts", "tests/harness/npm-static.test.ts"] },
{ "symbols": ["crypto.randomFill", "crypto.randomFillSync", "crypto.randomUUID", "crypto.randomUUIDv7"], "status": "supported", "evidence": ["tests/corpus/5442-crypto-random-values.ts", "tests/corpus/5443-crypto-random-arguments.js", "tests/corpus/1358-crypto-random.ts"] },
{ "symbols": ["crypto.createSecretKey", "crypto.generateKey", "crypto.generateKeySync", "KeyObject", "keyObject.export", "keyObject.equals", "keyObject.symmetricKeySize", "keyObject.type"], "status": "partial", "evidence": ["tests/corpus/5445-crypto-secret-key-objects.ts", "tests/corpus/5446-crypto-secret-key-errors.js"] },
package.json
+1 -1
View File
@@ -23,7 +23,7 @@
"test:sandbox": "node scripts/sandbox-test.mjs",
"test:sandbox:image": "node scripts/sandbox-image.mjs",
"test:watch": "vitest",
"lint": "oxlint --deny-warnings packages scripts internal/compatibility/src tests/harness vitest.config.ts",
"lint": "node docs/scripts/limitations-partials.mjs check && oxlint --deny-warnings packages scripts internal/compatibility/src tests/harness vitest.config.ts",
"lint:fix": "pnpm lint --fix",
"format": "oxfmt --write \"{packages,scripts,internal/compatibility/src,tests/harness}/**/*.{js,mjs,cjs,ts,mts,cts,tsx}\" vitest.config.ts .oxlintrc.jsonc .oxfmtrc.jsonc",
"format:check": "oxfmt --check \"{packages,scripts,internal/compatibility/src,tests/harness}/**/*.{js,mjs,cjs,ts,mts,cts,tsx}\" vitest.config.ts .oxlintrc.jsonc .oxfmtrc.jsonc",
packages/cli/test/native-link-info.test.ts
+2 -2
View File
@@ -72,7 +72,7 @@ describe.runIf(supported)("macOS arm64 native link info", () => {
minimum_os: "14.0",
},
program: { object, entry_symbol: "main" },
runtime_abi: { version: 22, marker: "scr_runtime_abi_v22" },
runtime_abi: { version: 23, marker: "scr_runtime_abi_v23" },
runtime_pack: {
kind: "precompiled",
package: "@scriptc/runtime-darwin-arm64",
@@ -165,6 +165,6 @@ describe.runIf(supported)("macOS arm64 native link info", () => {
() => null,
(failure: { stderr?: string }) => failure,
);
expect(error?.stderr).toContain("scr_runtime_abi_v22");
expect(error?.stderr).toContain("scr_runtime_abi_v23");
});
});
packages/compiler/ambient/scriptc-node-fallback.d.ts
Vendored
+35 -5
View File
@@ -2933,6 +2933,34 @@ declare module "crypto" {
iv: BinaryLike | null,
options?: { authTagLength?: number },
): Decipheriv;
export type ECDHKeyFormat = "compressed" | "uncompressed" | "hybrid";
export class ECDH {
private constructor();
static convertKey(
key: BinaryLike,
curve: string,
inputEncoding?: BinaryToTextEncoding,
outputEncoding?: "latin1" | "hex" | "base64" | "base64url",
format?: ECDHKeyFormat,
): Buffer | string;
generateKeys(): Buffer;
generateKeys(encoding: BinaryToTextEncoding, format?: ECDHKeyFormat): string;
computeSecret(otherPublicKey: ArrayBufferView): Buffer;
computeSecret(otherPublicKey: string, inputEncoding: BinaryToTextEncoding): Buffer;
computeSecret(otherPublicKey: ArrayBufferView, outputEncoding: BinaryToTextEncoding): string;
computeSecret(
otherPublicKey: string,
inputEncoding: BinaryToTextEncoding,
outputEncoding: BinaryToTextEncoding,
): string;
getPrivateKey(): Buffer;
getPrivateKey(encoding: BinaryToTextEncoding): string;
getPublicKey(encoding?: null, format?: ECDHKeyFormat): Buffer;
getPublicKey(encoding: BinaryToTextEncoding, format?: ECDHKeyFormat): string;
setPrivateKey(privateKey: ArrayBufferView): void;
setPrivateKey(privateKey: string, encoding: BinaryToTextEncoding): void;
}
export function createECDH(curveName: string): ECDH;
export interface ScryptOptions {
N?: number;
r?: number;
@@ -4940,15 +4968,17 @@ declare module "net" {
host?: string,
connectListener?: () => void,
): Socket;
/* The caller-resolver dial (@types/node's net.LookupFunction, narrowed
* to the lowered shape): the runtime invokes it as Node does and dials
* the answered addresses in order under autoSelectFamily: true. */
/* The caller-resolver dial (@types/node's net.LookupFunction): the
* runtime invokes it as Node's autoSelectFamily dial does (options
* { hints, all: true }, so the callback answers an address list) and
* dials the answered addresses in order. */
export type LookupFunction = (
hostname: string,
options: unknown,
options: import("node:dns").LookupOptions,
callback: (
err: NodeJS.ErrnoException | null,
addresses: { address: string; family: number }[],
address: string | import("node:dns").LookupAddress[],
family?: number,
) => void,
) => void;
export function connect(
packages/compiler/package.json
+1
View File
@@ -29,6 +29,7 @@
"files": [
"dist",
"ambient",
"shims",
"surface-manifest.json"
],
"publishConfig": {
packages/compiler/shims/fs-extra/core/api.ts
+1192
View File
File diff suppressed because it is too large Load Diff
packages/compiler/shims/fs-extra/index.ts
+9
View File
@@ -0,0 +1,9 @@
// scriptc's built-in implementation of the `fs-extra` package, 8.x-11.x API.
//
// Programs whose installed fs-extra is 8.x through 11.x compile this module
// in its place (see packages/compiler/src/frontend/package-shims.ts), and its
// declarations stand in for @types/fs-extra. The default import, the
// namespace import and named imports all read the same exports, as they read
// fs-extra's single CommonJS export object under Node.
export * from "./core/api.ts";
export * as default from "./core/api.ts";
packages/compiler/shims/undici/core/agent.ts
+163
View File
@@ -0,0 +1,163 @@
// undici's Agent (lib/dispatcher/agent.js): one dispatcher per origin,
// created by `factory` (default: a Pool, or a Client when connections is 1).
import { Client } from "./client.ts";
import { Dispatcher, DispatcherBase } from "./dispatcher.ts";
import { InvalidArgumentError } from "./errors.ts";
import { Pool } from "./pool.ts";
import type {
AgentOptions,
DispatchHandlers,
DispatchOptions,
PoolOptions,
Profile,
} from "./types.ts";
interface AgentEntry {
dispatcher: Dispatcher;
count: number;
}
export class Agent extends DispatcherBase {
__options: PoolOptions;
__factory: ((origin: string | URL, opts: object) => Dispatcher) | null;
__clients: Map<string, AgentEntry>;
__onDrain: (origin: URL, targets: Dispatcher[]) => void;
__onConnect: (origin: URL, targets: Dispatcher[]) => void;
__onDisconnect: (origin: URL, targets: Dispatcher[], err: Error) => void;
__onConnectionError: (origin: URL, targets: Dispatcher[], err: Error) => void;
constructor(options: AgentOptions | undefined, profile: Profile) {
super(profile);
const opts: AgentOptions = options ?? {};
const interceptors = opts.interceptors;
if (profile.major < 7 && interceptors !== undefined && interceptors.Agent !== undefined) {
this.__interceptors = interceptors.Agent;
}
this.__options = {
keepAliveTimeout: opts.keepAliveTimeout,
keepAliveMaxTimeout: opts.keepAliveMaxTimeout,
keepAliveTimeoutThreshold: opts.keepAliveTimeoutThreshold,
pipelining: opts.pipelining,
maxHeaderSize: opts.maxHeaderSize,
bodyTimeout: opts.bodyTimeout,
headersTimeout: opts.headersTimeout,
strictContentLength: opts.strictContentLength,
maxRequestsPerClient: opts.maxRequestsPerClient,
maxResponseSize: opts.maxResponseSize,
connectTimeout: opts.connectTimeout,
localAddress: opts.localAddress,
connections: opts.connections,
connect: opts.connect,
interceptors:
interceptors !== undefined
? { Pool: interceptors.Pool, Client: interceptors.Client }
: undefined,
};
const factory = opts.factory;
this.__factory = factory !== undefined ? factory : null;
this.__clients = new Map<string, AgentEntry>();
this.__onDrain = (origin: URL, targets: Dispatcher[]): void => {
this.emit("drain", origin, [this as Dispatcher, ...targets]);
};
this.__onConnect = (origin: URL, targets: Dispatcher[]): void => {
this.emit("connect", origin, [this as Dispatcher, ...targets]);
};
this.__onDisconnect = (origin: URL, targets: Dispatcher[], err: Error): void => {
this.emit("disconnect", origin, [this as Dispatcher, ...targets], err);
};
this.__onConnectionError = (origin: URL, targets: Dispatcher[], err: Error): void => {
this.emit("connectionError", origin, [this as Dispatcher, ...targets], err);
};
}
__runningCount(): number {
let ret = 0;
for (const entry of this.__clients.values()) ret += entry.dispatcher.__runningCount();
return ret;
}
__dispatch(opts: DispatchOptions, handler: DispatchHandlers): boolean {
const origin = opts.origin;
if (origin === undefined || origin === "") {
throw new InvalidArgumentError("opts.origin must be a non-empty string or URL.");
}
const key = typeof origin === "string" ? origin : origin.href;
const existing = this.__clients.get(key);
let dispatcher: Dispatcher;
if (existing !== undefined) {
dispatcher = existing.dispatcher;
} else {
const factory = this.__factory;
dispatcher =
factory !== null
? factory(origin, this.__options)
: this.__defaultFactory(origin, this.__options);
const entry: AgentEntry = { dispatcher, count: 0 };
if (this.__profile.major >= 7) {
// 7.x closes and forgets an origin's dispatcher once all of its
// connections are gone.
const closeIfUnused = (connected: boolean): void => {
const current = this.__clients.get(key);
if (current !== undefined) {
if (connected) current.count -= 1;
if (current.count <= 0) {
this.__clients.delete(key);
if (!current.dispatcher.__isDestroyed()) current.dispatcher.close();
}
}
};
dispatcher.on("drain", this.__onDrain);
dispatcher.on("connect", (o: URL, targets: Dispatcher[]) => {
const current = this.__clients.get(key);
if (current !== undefined) current.count += 1;
this.__onConnect(o, targets);
});
dispatcher.on("disconnect", (o: URL, targets: Dispatcher[], err: Error) => {
closeIfUnused(true);
this.__onDisconnect(o, targets, err);
});
dispatcher.on("connectionError", (o: URL, targets: Dispatcher[], err: Error) => {
closeIfUnused(false);
this.__onConnectionError(o, targets, err);
});
} else {
dispatcher.on("drain", this.__onDrain);
dispatcher.on("connect", this.__onConnect);
dispatcher.on("disconnect", this.__onDisconnect);
dispatcher.on("connectionError", this.__onConnectionError);
}
this.__clients.set(key, entry);
}
return dispatcher.dispatch(opts, handler);
}
/** The default factory: a Client when `connections` is 1, else a Pool, of
* the same undici version. */
__defaultFactory(origin: string | URL, opts: PoolOptions): Dispatcher {
return opts.connections === 1
? this.__createClient(origin, opts)
: this.__createPool(origin, opts);
}
__createClient(origin: string | URL, opts: PoolOptions): Dispatcher {
return new Client(origin, opts, this.__profile);
}
__createPool(origin: string | URL, opts: PoolOptions): Dispatcher {
return new Pool(origin, opts, this.__profile);
}
__close(): Promise<void> {
const closing: Promise<void>[] = [];
for (const entry of this.__clients.values()) closing.push(entry.dispatcher.close());
if (this.__profile.major >= 7) this.__clients.clear();
return Promise.all(closing).then(() => {});
}
__destroy(err: Error): Promise<void> {
const destroying: Promise<void>[] = [];
for (const entry of this.__clients.values()) destroying.push(entry.dispatcher.destroy(err));
if (this.__profile.major >= 7) this.__clients.clear();
return Promise.all(destroying).then(() => {});
}
}
packages/compiler/shims/undici/core/api.ts
+93
View File
@@ -0,0 +1,93 @@
// undici's top-level request() (index.js makeDispatcher over
// lib/api/api-request.js) and the global dispatcher
// (lib/global.js: one process-wide slot shared by every undici copy).
import { requestThrough } from "./dispatcher.ts";
import type { Dispatcher, RequestOptions, ResponseData } from "./dispatcher.ts";
import { parseOrigin, parseURL } from "./util.ts";
import type { IncomingHttpHeaders, Profile, RequestBody, UndiciHeaders } from "./types.ts";
/** request(url, options)'s options: Dispatcher.RequestOptions without
* origin/path, an optional method, and the dispatcher to use. */
export interface TopLevelRequestOptions {
dispatcher?: Dispatcher;
method?: string;
path?: string;
body?: RequestBody;
headers?: UndiciHeaders;
idempotent?: boolean;
blocking?: boolean;
headersTimeout?: number | null;
bodyTimeout?: number | null;
reset?: boolean;
throwOnError?: boolean;
opaque?: unknown;
signal?: AbortSignal | null;
onInfo?: (info: { statusCode: number; headers: IncomingHttpHeaders }) => void;
highWaterMark?: number;
}
let globalDispatcher: Dispatcher | null = null;
let globalDispatcherSet = false;
export function setGlobalDispatcher(dispatcher: Dispatcher): void {
globalDispatcher = dispatcher;
globalDispatcherSet = true;
}
/** The global dispatcher, created on first use by the asking version's
* Agent (undici stores whichever copy asked first). */
export function getGlobalDispatcher(create: () => Dispatcher): Dispatcher {
let dispatcher = globalDispatcher;
if (dispatcher === null) {
dispatcher = create();
globalDispatcher = dispatcher;
}
return dispatcher;
}
/** True once a program installed its own global dispatcher. */
export function hasCustomGlobalDispatcher(): boolean {
return globalDispatcherSet;
}
export function request(
url: string | URL,
options: TopLevelRequestOptions | undefined,
createGlobal: () => Dispatcher,
profile: Profile,
): Promise<ResponseData> {
// makeDispatcher validates synchronously: an invalid URL throws at the
// call rather than rejecting.
{
const opts: TopLevelRequestOptions = options ?? {};
let target: URL;
const path = opts.path;
if (path !== undefined) {
const prefixed = path.startsWith("/") ? path : `/${path}`;
target = new URL(parseOrigin(url, profile).origin + prefixed);
} else {
target = parseURL(url, profile);
}
const dispatcher = opts.dispatcher ?? getGlobalDispatcher(createGlobal);
const method =
opts.method !== undefined && opts.method !== "" ? opts.method : opts.body ? "PUT" : "GET";
const dispatchOptions: RequestOptions = {
origin: target.origin,
path: target.search ? `${target.pathname}${target.search}` : target.pathname,
method,
body: opts.body,
headers: opts.headers,
idempotent: opts.idempotent,
blocking: opts.blocking,
headersTimeout: opts.headersTimeout,
bodyTimeout: opts.bodyTimeout,
reset: opts.reset,
throwOnError: opts.throwOnError,
opaque: opts.opaque,
signal: opts.signal,
onInfo: opts.onInfo,
highWaterMark: opts.highWaterMark,
};
return requestThrough(dispatcher, dispatchOptions, profile);
}
}
packages/compiler/shims/undici/core/client.ts
+1304
View File
File diff suppressed because it is too large Load Diff
packages/compiler/shims/undici/core/connector.ts
+157
View File
@@ -0,0 +1,157 @@
// undici's buildConnector (lib/core/connect.js) over node:net and node:tls.
import * as net from "node:net";
import * as tls from "node:tls";
import { ConnectTimeoutError, InvalidArgumentError, NotSupportedError } from "./errors.ts";
import type {
Connector,
ConnectorBuildOptions,
ConnectorCallback,
ConnectorOptions,
LookupAddress,
} from "./types.ts";
export function buildConnector(options: ConnectorBuildOptions): Connector {
const maxCachedSessions = options.maxCachedSessions;
if (
maxCachedSessions !== undefined &&
maxCachedSessions !== null &&
(!Number.isInteger(maxCachedSessions) || maxCachedSessions < 0)
) {
throw new InvalidArgumentError("maxCachedSessions must be a positive integer or zero");
}
if (options.socketPath !== undefined && options.socketPath !== null) {
throw new NotSupportedError("scriptc's undici connector does not dial socketPath yet");
}
const timeout =
options.timeout === undefined || options.timeout === null ? 10e3 : options.timeout;
const lookup = options.lookup;
// The plain-TCP dial, with the caller's resolver when one is configured.
let dialTcp: (port: number, host: string) => net.Socket;
if (lookup !== undefined) {
// net.connect's lowered resolver answers every address at once; adapt a
// resolver that answers a single address string.
const userLookup: (
hostname: string,
options: unknown,
callback: (
err: NodeJS.ErrnoException | null,
address: string | LookupAddress[],
family?: number,
) => void,
) => void = lookup;
const resolver = (
hostname: string,
lookupOptions: unknown,
callback: (err: NodeJS.ErrnoException | null, addresses: LookupAddress[]) => void,
): void => {
userLookup(hostname, lookupOptions, (err, address, family) => {
if (err !== null) callback(err, []);
else if (typeof address === "string") {
callback(null, [{ address, family: family ?? (net.isIPv6(address) ? 6 : 4) }]);
} else callback(null, address);
});
};
dialTcp = (port: number, host: string): net.Socket =>
net.connect({ port, host, autoSelectFamily: true, lookup: resolver });
} else {
dialTcp = (port: number, host: string): net.Socket => net.connect({ port, host });
}
const rejectUnauthorized = options.rejectUnauthorized;
const ca = options.ca;
const defaultServername = options.servername;
return (params: ConnectorOptions, callback: ConnectorCallback): net.Socket => {
let pending: ConnectorCallback | null = callback;
const https = params.protocol === "https:";
const port = params.port !== "" ? Number(params.port) : https ? 443 : 80;
let socket: net.Socket;
if (https) {
if (lookup !== undefined) {
throw new NotSupportedError(
"scriptc's undici connector does not apply a custom lookup to TLS connections yet",
);
}
const servername =
params.servername !== undefined && params.servername !== ""
? params.servername
: defaultServername !== undefined
? defaultServername
: serverNameOf(params.host ?? params.hostname);
socket = tls.connect({
port,
host: params.hostname,
servername,
rejectUnauthorized: rejectUnauthorized ?? true,
...(ca !== undefined ? { ca } : {}),
});
} else {
socket = dialTcp(port, params.hostname);
}
let cancelTimer: (() => void) | null = null;
if (timeout !== 0) {
const timer = setTimeout(() => {
cancelTimer = null;
setImmediate(() => {
const err = new ConnectTimeoutError();
socket.destroy();
if (pending !== null) {
const cb = pending;
pending = null;
cb(err, null);
}
});
}, timeout);
cancelTimer = () => clearTimeout(timer);
}
const cancelTimeout = (): void => {
const cancel = cancelTimer;
cancelTimer = null;
if (cancel !== null) cancel();
};
socket.setNoDelay(true);
const onConnected = (): void => {
cancelTimeout();
if (pending !== null) {
const cb = pending;
pending = null;
cb(null, socket);
}
};
if (https) socket.once("secureConnect", onConnected);
else socket.once("connect", onConnected);
socket.on("error", (err: Error) => {
cancelTimeout();
if (pending !== null) {
const cb = pending;
pending = null;
cb(err, null);
}
});
return socket;
};
}
/** undici's util.getServerName: the host without its port, or null for an
* IP literal (no SNI). */
function serverNameOf(host: string): string | undefined {
if (host === "") return undefined;
let servername = host;
if (servername.startsWith("[")) {
const idx = servername.indexOf("]");
servername = idx === -1 ? servername : servername.slice(1, idx);
} else {
const idx = servername.indexOf(":");
if (idx !== -1) servername = servername.slice(0, idx);
}
return net.isIP(servername) !== 0 ? undefined : servername;
}
/** The `buildConnector.*` type namespace of undici's types/connector.d.ts. */
export declare namespace buildConnector {
type BuildOptions = ConnectorBuildOptions;
type Options = ConnectorOptions;
type Callback = ConnectorCallback;
type connector = Connector;
}
packages/compiler/shims/undici/core/diagnostics-types.ts
+68
View File
@@ -0,0 +1,68 @@
// undici's DiagnosticsChannel message types (types/diagnostics-channel.d.ts).
import type { Socket } from "node:net";
import type { Connector } from "./types.ts";
export declare namespace DiagnosticsChannel {
interface Request {
origin?: string | URL;
completed: boolean;
method?: string;
path: string;
headers: string | string[];
addHeader(key: string, value: string): Request;
}
interface Response {
statusCode: number;
statusText: string;
headers: Array<Buffer>;
}
type Error = unknown;
interface ConnectParams {
host: string;
hostname: string;
protocol: string;
port: string;
servername: string | null;
}
type Connector = import("./types.ts").Connector;
interface RequestCreateMessage {
request: Request;
}
interface RequestBodySentMessage {
request: Request;
}
interface RequestHeadersMessage {
request: Request;
response: Response;
}
interface RequestTrailersMessage {
request: Request;
trailers: Array<Buffer>;
}
interface RequestErrorMessage {
request: Request;
error: Error;
}
interface ClientSendHeadersMessage {
request: Request;
headers: string;
socket: Socket;
}
interface ClientBeforeConnectMessage {
connectParams: ConnectParams;
connector: Connector;
}
interface ClientConnectedMessage {
socket: Socket;
connectParams: ConnectParams;
connector: Connector;
}
interface ClientConnectErrorMessage {
error: Error;
socket: Socket;
connectParams: ConnectParams;
connector: Connector;
}
}
export type { Connector };
packages/compiler/shims/undici/core/diagnostics.ts
+16
View File
@@ -0,0 +1,16 @@
// undici's diagnostics_channel publications (lib/core/diagnostics.js). The
// channel names and message shapes are undici's; subscribers see the same
// request objects across one request's lifetime.
import { channel } from "node:diagnostics_channel";
export const channels = {
create: channel("undici:request:create"),
bodySent: channel("undici:request:bodySent"),
headers: channel("undici:request:headers"),
trailers: channel("undici:request:trailers"),
error: channel("undici:request:error"),
beforeConnect: channel("undici:client:beforeConnect"),
connected: channel("undici:client:connected"),
connectError: channel("undici:client:connectError"),
sendHeaders: channel("undici:client:sendHeaders"),
};
packages/compiler/shims/undici/core/dispatcher.ts
+574
View File
@@ -0,0 +1,574 @@
// undici's Dispatcher (lib/dispatcher/dispatcher.js), DispatcherBase
// (dispatcher-base.js) and the request() API's handler (lib/api/api-request.js).
import { EventEmitter } from "node:events";
import { BodyReadable } from "./readable.ts";
import {
ClientClosedError,
ClientDestroyedError,
InvalidArgumentError,
RequestAbortedError,
ResponseStatusCodeError,
} from "./errors.ts";
import { parseHeaders, rawHeaderStrings } from "./util.ts";
import type {
DispatchController,
DispatchFn,
DispatchHandlers,
DispatchInterceptor,
DispatchOptions,
IncomingHttpHeaders,
Profile,
} from "./types.ts";
/** Dispatcher.RequestOptions: the dispatch options (which carry the
* request members, see DispatchOptions). */
export type RequestOptions = DispatchOptions;
export interface ResponseData {
statusCode: number;
/** 7.x only. */
statusText?: string;
headers: IncomingHttpHeaders;
body: BodyReadable;
trailers: Record<string, string>;
opaque: unknown;
context: object | undefined;
}
const defaultProfile: Profile = { major: 7 };
export class Dispatcher extends EventEmitter {
// An explicit constructor: the implicit one forwards its arguments to
// EventEmitter's options-taking constructor, which stays a runtime fence.
constructor() {
super();
}
dispatch(_options: DispatchOptions, _handler: DispatchHandlers): boolean {
throw new Error("not implemented");
}
close(): Promise<void> {
return Promise.reject(new Error("not implemented"));
}
destroy(_err?: Error | null): Promise<void> {
return Promise.reject(new Error("not implemented"));
}
/** Dispatcher.prototype.request: dispatches through this dispatcher and
* resolves with the response once its headers arrive. */
request(options: RequestOptions): Promise<ResponseData> {
return requestThrough(this, options, this.__undiciProfile());
}
__undiciProfile(): Profile {
return defaultProfile;
}
// Pool statistics hooks (undici's kConnected/kPending/kRunning/kSize/
// kBusy); dispatchers that are not clients or pools count nothing.
__connectedFlag(): boolean {
return false;
}
__isDestroyed(): boolean {
return false;
}
__connectedCount(): number {
return 0;
}
__freeCount(): number {
return 0;
}
__pendingCount(): number {
return 0;
}
__queuedCount(): number {
return 0;
}
__runningCount(): number {
return 0;
}
__sizeCount(): number {
return 0;
}
}
export class DispatcherBase extends Dispatcher {
__profile: Profile;
__destroyed: boolean;
__closed: boolean;
__onDestroyed: (() => void)[] | null;
__onClosed: (() => void)[] | null;
__interceptors: readonly DispatchInterceptor[];
__interceptedDispatch: DispatchFn | null;
constructor(profile: Profile) {
super();
this.__profile = profile;
this.__destroyed = false;
this.__closed = false;
this.__onDestroyed = [];
this.__onClosed = [];
this.__interceptors = [];
this.__interceptedDispatch = null;
}
__undiciProfile(): Profile {
return this.__profile;
}
get destroyed(): boolean {
return this.__destroyed;
}
__isDestroyed(): boolean {
return this.__destroyed;
}
get closed(): boolean {
return this.__closed;
}
close(): Promise<void> {
return new Promise<void>((resolve, reject) => {
if (this.__destroyed) {
queueMicrotask(() => reject(new ClientDestroyedError()));
return;
}
if (this.__closed) {
const waiting = this.__onClosed;
if (waiting !== null) waiting.push(() => resolve());
else queueMicrotask(() => resolve());
return;
}
this.__closed = true;
this.__onClosed!.push(() => resolve());
this.__close()
.then(() => this.destroy())
.then(() => {
queueMicrotask(() => {
const callbacks = this.__onClosed;
this.__onClosed = null;
if (callbacks !== null) for (const cb of callbacks) cb();
});
});
});
}
destroy(err?: Error | null): Promise<void> {
return new Promise<void>((resolve) => {
if (this.__destroyed) {
const waiting = this.__onDestroyed;
if (waiting !== null) waiting.push(() => resolve());
else queueMicrotask(() => resolve());
return;
}
const error = err ?? new ClientDestroyedError();
this.__destroyed = true;
this.__onDestroyed!.push(() => resolve());
this.__destroy(error).then(() => {
queueMicrotask(() => {
const callbacks = this.__onDestroyed;
this.__onDestroyed = null;
if (callbacks !== null) for (const cb of callbacks) cb();
});
});
});
}
dispatch(options: DispatchOptions, handler: DispatchHandlers): boolean {
const target = this.__profile.major >= 7 ? unwrapHandler(handler, this.__profile) : handler;
try {
if (this.__destroyed) throw new ClientDestroyedError();
if (this.__closed) throw new ClientClosedError();
return this.__dispatchIntercepted(options, target);
} catch (err) {
if (target.onError === undefined) {
if (this.__profile.major >= 7) throw err;
throw new InvalidArgumentError("invalid onError method");
}
target.onError!(err as Error);
return false;
}
}
__dispatchIntercepted(options: DispatchOptions, handler: DispatchHandlers): boolean {
let dispatch = this.__interceptedDispatch;
if (dispatch === null) {
dispatch = (o: DispatchOptions, h: DispatchHandlers): boolean => this.__dispatch(o, h);
for (let i = this.__interceptors.length - 1; i >= 0; i--) {
dispatch = this.__interceptors[i]!(dispatch);
}
this.__interceptedDispatch = dispatch;
}
return dispatch(options, handler);
}
__dispatch(_options: DispatchOptions, _handler: DispatchHandlers): boolean {
throw new Error("not implemented");
}
__close(): Promise<void> {
return Promise.resolve();
}
__destroy(_err: Error): Promise<void> {
return Promise.resolve();
}
}
/** The request() API over a dispatcher (api-request.js's request with the
* promise callback). */
export function requestThrough(
dispatcher: Dispatcher,
options: RequestOptions,
profile: Profile,
): Promise<ResponseData> {
return new Promise<ResponseData>((resolve, reject) => {
const callback = (err: Error | null, data: ResponseData | null): void => {
if (err !== null) reject(err);
else resolve(data!);
};
try {
const handler = createRequestHandler(options, profile, callback);
dispatcher.dispatch(options, handler);
} catch (err) {
queueMicrotask(() => callback(err as Error, null));
}
});
}
/** api-request.js's RequestHandler, as a handler record over closure state. */
function createRequestHandler(
opts: RequestOptions,
profile: Profile,
done: (err: Error | null, data: ResponseData | null) => void,
): DispatchHandlers {
if (opts.method === "CONNECT") throw new InvalidArgumentError("invalid method");
if (
opts.highWaterMark !== undefined &&
(!Number.isFinite(opts.highWaterMark) || opts.highWaterMark < 0)
) {
throw new InvalidArgumentError("invalid highWaterMark");
}
const v7 = profile.major >= 7;
const opaque: unknown = opts.opaque ?? null;
const onInfo = opts.onInfo ?? null;
const throwOnError = opts.throwOnError === true;
const trailers: Record<string, string> = {};
let callback: ((err: Error | null, data: ResponseData | null) => void) | null = done;
let res: BodyReadable | null = null;
let abortFn: ((err?: Error) => void) | null = null;
let reason: Error | null = null;
let removeAbortListener: (() => void) | null = null;
const removeSignal = (): void => {
const remove = removeAbortListener;
removeAbortListener = null;
if (remove !== null) remove();
};
const onError = (err: Error): void => {
removeSignal();
const cb = callback;
if (cb !== null) {
callback = null;
queueMicrotask(() => cb(err, null));
}
const body = res;
if (body !== null) {
res = null;
queueMicrotask(() => {
body.on("error", () => {});
body.destroy(err);
});
}
};
// 5.x: abort() before the request reached a socket fails it at once.
const abortV5 = (): void => {
const abort = abortFn;
if (abort !== null) abort();
else onError(new RequestAbortedError());
};
const signal = opts.signal ?? null;
if (signal !== null) {
if (v7) {
if (signal.aborted) {
reason = abortReason(signal);
} else {
const listener = (): void => {
const why = abortReason(signal);
reason = why;
const body = res;
const abort = abortFn;
if (body !== null) {
body.on("error", () => {});
body.destroy(why);
} else if (abort !== null) {
abort(why);
}
};
signal.addEventListener("abort", listener);
removeAbortListener = () => signal.removeEventListener("abort", listener);
}
} else if (signal.aborted) {
abortV5();
} else {
const listener = (): void => abortV5();
signal.addEventListener("abort", listener);
removeAbortListener = () => signal.removeEventListener("abort", listener);
}
}
return {
onConnect: (abort: (err?: Error) => void): void => {
if (v7) {
if (reason !== null) {
abort(reason);
return;
}
} else if (callback === null) {
throw new RequestAbortedError();
}
abortFn = abort;
},
onHeaders: (
statusCode: number,
rawHeaders: Buffer[],
resume: () => void,
statusText: string,
): boolean => {
const headers = parseHeaders(rawHeaderStrings(rawHeaders, profile), profile);
if (statusCode < 200) {
if (onInfo !== null) onInfo({ statusCode, headers });
return true;
}
const contentTypeValue = headers["content-type"];
const contentType = typeof contentTypeValue === "string" ? contentTypeValue : "";
const abort = abortFn;
const body = new BodyReadable(
resume,
(err: Error) => {
if (abort !== null) {
if (v7) abort(err);
else abort();
}
},
contentType,
profile,
);
if (v7 && removeAbortListener !== null) body.on("close", () => removeSignal());
const cb = callback;
callback = null;
res = body;
if (cb !== null) {
if (!v7 && throwOnError && statusCode >= 400) {
resolveErrorBody(cb, body, contentType, statusCode, statusText, headers);
return true;
}
const data: ResponseData = v7
? { statusCode, statusText, headers, trailers, opaque, body, context: undefined }
: { statusCode, headers, trailers, opaque, body, context: undefined };
cb(null, data);
}
return true;
},
onData: (chunk: Buffer): boolean => {
const body = res;
return body !== null ? body.push(chunk) : true;
},
onComplete: (rawTrailers: string[] | null): void => {
if (!v7) removeSignal();
if (rawTrailers !== null) {
const parsed = parseHeaders(rawTrailers, profile);
for (const key of Object.keys(parsed)) {
const value = parsed[key];
if (typeof value === "string") trailers[key] = value;
else if (value !== undefined) trailers[key] = value.join(", ");
}
}
const body = res;
if (body !== null) body.push(null);
},
onError,
};
}
function abortReason(signal: AbortSignal): Error {
const reason: unknown = signal.reason;
return reason instanceof Error ? reason : new RequestAbortedError();
}
/** 5.x throwOnError: reject with ResponseStatusCodeError carrying the parsed
* JSON or text body. */
function resolveErrorBody(
callback: (err: Error | null, data: ResponseData | null) => void,
body: BodyReadable,
contentType: string,
statusCode: number,
statusMessage: string,
headers: IncomingHttpHeaders,
): void {
const message = `Response status code ${statusCode}${statusMessage ? `: ${statusMessage}` : ""}`;
const record: Record<string, string | string[]> = {};
for (const key of Object.keys(headers)) {
const value = headers[key];
if (value !== undefined) record[key] = value;
}
if (statusCode === 204 || contentType === "") {
body.dump();
process.nextTick(() =>
callback(new ResponseStatusCodeError(message, statusCode, record), null),
);
return;
}
const finish = (payload: unknown): void => {
process.nextTick(() =>
callback(new ResponseStatusCodeError(message, statusCode, record, payload), null),
);
};
const fallback = (): void => {
body.dump();
process.nextTick(() =>
callback(new ResponseStatusCodeError(message, statusCode, record), null),
);
};
if (contentType.startsWith("application/json")) {
body.json().then((payload: unknown) => finish(payload), fallback);
} else if (contentType.startsWith("text/")) {
body.text().then((payload: string) => finish(payload), fallback);
} else {
fallback();
}
}
/** 7.x's UnwrapHandler: a controller-API handler (onRequestStart, ...)
* driven through the legacy callbacks the clients invoke. */
function unwrapHandler(handler: DispatchHandlers, profile: Profile): DispatchHandlers {
if (handler.onRequestStart === undefined) return handler;
let controller: UnwrapController | null = null;
const controllerOf = (): UnwrapController => {
let c = controller;
if (c === null) {
c = new UnwrapController(() => {});
controller = c;
}
return c;
};
return {
onConnect: (abort: (err?: Error) => void): void => {
const c = new UnwrapController(abort);
controller = c;
if (handler.onRequestStart !== undefined) handler.onRequestStart!(c, undefined);
},
onHeaders: (
statusCode: number,
rawHeaders: Buffer[],
resume: () => void,
statusText: string,
): boolean => {
const c = controllerOf();
c.__resume = resume;
if (handler.onResponseStart !== undefined) {
handler.onResponseStart!(
c,
statusCode,
parseHeaders(rawHeaderStrings(rawHeaders, profile), profile),
statusText,
);
}
return !c.paused;
},
onData: (chunk: Buffer): boolean => {
const c = controllerOf();
if (handler.onResponseData !== undefined) handler.onResponseData!(c, chunk);
return !c.paused;
},
onComplete: (trailers: string[] | null): void => {
if (handler.onResponseEnd !== undefined) {
handler.onResponseEnd!(controllerOf(), parseHeaders(trailers ?? [], profile));
}
},
onError: (err: Error): void => {
if (handler.onResponseError === undefined)
throw new InvalidArgumentError("invalid onError method");
handler.onResponseError!(controllerOf(), err);
},
};
}
/** 7.x's UnwrapController. */
class UnwrapController implements DispatchController {
__paused: boolean;
__reason: Error | null;
__aborted: boolean;
__abort: (err?: Error) => void;
__resume: (() => void) | null;
constructor(abort: (err?: Error) => void) {
this.__paused = false;
this.__reason = null;
this.__aborted = false;
this.__abort = abort;
this.__resume = null;
}
pause(): void {
this.__paused = true;
}
resume(): void {
if (this.__paused) {
this.__paused = false;
const resume = this.__resume;
if (resume !== null) resume();
}
}
abort(reason: Error): void {
if (!this.__aborted) {
this.__aborted = true;
this.__reason = reason;
this.__abort(reason);
}
}
get aborted(): boolean {
return this.__aborted;
}
get reason(): Error | null {
return this.__reason;
}
get paused(): boolean {
return this.__paused;
}
}
/** The `Dispatcher.*` type namespace of undici's types/dispatcher.d.ts
* (5.x and 7.x spellings). */
export declare namespace Dispatcher {
type DispatchOptions = import("./types.ts").DispatchOptions;
type DispatchHandlers = import("./types.ts").DispatchHandlers;
type DispatchHandler = import("./types.ts").DispatchHandlers;
type DispatchController = import("./types.ts").DispatchController;
type DispatchInterceptor = import("./types.ts").DispatchInterceptor;
type Dispatch = import("./types.ts").DispatchFn;
type HttpMethod = string;
type UndiciHeaders = import("./types.ts").UndiciHeaders;
// The 7.x spellings take an opaque type parameter; the opaque value is
// `unknown` here.
type RequestOptions<_TOpaque = null> = import("./dispatcher.ts").RequestOptions;
type ResponseData<_TOpaque = null> = import("./dispatcher.ts").ResponseData;
}
packages/compiler/shims/undici/core/errors-v5.ts
+22
View File
@@ -0,0 +1,22 @@
// undici 5.x's `errors` export (lib/core/errors.js module.exports).
export {
HTTPParserError,
UndiciError,
HeadersTimeoutError,
HeadersOverflowError,
BodyTimeoutError,
RequestContentLengthMismatchError,
ConnectTimeoutError,
ResponseStatusCodeError,
InvalidArgumentError,
InvalidReturnValueError,
RequestAbortedError,
ClientDestroyedError,
ClientClosedError,
InformationalError,
SocketError,
NotSupportedError,
ResponseContentLengthMismatchError,
BalancedPoolMissingUpstreamError,
ResponseExceededMaxSizeError,
} from "./errors.ts";
packages/compiler/shims/undici/core/errors-v7.ts
+25
View File
@@ -0,0 +1,25 @@
// undici 7.x's `errors` export (the implemented classes of
// lib/core/errors.js module.exports).
export {
AbortError,
HTTPParserError,
UndiciError,
HeadersTimeoutError,
HeadersOverflowError,
BodyTimeoutError,
RequestContentLengthMismatchError,
ConnectTimeoutError,
InvalidArgumentError,
InvalidReturnValueError,
RequestAbortedError,
ClientDestroyedError,
ClientClosedError,
InformationalError,
SocketError,
NotSupportedError,
ResponseContentLengthMismatchError,
BalancedPoolMissingUpstreamError,
ResponseExceededMaxSizeError,
RequestRetryError,
ResponseError,
} from "./errors.ts";
packages/compiler/shims/undici/core/errors.ts
+255
View File
@@ -0,0 +1,255 @@
// undici's error classes (lib/core/errors.js), shared by the 5.x and 7.x
// entries. Names, messages and codes are undici's; the 7.x-only classes are
// exported by the 7.x entry alone.
export interface SocketInfo {
localAddress?: string;
localPort?: number;
remoteAddress?: string;
remotePort?: number;
remoteFamily?: string;
timeout?: number;
bytesWritten?: number;
bytesRead?: number;
}
export class UndiciError extends Error {
code: string;
constructor(message?: string) {
super(message);
this.name = "UndiciError";
this.code = "UND_ERR";
}
}
export class ConnectTimeoutError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "ConnectTimeoutError";
this.message = message || "Connect Timeout Error";
this.code = "UND_ERR_CONNECT_TIMEOUT";
}
}
export class HeadersTimeoutError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "HeadersTimeoutError";
this.message = message || "Headers Timeout Error";
this.code = "UND_ERR_HEADERS_TIMEOUT";
}
}
export class HeadersOverflowError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "HeadersOverflowError";
this.message = message || "Headers Overflow Error";
this.code = "UND_ERR_HEADERS_OVERFLOW";
}
}
export class BodyTimeoutError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "BodyTimeoutError";
this.message = message || "Body Timeout Error";
this.code = "UND_ERR_BODY_TIMEOUT";
}
}
export class InvalidArgumentError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "InvalidArgumentError";
this.message = message || "Invalid Argument Error";
this.code = "UND_ERR_INVALID_ARG";
}
}
export class InvalidReturnValueError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "InvalidReturnValueError";
this.message = message || "Invalid Return Value Error";
this.code = "UND_ERR_INVALID_RETURN_VALUE";
}
}
/** 7.x's base of RequestAbortedError (5.x has no AbortError export; its
* RequestAbortedError extends UndiciError directly, which no 5.x program can
* observe through the exported classes). */
export class AbortError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "AbortError";
this.message = message || "The operation was aborted";
this.code = "UND_ERR_ABORT";
}
}
export class RequestAbortedError extends AbortError {
constructor(message?: string) {
super(message);
this.name = "AbortError";
this.message = message || "Request aborted";
this.code = "UND_ERR_ABORTED";
}
}
export class InformationalError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "InformationalError";
this.message = message || "Request information";
this.code = "UND_ERR_INFO";
}
}
export class RequestContentLengthMismatchError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "RequestContentLengthMismatchError";
this.message = message || "Request body length does not match content-length header";
this.code = "UND_ERR_REQ_CONTENT_LENGTH_MISMATCH";
}
}
export class ResponseContentLengthMismatchError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "ResponseContentLengthMismatchError";
this.message = message || "Response body length does not match content-length header";
this.code = "UND_ERR_RES_CONTENT_LENGTH_MISMATCH";
}
}
export class ClientDestroyedError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "ClientDestroyedError";
this.message = message || "The client is destroyed";
this.code = "UND_ERR_DESTROYED";
}
}
export class ClientClosedError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "ClientClosedError";
this.message = message || "The client is closed";
this.code = "UND_ERR_CLOSED";
}
}
export class SocketError extends UndiciError {
socket: SocketInfo | null;
constructor(message?: string, socket?: SocketInfo | null) {
super(message);
this.name = "SocketError";
this.message = message || "Socket error";
this.code = "UND_ERR_SOCKET";
this.socket = socket ?? null;
}
}
export class NotSupportedError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "NotSupportedError";
this.message = message || "Not supported error";
this.code = "UND_ERR_NOT_SUPPORTED";
}
}
export class BalancedPoolMissingUpstreamError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "MissingUpstreamError";
this.message = message || "No upstream has been added to the BalancedPool";
this.code = "UND_ERR_BPL_MISSING_UPSTREAM";
}
}
export class HTTPParserError extends Error {
code: string | undefined;
data: string | undefined;
constructor(message?: string, code?: string, data?: string) {
super(message);
this.name = "HTTPParserError";
this.code = code ? `HPE_${code}` : undefined;
this.data = data ? data : undefined;
}
}
export class ResponseExceededMaxSizeError extends UndiciError {
constructor(message?: string) {
super(message);
this.name = "ResponseExceededMaxSizeError";
this.message = message || "Response content exceeded max size";
this.code = "UND_ERR_RES_EXCEEDED_MAX_SIZE";
}
}
/** 5.x only (throwOnError). */
export class ResponseStatusCodeError extends UndiciError {
body: unknown;
status: number;
statusCode: number;
headers: Record<string, string | string[]> | null;
constructor(
message?: string,
statusCode?: number,
headers?: Record<string, string | string[]> | null,
body?: unknown,
) {
super(message);
this.name = "ResponseStatusCodeError";
this.message = message || "Response Status Code Error";
this.code = "UND_ERR_RESPONSE_STATUS_CODE";
this.body = body;
this.status = statusCode ?? 0;
this.statusCode = statusCode ?? 0;
this.headers = headers ?? null;
}
}
/** 7.x only. */
export class RequestRetryError extends UndiciError {
statusCode: number;
data: { count: number };
headers: Record<string, string | string[]>;
constructor(
message: string,
code: number,
options: { headers?: Record<string, string | string[]>; data?: { count: number } },
) {
super(message);
this.name = "RequestRetryError";
this.message = message || "Request retry error";
this.code = "UND_ERR_REQ_RETRY";
this.statusCode = code;
this.data = options.data ?? { count: 0 };
this.headers = options.headers ?? {};
}
}
/** 7.x only. */
export class ResponseError extends UndiciError {
statusCode: number;
body: unknown;
headers: Record<string, string | string[]> | null;
constructor(
message: string,
code: number,
options: { headers?: Record<string, string | string[]> | null; body?: unknown },
) {
super(message);
this.name = "ResponseError";
this.message = message || "Response error";
this.code = "UND_ERR_RESPONSE";
this.statusCode = code;
this.body = options.body;
this.headers = options.headers ?? null;
}
}
packages/compiler/shims/undici/core/fetch.ts
+635
View File
@@ -0,0 +1,635 @@
// undici's fetch(). Without a dispatcher (no init.dispatcher and no
// program-installed global dispatcher) it is the runtime's own fetch, which
// implements the same WHATWG algorithm. With one, the request goes through
// that dispatcher (its connector, lookup and pool), following undici's
// fetch for request headers, redirects and errors. The promise resolves
// when the response headers arrive, and the Response body streams from the
// connection with undici's back-pressure (see dispatchOnce).
//
// Known gaps: a Response built here reports `url` as "" and `redirected` as
// false (the runtime's Response has no settable URL yet), and https requests
// offer `gzip, deflate` rather than undici's `br, gzip, deflate` (the runtime
// has no Brotli decoder).
import type { Transform } from "node:stream";
import * as zlib from "node:zlib";
import type { Dispatcher } from "./dispatcher.ts";
import { getGlobalDispatcher, hasCustomGlobalDispatcher } from "./api.ts";
import type { DispatchHandlers, Profile } from "./types.ts";
/** undici's RequestInit, restricted to the implemented members: string and
* byte bodies, and the init members the dispatcher path honors. */
export interface UndiciRequestInit {
method?: string;
headers?: Headers | [string, string][] | Record<string, string>;
body?: string | Uint8Array | null;
signal?: AbortSignal | null;
redirect?: "follow" | "manual" | "error";
dispatcher?: Dispatcher;
}
/** The fetch standard's blocked ports (undici lib/web/fetch/constants.js). */
const badPorts = new Set([
"1",
"7",
"9",
"11",
"13",
"15",
"17",
"19",
"20",
"21",
"22",
"23",
"25",
"37",
"42",
"43",
"53",
"69",
"77",
"79",
"87",
"95",
"101",
"102",
"103",
"104",
"109",
"110",
"111",
"113",
"115",
"117",
"119",
"123",
"135",
"137",
"139",
"143",
"161",
"179",
"389",
"427",
"465",
"512",
"513",
"514",
"515",
"526",
"530",
"531",
"532",
"540",
"548",
"554",
"556",
"563",
"587",
"601",
"636",
"989",
"990",
"993",
"995",
"1719",
"1720",
"1723",
"2049",
"3659",
"4045",
"4190",
"5060",
"5061",
"6000",
"6566",
"6665",
"6666",
"6667",
"6668",
"6669",
"6679",
"6697",
"10080",
]);
interface RawResponse {
status: number;
statusText: string;
headers: string[];
/** null when the response has no body to expose: a null-body status or
* HEAD (its bytes, if any, are read and dropped), or a redirect fetch
* follows or rejects (the request is aborted, closing its connection). */
body: ReadableStream<Uint8Array> | null;
}
const redirectStatuses = [301, 302, 303, 307, 308];
const nullBodyStatuses = [101, 204, 205, 304];
export function undiciFetch(
input: string | URL,
init: UndiciRequestInit | undefined,
createGlobal: () => Dispatcher,
profile: Profile,
): Promise<Response> {
const explicit = init !== undefined ? init.dispatcher : undefined;
if (explicit === undefined && !hasCustomGlobalDispatcher()) {
if (init === undefined) return fetch(input);
return fetch(input, {
method: init.method,
headers: init.headers,
body: init.body,
signal: init.signal ?? undefined,
redirect: init.redirect,
});
}
const dispatcher = explicit ?? getGlobalDispatcher(createGlobal);
return fetchThrough(dispatcher, input, init ?? {}, profile);
}
function normalizeMethod(method: string): string {
const upper = method.toUpperCase();
return ["DELETE", "GET", "HEAD", "OPTIONS", "POST", "PUT", "PATCH"].includes(upper)
? upper
: method;
}
async function fetchThrough(
dispatcher: Dispatcher,
input: string | URL,
init: UndiciRequestInit,
profile: Profile,
): Promise<Response> {
let url = typeof input === "string" ? new URL(input) : new URL(input.href);
let method = normalizeMethod(init.method ?? "GET");
const signal = init.signal ?? null;
const redirectMode = init.redirect ?? "follow";
// Request headers: the caller's (as a Headers list: lower-cased, sorted,
// combined), then undici fetch's defaults.
const headers = headerListOf(init.headers);
let body: Buffer | null = null;
const rawBody = init.body ?? null;
if (rawBody !== null) {
if (typeof rawBody === "string") {
body = Buffer.from(rawBody);
if (!headers.has("content-type")) headers.set("content-type", "text/plain;charset=UTF-8");
} else {
body = Buffer.from(rawBody.buffer, rawBody.byteOffset, rawBody.byteLength);
}
}
if (!headers.has("accept")) headers.set("accept", "*/*");
if (!headers.has("accept-language")) headers.set("accept-language", "*");
for (let redirects = 0; ; redirects++) {
if (signal !== null && signal.aborted) throw abortReason(signal);
if (badPorts.has(url.port)) {
throw new TypeError("fetch failed", { cause: new Error("bad port") });
}
const requestHeaders = new HeaderList(headers);
if (body !== null) {
if (!requestHeaders.has("content-length"))
requestHeaders.set("content-length", String(body.length));
} else if ((method === "POST" || method === "PUT") && !requestHeaders.has("content-length")) {
requestHeaders.set("content-length", "0");
}
requestHeaders.set("sec-fetch-mode", "cors");
if (!requestHeaders.has("user-agent")) requestHeaders.set("user-agent", "undici");
// undici also offers br over https; the native runtime has no Brotli
// decoder, so only the codings it can decode are offered.
if (!requestHeaders.has("accept-encoding"))
requestHeaders.set("accept-encoding", "gzip, deflate");
const pairs = requestHeaders.pairs();
let raw: RawResponse;
try {
raw = await dispatchOnce(dispatcher, url, method, pairs, body, signal, redirectMode, profile);
} catch (err) {
if (signal !== null && signal.aborted) throw abortReason(signal);
throw new TypeError("fetch failed", { cause: err });
}
const responseHeaders = new Headers();
for (let i = 0; i + 1 < raw.headers.length; i += 2) {
responseHeaders.append(raw.headers[i]!, raw.headers[i + 1]!);
}
const location = responseHeaders.get("location");
if (redirectStatuses.includes(raw.status) && location !== null && redirectMode !== "manual") {
if (redirectMode === "error") {
throw new TypeError("fetch failed", { cause: new Error("unexpected redirect") });
}
if (redirects >= 20) {
throw new TypeError("fetch failed", { cause: new Error("redirect count exceeded") });
}
const next = new URL(location, url);
if (
raw.status === 303 && method !== "GET" && method !== "HEAD"
? true
: (raw.status === 301 || raw.status === 302) && method === "POST"
) {
method = "GET";
body = null;
for (const name of [
"content-encoding",
"content-language",
"content-location",
"content-type",
"content-length",
]) {
headers.delete(name);
}
}
if (next.origin !== url.origin) {
headers.delete("authorization");
headers.delete("proxy-authorization");
headers.delete("cookie");
headers.delete("host");
}
url = next;
continue;
}
return new Response(raw.body, {
status: raw.status,
statusText: raw.statusText,
headers: responseHeaders,
});
}
}
/** A Fetch header list: lower-cased names, values combined with ", " in
* insertion order, serialized sorted by name (set-cookie values stay
* separate) — what a Headers object yields when iterated. */
class HeaderList {
names: string[];
values: Map<string, string[]>;
constructor(from: HeaderList | null) {
this.names = from !== null ? from.names.slice() : [];
this.values = new Map<string, string[]>();
if (from !== null)
for (const name of from.names) this.values.set(name, from.values.get(name)!.slice());
}
has(name: string): boolean {
return this.values.has(name.toLowerCase());
}
append(name: string, value: string): void {
const key = name.toLowerCase();
const normalized = value.replace(/^[\t\n\r ]+|[\t\n\r ]+$/g, "");
const existing = this.values.get(key);
if (existing !== undefined) existing.push(normalized);
else {
this.names.push(key);
this.values.set(key, [normalized]);
}
}
set(name: string, value: string): void {
this.delete(name);
this.append(name, value);
}
delete(name: string): void {
const key = name.toLowerCase();
if (!this.values.has(key)) return;
this.values.delete(key);
this.names = this.names.filter((n) => n !== key);
}
pairs(): string[] {
const out: string[] = [];
for (const name of this.names.slice().sort()) {
const list = this.values.get(name)!;
if (name === "set-cookie") for (const value of list) out.push(name, value);
else out.push(name, list.join(", "));
}
return out;
}
}
/** The caller's headers init as a header list. Records and pair arrays read
* directly; a Headers object is read through forEach. */
function headerListOf(
init: Headers | [string, string][] | Record<string, string> | undefined,
): HeaderList {
const list = new HeaderList(null);
if (init === undefined) return list;
if (init instanceof Headers) {
init.forEach((value: string, name: string) => list.append(name, value));
} else if (Array.isArray(init)) {
for (const pair of init) list.append(pair[0], pair[1]);
} else {
for (const name of Object.keys(init)) list.append(name, init[name]!);
}
return list;
}
/** undici 7's limit on a response's content codings, applied in both
* profiles: undici 5 has none, and builds one decoder per listed coding. */
const maxContentEncodings = 5;
/** Whether every coding has a decoder; otherwise the body passes through
* as sent. */
function decodable(codings: string[]): boolean {
if (codings.length === 0) return false;
for (const coding of codings) {
if (coding !== "gzip" && coding !== "x-gzip" && coding !== "deflate") return false;
}
return true;
}
/** A decoder for one coding. undici 7 decodes with Z_SYNC_FLUSH, which
* accepts a truncated stream; undici 5 uses zlib's defaults. */
function decoderFor(coding: string, profile: Profile): Transform {
const options =
profile.major >= 7
? { flush: zlib.constants.Z_SYNC_FLUSH, finishFlush: zlib.constants.Z_SYNC_FLUSH }
: {};
return coding === "deflate" ? zlib.createInflate(options) : zlib.createGunzip(options);
}
function abortReason(signal: AbortSignal): Error {
const reason: unknown = signal.reason;
return reason instanceof Error
? reason
: new DOMException("This operation was aborted", "AbortError");
}
/** Unread bytes held beside the response stream before the connection
* pauses: Node's Readable highWaterMark, which undici's fetch body uses. */
const bodyHighWaterMark = 64 * 1024;
/** Dispatches one request and resolves when its response headers arrive.
* The body streams as undici's fetch reads one: up to bodyHighWaterMark
* unread bytes wait beside the stream, past that the connection (or the
* last decoder) pauses until the reader pulls again; cancelling the stream
* aborts the request, and an error after the headers errors the stream
* with TypeError('terminated'). */
function dispatchOnce(
dispatcher: Dispatcher,
url: URL,
method: string,
headers: string[],
body: Buffer | null,
signal: AbortSignal | null,
redirectMode: "follow" | "manual" | "error",
profile: Profile,
): Promise<RawResponse> {
return new Promise<RawResponse>((resolve, reject) => {
let resolved = false;
let done = false;
let dump = false;
let abortFn: ((err?: Error) => void) | null = null;
// The response body: its stream controller, the unread bytes beside it,
// and how to restart the paused producer.
let controller: ReadableStreamDefaultController<Uint8Array> | null = null;
let hasBody = false;
const queue: Buffer[] = [];
let queued = 0;
let ended = false;
let closed = false;
let onDrain: (() => void) | null = null;
// The decoders, first-applied last; their listeners hold this scope, so
// releaseDecoders drops them once the body ends, fails or is cancelled.
let decoding = false;
let decoders: Transform[] = [];
let first: Transform | null = null;
let encodedBytes = 0;
let resumeConnection: (() => void) | null = null;
const abortRequest = (err: Error): void => {
const abort = abortFn;
if (abort !== null && !done) abort(err);
};
const onAbort = (): void => {
if (signal === null) return;
const reason = abortReason(signal);
// After the headers undici 7 errors the body with the signal's
// reason; undici 5 aborts its fetch controller without one, so the
// body gets the controller's default AbortError.
errorBody(
profile.major >= 7 ? reason : new DOMException("The operation was aborted.", "AbortError"),
);
abortRequest(reason);
};
const stopListening = (): void => {
if (signal !== null) signal.removeEventListener("abort", onAbort);
};
if (signal !== null) signal.addEventListener("abort", onAbort);
const releaseDecoders = (): void => {
const chain = decoders;
decoders = [];
first = null;
for (const decoder of chain) {
decoder.removeAllListeners();
decoder.destroy();
}
};
const closeBody = (): void => {
closed = true;
controller!.close();
stopListening();
};
const errorBody = (err: unknown): void => {
if (!hasBody || closed) return;
closed = true;
queue.length = 0;
queued = 0;
controller!.error(err);
stopListening();
releaseDecoders();
};
const endBody = (): void => {
if (closed) return;
ended = true;
if (queue.length === 0) closeBody();
};
/** Takes a chunk; false when the producer should pause until onDrain. */
const pushBody = (chunk: Buffer): boolean => {
if (closed) return true;
if (queue.length === 0 && (controller!.desiredSize ?? 0) > 0) {
controller!.enqueue(new Uint8Array(chunk));
return true;
}
queue.push(chunk);
queued += chunk.length;
return queued < bodyHighWaterMark;
};
const pullBody = (): void => {
if (closed) return;
const next = queue.shift();
if (next !== undefined) {
queued -= next.length;
controller!.enqueue(new Uint8Array(next));
}
if (queue.length === 0 && ended) {
closeBody();
return;
}
const drain = onDrain;
if (drain !== null && queued < bodyHighWaterMark) {
onDrain = null;
drain();
}
};
const cancelBody = (reason: unknown): void => {
if (closed) return;
closed = true;
queue.length = 0;
queued = 0;
stopListening();
releaseDecoders();
abortRequest(
reason instanceof Error
? reason
: new DOMException("The operation was aborted.", "AbortError"),
);
};
const failBody = (err: Error): void => {
errorBody(new TypeError("terminated", { cause: err }));
abortRequest(err);
};
const handler: DispatchHandlers = {
onConnect: (abort: (err?: Error) => void): void => {
abortFn = abort;
if (signal !== null && signal.aborted) abort(abortReason(signal));
},
onHeaders: (statusCode: number, raw: Buffer[], resume: () => void, text: string): boolean => {
if (statusCode < 200) return true;
resolved = true;
resumeConnection = resume;
const rawHeaders: string[] = [];
let location = false;
let encoding: string | null = null;
for (let i = 0; i + 1 < raw.length; i += 2) {
const name = raw[i]!.toString("latin1");
const value = raw[i + 1]!.toString("latin1");
rawHeaders.push(name, value);
const lower = name.toLowerCase();
if (lower === "location") location = true;
else if (lower === "content-encoding")
encoding = encoding === null ? value : `${encoding}, ${value}`;
}
const follows =
redirectStatuses.includes(statusCode) && location && redirectMode !== "manual";
if (follows || nullBodyStatuses.includes(statusCode) || method === "HEAD") {
dump = true;
stopListening();
resolve({ status: statusCode, statusText: text, headers: rawHeaders, body: null });
// A redirect that is followed or rejected never reads its body:
// abort the request so the connection is released instead of
// draining a body that may never end.
if (follows) abortRequest(new DOMException("The operation was aborted.", "AbortError"));
return true;
}
const codings =
encoding === null
? []
: encoding
.toLowerCase()
.split(",")
.map((c) => c.trim());
if (codings.length > maxContentEncodings) {
dump = true;
stopListening();
reject(
new Error(
`too many content-encodings in response: ${codings.length}, maximum allowed is ${maxContentEncodings}`,
),
);
abortRequest(new DOMException("The operation was aborted.", "AbortError"));
return true;
}
hasBody = true;
const stream = new ReadableStream<Uint8Array>({
start: (c: ReadableStreamDefaultController<Uint8Array>): void => {
controller = c;
},
pull: (): void => pullBody(),
cancel: (reason: unknown): void => cancelBody(reason),
});
if (decodable(codings)) {
// Decoders run last-applied coding first, piped into each other;
// the last one feeds the body.
decoding = true;
let last: Transform | null = null;
for (let i = codings.length - 1; i >= 0; i--) {
const decoder = decoderFor(codings[i]!, profile);
decoder.on("error", (err: Error) => failBody(err));
if (last === null) first = decoder;
else last.pipe(decoder);
decoders.push(decoder);
last = decoder;
}
const output = last!;
output.on("data", (chunk: Buffer) => {
if (!pushBody(chunk)) {
output.pause();
onDrain = (): void => {
output.resume();
};
}
});
output.on("end", () => {
releaseDecoders();
endBody();
});
first!.on("drain", () => resume());
}
resolve({ status: statusCode, statusText: text, headers: rawHeaders, body: stream });
return true;
},
onData: (chunk: Buffer): boolean => {
if (dump || !hasBody || closed) return true;
if (decoding) {
const decoder = first;
if (decoder === null) return true;
encodedBytes += chunk.length;
return decoder.write(chunk);
}
if (pushBody(chunk)) return true;
onDrain = resumeConnection;
return false;
},
onComplete: (_trailers: string[] | null): void => {
done = true;
if (!hasBody) {
stopListening();
return;
}
// zlib rejects an empty stream; undici ends such a body unread.
const decoder = first;
if (decoder !== null && encodedBytes > 0) {
decoder.end();
} else {
releaseDecoders();
endBody();
}
},
onError: (err: Error): void => {
done = true;
if (!resolved) {
stopListening();
reject(err);
return;
}
errorBody(new TypeError("terminated", { cause: err }));
},
};
dispatcher.dispatch(
{
origin: url.origin,
path: url.pathname + url.search,
method,
headers,
body,
},
handler,
);
});
}
packages/compiler/shims/undici/core/pool.ts
+280
View File
@@ -0,0 +1,280 @@
// undici's Pool (lib/dispatcher/pool.js, pool-base.js, pool-stats.js): a
// set of Clients for one origin, created on demand up to `connections`, with
// a queue for requests no client can take.
import { buildConnector } from "./connector.ts";
import { Client } from "./client.ts";
import { Dispatcher, DispatcherBase } from "./dispatcher.ts";
import { InvalidArgumentError } from "./errors.ts";
import { parseOrigin } from "./util.ts";
import type {
ClientOptions,
DispatchHandlers,
DispatchOptions,
PoolOptions,
Profile,
} from "./types.ts";
interface QueuedDispatch {
opts: DispatchOptions;
handler: DispatchHandlers;
}
/** pool-stats.js: live counters over a pool's clients. */
export class PoolStats {
__pool: PoolBase;
constructor(pool: PoolBase) {
this.__pool = pool;
}
get connected(): number {
return this.__pool.__connectedCount();
}
get free(): number {
return this.__pool.__freeCount();
}
get pending(): number {
return this.__pool.__pendingCount();
}
get queued(): number {
return this.__pool.__queuedCount();
}
get running(): number {
return this.__pool.__runningCount();
}
get size(): number {
return this.__pool.__sizeCount();
}
}
export class PoolBase extends DispatcherBase {
__clients: Dispatcher[];
__clientNeedDrain: Map<Dispatcher, boolean>;
__queue: QueuedDispatch[];
__queued: number;
__needDrain: boolean;
__closedResolve: (() => void) | null;
__stats: PoolStats;
__onDrain: (origin: URL, targets: Dispatcher[]) => void;
__onConnect: (origin: URL, targets: Dispatcher[]) => void;
__onDisconnect: (origin: URL, targets: Dispatcher[], err: Error) => void;
__onConnectionError: (origin: URL, targets: Dispatcher[], err: Error) => void;
constructor(profile: Profile) {
super(profile);
this.__clients = [];
this.__clientNeedDrain = new Map<Dispatcher, boolean>();
this.__queue = [];
this.__queued = 0;
this.__needDrain = false;
this.__closedResolve = null;
this.__stats = new PoolStats(this);
this.__onDrain = (origin: URL, targets: Dispatcher[]): void => {
const client = targets[0]!;
let needDrain = false;
while (!needDrain) {
const item = this.__queue.shift();
if (item === undefined) break;
this.__queued--;
needDrain = !client.dispatch(item.opts, item.handler);
}
this.__clientNeedDrain.set(client, needDrain);
if (!needDrain && this.__needDrain) {
this.__needDrain = false;
this.emit("drain", origin, [this as Dispatcher, ...targets]);
}
const closed = this.__closedResolve;
if (closed !== null && this.__queue.length === 0) {
Promise.all(this.__clients.map((c) => c.close())).then(() => closed());
}
};
this.__onConnect = (origin: URL, targets: Dispatcher[]): void => {
this.emit("connect", origin, [this as Dispatcher, ...targets]);
};
this.__onDisconnect = (origin: URL, targets: Dispatcher[], err: Error): void => {
this.emit("disconnect", origin, [this as Dispatcher, ...targets], err);
};
this.__onConnectionError = (origin: URL, targets: Dispatcher[], err: Error): void => {
this.emit("connectionError", origin, [this as Dispatcher, ...targets], err);
};
}
get stats(): PoolStats {
return this.__stats;
}
__busyFlag(): boolean {
return this.__needDrain;
}
__connectedCount(): number {
return this.__clients.filter((c) => c.__connectedFlag()).length;
}
__freeCount(): number {
return this.__clients.filter((c) => c.__connectedFlag() && !this.__clientNeedDrain.get(c))
.length;
}
__pendingCount(): number {
let ret = this.__queued;
for (const c of this.__clients) ret += c.__pendingCount();
return ret;
}
__queuedCount(): number {
return this.__queued;
}
__runningCount(): number {
let ret = 0;
for (const c of this.__clients) ret += c.__runningCount();
return ret;
}
__sizeCount(): number {
let ret = this.__queued;
for (const c of this.__clients) ret += c.__sizeCount();
return ret;
}
__close(): Promise<void> {
if (this.__queue.length === 0) {
return Promise.all(this.__clients.map((c) => c.close())).then(() => {});
}
return new Promise<void>((resolve) => {
this.__closedResolve = resolve;
});
}
__destroy(err: Error): Promise<void> {
for (;;) {
const item = this.__queue.shift();
if (item === undefined) break;
if (item.handler.onError !== undefined) item.handler.onError!(err);
}
return Promise.all(this.__clients.map((c) => c.destroy(err))).then(() => {});
}
__dispatch(opts: DispatchOptions, handler: DispatchHandlers): boolean {
const dispatcher = this.__getDispatcher();
if (dispatcher === null) {
this.__needDrain = true;
this.__queue.push({ opts, handler });
this.__queued++;
} else if (!dispatcher.dispatch(opts, handler)) {
this.__clientNeedDrain.set(dispatcher, true);
this.__needDrain = this.__getDispatcher() === null;
}
return !this.__needDrain;
}
__getDispatcher(): Dispatcher | null {
return null;
}
__addClient(client: Dispatcher): void {
client.on("drain", this.__onDrain);
client.on("connect", this.__onConnect);
client.on("disconnect", this.__onDisconnect);
client.on("connectionError", this.__onConnectionError);
this.__clients.push(client);
this.__clientNeedDrain.set(client, false);
if (this.__needDrain) {
process.nextTick(() => {
if (this.__needDrain) this.__onDrain(clientUrl(client), [client, this as Dispatcher]);
});
}
}
}
function clientUrl(client: Dispatcher): URL {
return client instanceof Client ? client.__url : new URL("http://localhost");
}
export class Pool extends PoolBase {
__connections: number | null;
__url: URL;
__options: ClientOptions;
__factory: ((origin: URL, opts: object) => Dispatcher) | null;
constructor(origin: string | URL, options: PoolOptions | undefined, profile: Profile) {
super(profile);
const opts: PoolOptions = options ?? {};
const connections = opts.connections ?? null;
if (connections !== null && (!Number.isFinite(connections) || connections < 0)) {
throw new InvalidArgumentError("invalid connections");
}
const connect = opts.connect ?? null;
const connector =
typeof connect === "function"
? connect
: buildConnector(
connect !== null
? {
...connect,
timeout:
connect.timeout !== undefined ? connect.timeout : (opts.connectTimeout ?? 10e3),
}
: { timeout: opts.connectTimeout ?? 10e3 },
);
const interceptors = opts.interceptors;
if (profile.major < 7 && interceptors !== undefined && interceptors.Pool !== undefined) {
this.__interceptors = interceptors.Pool;
}
this.__connections = connections !== null && connections !== 0 ? connections : null;
this.__url = parseOrigin(origin, profile);
this.__options = {
keepAliveTimeout: opts.keepAliveTimeout,
keepAliveMaxTimeout: opts.keepAliveMaxTimeout,
keepAliveTimeoutThreshold: opts.keepAliveTimeoutThreshold,
pipelining: opts.pipelining,
maxHeaderSize: opts.maxHeaderSize,
bodyTimeout: opts.bodyTimeout,
headersTimeout: opts.headersTimeout,
strictContentLength: opts.strictContentLength,
maxRequestsPerClient: opts.maxRequestsPerClient,
maxResponseSize: opts.maxResponseSize,
localAddress: opts.localAddress,
interceptors: interceptors !== undefined ? { Client: interceptors.Client } : undefined,
connect: connector,
};
const factory = opts.factory;
this.__factory = factory !== undefined ? factory : null;
if (profile.major >= 7) {
// 7.x drops a client whose connection attempt failed.
this.on("connectionError", (_origin: URL, targets: Dispatcher[], _err: Error) => {
for (const target of targets) {
const idx = this.__clients.indexOf(target);
if (idx !== -1) this.__clients.splice(idx, 1);
}
});
}
}
__getDispatcher(): Dispatcher | null {
for (const client of this.__clients) {
if (!this.__clientNeedDrain.get(client)) return client;
}
if (this.__connections === null || this.__clients.length < this.__connections) {
const factory = this.__factory;
const client =
factory !== null
? factory(this.__url, this.__options)
: this.__createClient(this.__url, this.__options);
this.__addClient(client);
return client;
}
return null;
}
/** The default factory: a Client of the same undici version. */
__createClient(origin: URL, options: ClientOptions): Dispatcher {
return new Client(origin, options, this.__profile);
}
}
packages/compiler/shims/undici/core/readable.ts
+268
View File
@@ -0,0 +1,268 @@
// undici's response body (lib/api/readable.js): a readable byte stream fed
// by the dispatch handler, with the fetch body mixin (text/json/
// arrayBuffer/blob/bytes) and dump().
//
// It follows node:stream's Readable contract for the members it declares
// (flowing on a 'data' listener or resume(), pause(), 'data'/'end'/'close'/
// 'error' events and their ordering, destroy(err), back-pressure through
// push()'s return value), built on EventEmitter: scriptc compiles program
// classes extending EventEmitter under both its own Node declarations and
// @types/node, which a node:stream subclass does not get.
import { EventEmitter } from "node:events";
import { NotSupportedError, RequestAbortedError } from "./errors.ts";
import type { Profile } from "./types.ts";
const HIGH_WATER_MARK = 64 * 1024;
export class BodyReadable extends EventEmitter {
__resumeFn: () => void;
__abortFn: (err: Error) => void;
__contentType: string;
__consuming: boolean;
__profile: Profile;
__buffer: Buffer[];
__length: number;
__flowing: boolean;
__ended: boolean;
__endEmitted: boolean;
__destroyed: boolean;
__flowScheduled: boolean;
__needsResume: boolean;
constructor(
resume: () => void,
abort: (err: Error) => void,
contentType: string,
profile: Profile,
) {
super();
this.__resumeFn = resume;
this.__abortFn = abort;
this.__contentType = contentType;
this.__consuming = false;
this.__profile = profile;
this.__buffer = [];
this.__length = 0;
this.__flowing = false;
this.__ended = false;
this.__endEmitted = false;
this.__destroyed = false;
this.__flowScheduled = false;
this.__needsResume = false;
// Readable.on('data') switches the stream to flowing mode.
this.on("newListener", (event: string) => {
if (event === "data" && !this.__flowing) this.resume();
});
}
get destroyed(): boolean {
return this.__destroyed;
}
get readableEnded(): boolean {
return this.__endEmitted;
}
get readableLength(): number {
return this.__length;
}
get readableFlowing(): boolean {
return this.__flowing;
}
/** Readable.push: queue a chunk (null ends the body). Returns false once
* the buffer reaches the high-water mark; the dispatcher then pauses the
* connection until the consumer drains it. */
push(chunk: Buffer | null): boolean {
if (this.__destroyed || this.__ended) return false;
if (chunk === null) {
this.__ended = true;
if (this.__length === 0) this.__scheduleEnd();
else this.__scheduleFlow();
return false;
}
if (this.__flowing && this.__length === 0 && this.listenerCount("data") > 0) {
this.emit("data", chunk);
return true;
}
this.__buffer.push(chunk);
this.__length += chunk.length;
if (this.__flowing) this.__scheduleFlow();
const below = this.__length < HIGH_WATER_MARK;
if (!below) this.__needsResume = true;
return below;
}
resume(): this {
if (!this.__flowing) {
this.__flowing = true;
this.__scheduleFlow();
}
return this;
}
pause(): this {
this.__flowing = false;
return this;
}
destroy(err?: Error): this {
if (this.__destroyed) return this;
this.__destroyed = true;
let error: Error | null = err ?? null;
if (error === null && !this.__endEmitted) error = new RequestAbortedError();
if (error !== null) this.__abortFn(error);
this.__buffer = [];
this.__length = 0;
const reported = error;
process.nextTick(() => {
if (reported !== null) this.emit("error", reported);
this.emit("close");
});
return this;
}
__scheduleFlow(): void {
if (this.__flowScheduled) return;
this.__flowScheduled = true;
process.nextTick(() => {
this.__flowScheduled = false;
this.__flow();
});
}
__flow(): void {
while (this.__flowing && !this.__destroyed && this.__buffer.length > 0) {
const chunk = this.__buffer.shift()!;
this.__length -= chunk.length;
this.emit("data", chunk);
}
if (this.__destroyed) return;
if (this.__needsResume && this.__length < HIGH_WATER_MARK) {
this.__needsResume = false;
this.__resumeFn();
}
if (this.__ended && this.__length === 0 && this.__flowing) this.__scheduleEnd();
}
__scheduleEnd(): void {
process.nextTick(() => {
if (this.__endEmitted || this.__destroyed || !this.__flowing || this.__length > 0) return;
this.__endEmitted = true;
this.emit("end");
// autoDestroy: a fully read body closes.
process.nextTick(() => {
if (this.__destroyed) return;
this.__destroyed = true;
this.emit("close");
});
});
}
/** True once text()/json()/arrayBuffer()/bytes()/blob()/dump() started
* consuming the body. */
get bodyUsed(): boolean {
return this.__consuming;
}
text(): Promise<string> {
return this.__consume().then((bytes) => decodeText(bytes, this.__profile));
}
// undici declares json() as Promise<any>: callers assign the parsed value
// to their own annotated types without a cast, exactly as with undici.
// oxlint-disable-next-line typescript/no-explicit-any
json(): Promise<any> {
return this.__consume().then((bytes) => JSON.parse(decodeText(bytes, this.__profile)));
}
arrayBuffer(): Promise<ArrayBuffer> {
return this.__consume().then((bytes) => {
const out = new ArrayBuffer(bytes.length);
new Uint8Array(out).set(bytes);
return out;
});
}
bytes(): Promise<Uint8Array> {
return this.__consume().then((bytes) => new Uint8Array(bytes));
}
blob(): Promise<Blob> {
return this.__consume().then((bytes) => new Blob([bytes], { type: this.__contentType }));
}
formData(): Promise<never> {
return Promise.reject(new NotSupportedError());
}
/** Reads and discards up to `limit` bytes (default 256 KiB on 5.x, 128 KiB
* on 7.x), then resolves; read errors are swallowed like undici's. */
dump(opts?: { limit: number }): Promise<void> {
let limit =
opts !== undefined && Number.isFinite(opts.limit)
? opts.limit
: this.__profile.major >= 7
? 131072
: 262144;
this.__consuming = true;
return new Promise<void>((resolve) => {
if (this.__endEmitted || this.__destroyed) {
resolve();
return;
}
this.on("data", (chunk: Buffer) => {
limit -= chunk.length;
if (limit < 0) {
this.destroy();
resolve();
}
});
this.on("end", () => resolve());
this.on("error", () => resolve());
this.on("close", () => resolve());
});
}
__consume(): Promise<Buffer> {
if (this.bodyUsed) return Promise.reject(new TypeError("unusable"));
this.__consuming = true;
return new Promise<Buffer>((resolve, reject) => {
const chunks: Buffer[] = [];
let done = false;
this.on("data", (chunk: Buffer) => {
chunks.push(chunk);
});
this.on("end", () => {
if (done) return;
done = true;
resolve(Buffer.concat(chunks));
});
this.on("error", (err: Error) => {
if (done) return;
done = true;
reject(err);
});
this.on("close", () => {
if (done) return;
done = true;
reject(new RequestAbortedError());
});
});
}
}
/** 5.x decodes UTF-8 as-is; 7.x strips a leading byte-order mark first. */
function decodeText(bytes: Buffer, profile: Profile): string {
if (
profile.major >= 7 &&
bytes.length >= 3 &&
bytes[0] === 0xef &&
bytes[1] === 0xbb &&
bytes[2] === 0xbf
) {
return bytes.subarray(3).toString("utf8");
}
return bytes.toString("utf8");
}
packages/compiler/shims/undici/core/request.ts
+313
View File
@@ -0,0 +1,313 @@
// undici's per-dispatch request record (lib/core/request.js): option
// validation, header processing, and the handler callbacks with their
// diagnostics_channel publications.
import { channels } from "./diagnostics.ts";
import { InvalidArgumentError, NotSupportedError } from "./errors.ts";
import type { DispatchHandlers, DispatchOptions, Profile } from "./types.ts";
const tokenRegExp = /^[\^_`a-zA-Z\-0-9!#$%&'*+.|~]+$/;
const headerCharRegex = /[^\t\x20-\x7e\x80-\xff]/;
const invalidPathRegex = /[^!-ÿ]/;
export class Request {
origin: string;
method: string;
path: string;
body: Buffer | null;
/** Header name/value pairs in insertion order (host and content-length
* are consumed into their own fields, like undici). */
headerPairs: string[];
host: string | null;
contentLength: number | null;
contentType: string | null;
completed: boolean;
aborted: boolean;
idempotent: boolean;
blocking: boolean;
reset: boolean | null;
headersTimeout: number | null;
bodyTimeout: number | null;
throwOnError: boolean;
servername: string | null;
error: Error | null;
abortFn: ((err?: Error) => void) | null;
handler: DispatchHandlers;
profile: Profile;
/** The request as diagnostics_channel subscribers see it: one untyped
* object per request, so its identity is stable across publications.
* The request keeps its completed and headers fields current. */
diagnostic: unknown;
constructor(origin: string, opts: DispatchOptions, handler: DispatchHandlers, profile: Profile) {
const path = opts.path;
const method = opts.method;
if (typeof path !== "string") {
throw new InvalidArgumentError("path must be a string");
} else if (
path[0] !== "/" &&
!(path.startsWith("http://") || path.startsWith("https://")) &&
method !== "CONNECT"
) {
throw new InvalidArgumentError("path must be an absolute URL or start with a slash");
} else if (invalidPathRegex.test(path)) {
throw new InvalidArgumentError("invalid request path");
}
if (typeof method !== "string") {
throw new InvalidArgumentError("method must be a string");
} else if (!tokenRegExp.test(method)) {
throw new InvalidArgumentError("invalid request method");
}
if (method === "CONNECT") {
throw new NotSupportedError(
"scriptc's undici client does not implement CONNECT requests yet",
);
}
const headersTimeout = opts.headersTimeout ?? null;
if (headersTimeout !== null && (!Number.isFinite(headersTimeout) || headersTimeout < 0)) {
throw new InvalidArgumentError("invalid headersTimeout");
}
const bodyTimeout = opts.bodyTimeout ?? null;
if (bodyTimeout !== null && (!Number.isFinite(bodyTimeout) || bodyTimeout < 0)) {
throw new InvalidArgumentError("invalid bodyTimeout");
}
this.origin = origin;
this.method = method;
this.path = path;
this.headersTimeout = headersTimeout;
this.bodyTimeout = bodyTimeout;
// 7.x dropped throwOnError (the responseError interceptor replaced it).
if (profile.major >= 7 && opts.throwOnError !== undefined) {
throw new InvalidArgumentError("invalid throwOnError");
}
this.throwOnError = opts.throwOnError === true;
const body = opts.body ?? null;
if (body === null) {
this.body = null;
} else if (typeof body === "string") {
this.body = body.length > 0 ? Buffer.from(body) : null;
} else {
this.body =
body.byteLength > 0 ? Buffer.from(body.buffer, body.byteOffset, body.byteLength) : null;
}
this.completed = false;
this.profile = profile;
this.aborted = false;
this.idempotent = opts.idempotent ?? (method === "HEAD" || method === "GET");
this.blocking = opts.blocking ?? (profile.major >= 7 ? method !== "HEAD" : false);
this.reset = opts.reset ?? null;
this.host = null;
this.contentLength = null;
this.contentType = null;
this.headerPairs = [];
this.error = null;
this.abortFn = null;
const headers = opts.headers ?? null;
if (headers !== null) {
if (Array.isArray(headers)) {
if (headers.length % 2 !== 0) throw new InvalidArgumentError("headers array must be even");
for (let i = 0; i < headers.length; i += 2) {
processHeader(this, headers[i]!, headers[i + 1]!, profile);
}
} else {
for (const key of Object.keys(headers)) {
const value = headers[key];
if (value === undefined) continue;
if (typeof value === "string") {
processHeader(this, key, value, profile);
} else if (profile.major >= 7) {
for (const item of value) processHeader(this, key, item, profile);
} else {
throw new InvalidArgumentError(`invalid ${key} header`);
}
}
}
}
this.servername = serverName(this.host);
this.handler = handler;
this.diagnostic = null;
if (channels.create.hasSubscribers) channels.create.publish({ request: this.diagnosticView() });
}
diagnosticView(): unknown {
if (this.diagnostic === null) {
// addHeader adds a header as undici's Request.addHeader does (the
// tracing instrumentations' header injection).
const view: any = {
origin: this.origin,
completed: this.completed,
method: this.method,
path: this.path,
headers: this.diagnosticHeaders(),
addHeader: (key: string, value: string): unknown => {
this.addHeader(key, value);
return view;
},
};
this.diagnostic = view;
}
return this.diagnostic;
}
/** The diagnostics view's headers: undici 7 publishes its header array,
* undici 5 the raw block. */
diagnosticHeaders(): string | string[] {
return this.profile.major >= 7 ? this.headerPairs.slice() : this.headers;
}
/** The DiagnosticsChannel.Request view's raw header block. */
get headers(): string {
let out = "";
for (let i = 0; i < this.headerPairs.length; i += 2) {
out += `${this.headerPairs[i]}: ${this.headerPairs[i + 1]}\r\n`;
}
return out;
}
addHeader(key: string, value: string): Request {
processHeader(this, key, value, this.profile);
const view: any = this.diagnostic;
if (view !== null) view.headers = this.diagnosticHeaders();
return this;
}
onBodySent(chunk: Buffer): void {
const cb = this.handler.onBodySent;
if (cb !== undefined) {
try {
this.handler.onBodySent!(chunk.length, chunk.length);
} catch (err) {
this.onError(err as Error);
}
}
}
onRequestSent(): void {
if (channels.bodySent.hasSubscribers)
channels.bodySent.publish({ request: this.diagnosticView() });
}
onConnect(abort: (err?: Error) => void): void {
if (this.error !== null) {
abort(this.error);
return;
}
this.abortFn = abort;
if (this.handler.onConnect !== undefined) this.handler.onConnect!(abort);
}
onHeaders(
statusCode: number,
headers: Buffer[],
resume: () => void,
statusText: string,
): boolean {
if (channels.headers.hasSubscribers) {
channels.headers.publish({
request: this.diagnosticView(),
response: { statusCode, headers, statusText },
});
}
if (this.handler.onHeaders === undefined) return true;
return this.handler.onHeaders!(statusCode, headers, resume, statusText);
}
onData(chunk: Buffer): boolean {
if (this.handler.onData === undefined) return true;
return this.handler.onData!(chunk);
}
onComplete(trailers: Buffer[]): void {
this.completed = true;
const view: any = this.diagnostic;
if (view !== null) view.completed = true;
if (channels.trailers.hasSubscribers)
channels.trailers.publish({ request: this.diagnosticView(), trailers });
if (this.handler.onComplete !== undefined) {
const raw: string[] = [];
for (const t of trailers) raw.push(t.toString("latin1"));
this.handler.onComplete!(raw);
}
}
onError(error: Error): void {
if (channels.error.hasSubscribers)
channels.error.publish({ request: this.diagnosticView(), error });
if (this.aborted) return;
this.aborted = true;
if (this.handler.onError !== undefined) this.handler.onError!(error);
}
}
function processHeader(request: Request, key: string, val: string, profile: Profile): void {
const lower = key.toLowerCase();
// undici 7 checks the name and every value before the special cases;
// undici 5 checks the first Host value before storing it. Either way a
// CR or LF in a value never reaches the request head.
if (profile.major >= 7) {
if (!tokenRegExp.test(lower)) throw new InvalidArgumentError("invalid header key");
if (headerCharRegex.test(val)) throw new InvalidArgumentError(`invalid ${key} header`);
}
if (lower === "host") {
// The client writes the Host value into the request head as given.
if (headerCharRegex.test(val)) throw new InvalidArgumentError(`invalid ${key} header`);
if (request.host === null) {
if (headerCharRegex.test(val)) throw new InvalidArgumentError(`invalid ${key} header`);
request.host = val;
return;
}
if (profile.major >= 7) throw new InvalidArgumentError("duplicate host header");
} else if (lower === "content-length") {
if (request.contentLength === null) {
const n = parseInt(val, 10);
if (!Number.isFinite(n) || (profile.major >= 7 && !/^\d+$/.test(val))) {
throw new InvalidArgumentError("invalid content-length header");
}
request.contentLength = n;
return;
}
// The client writes the one Content-Length it validated or computed
// from the body; a second field would frame the request twice. Both
// profiles reject it with undici 7's error (5.x sends it as given).
throw new InvalidArgumentError("duplicate content-length header");
}
if (lower === "content-type" && request.contentType === null && !headerCharRegex.test(val)) {
request.contentType = val;
request.headerPairs.push(key, val);
return;
}
if (lower === "transfer-encoding" || lower === "keep-alive" || lower === "upgrade") {
throw new InvalidArgumentError(`invalid ${lower} header`);
}
if (lower === "connection") {
if (profile.major < 7) throw new InvalidArgumentError("invalid connection header");
for (const token of val.toLowerCase().split(",")) {
const trimmed = token.trim();
if (!tokenRegExp.test(trimmed)) throw new InvalidArgumentError("invalid connection header");
if (trimmed === "close") request.reset = true;
}
return;
}
if (lower === "expect") throw new NotSupportedError("expect header not supported");
if (!tokenRegExp.test(key)) throw new InvalidArgumentError("invalid header key");
if (headerCharRegex.test(val)) throw new InvalidArgumentError(`invalid ${key} header`);
request.headerPairs.push(key, val);
}
/** util.getServerName: the Host header without its port; null for IPs. */
function serverName(host: string | null): string | null {
if (host === null || host === "") return null;
let name = host;
if (name.startsWith("[")) {
const idx = name.indexOf("]");
if (idx !== -1) name = name.slice(1, idx);
return null;
}
const idx = name.indexOf(":");
if (idx !== -1) name = name.slice(0, idx);
return /^\d+\.\d+\.\d+\.\d+$/.test(name) ? null : name;
}
packages/compiler/shims/undici/core/timers.ts
+80
View File
@@ -0,0 +1,80 @@
// undici 7.x's fast timers (lib/util/timers.js): one unref'd native timer
// ticking every 499 ms drives every headers/body timeout, so those fire on
// the first tick at least `delay` after the timer (re)started.
const RESOLUTION_MS = 1e3;
const TICK_MS = (RESOLUTION_MS >> 1) - 1;
const NOT_IN_LIST = -2;
const TO_BE_CLEARED = -1;
const PENDING = 0;
const ACTIVE = 1;
let fastNow = 0;
// The ticking native timer, held as its refresh closure.
let refreshTick: (() => void) | null = null;
const fastTimers: FastTimer[] = [];
function onTick(): void {
fastNow += TICK_MS;
let idx = 0;
let len = fastTimers.length;
while (idx < len) {
const timer = fastTimers[idx]!;
if (timer.state === PENDING) {
timer.idleStart = fastNow - TICK_MS;
timer.state = ACTIVE;
} else if (timer.state === ACTIVE && fastNow >= timer.idleStart + timer.idleTimeout) {
timer.state = TO_BE_CLEARED;
timer.idleStart = -1;
timer.onTimeout();
}
if (timer.state === TO_BE_CLEARED) {
timer.state = NOT_IN_LIST;
if (--len !== 0) fastTimers[idx] = fastTimers[len]!;
} else {
++idx;
}
}
fastTimers.length = len;
// An idle tick stops; the next timer schedules a new one.
if (fastTimers.length !== 0) refreshTimeout();
else refreshTick = null;
}
function refreshTimeout(): void {
const refresh = refreshTick;
if (refresh !== null) {
refresh();
} else {
const timeout = setTimeout(onTick, TICK_MS);
timeout.unref();
refreshTick = () => {
timeout.refresh();
};
}
}
export class FastTimer {
state: number;
idleTimeout: number;
idleStart: number;
onTimeout: () => void;
constructor(callback: () => void, delay: number) {
this.state = NOT_IN_LIST;
this.idleTimeout = delay;
this.idleStart = -1;
this.onTimeout = callback;
this.refresh();
}
refresh(): void {
if (this.state === NOT_IN_LIST) fastTimers.push(this);
if (refreshTick === null || fastTimers.length === 1) refreshTimeout();
this.state = PENDING;
}
clear(): void {
this.state = TO_BE_CLEARED;
this.idleStart = -1;
}
}
packages/compiler/shims/undici/core/types.ts
+167
View File
@@ -0,0 +1,167 @@
// Types shared by the undici entries. They restate the subset of undici's
// own declarations (types/dispatcher.d.ts, connector.d.ts, ...) that the
// shim implements; test/package-shims/undici-types.test.ts checks them
// against the installed 5.x and 7.x declarations.
import type { Socket } from "node:net";
import type { Dispatcher } from "./dispatcher.ts";
/** Response and request header records (undici's IncomingHttpHeaders). */
export type IncomingHttpHeaders = Record<string, string | string[] | undefined>;
/** What a request's `headers` option accepts here: a record or a flat
* [name, value, ...] array. */
export type UndiciHeaders = Record<string, string | string[] | undefined> | string[] | null;
/** The payloads the shim writes. Streams, iterables, Blob and FormData
* bodies are outside the implemented subset. */
export type RequestBody = string | Buffer | Uint8Array | null;
export interface DispatchOptions {
origin?: string | URL;
path: string;
method: string;
body?: RequestBody;
headers?: UndiciHeaders;
idempotent?: boolean;
blocking?: boolean;
headersTimeout?: number | null;
bodyTimeout?: number | null;
reset?: boolean;
throwOnError?: boolean;
// Dispatcher.RequestOptions' members live here too: a record passed to
// dispatch() is copied to this shape, and interceptors read them back
// through a `params as Dispatcher.RequestOptions` cast, as with undici.
opaque?: unknown;
signal?: AbortSignal | null;
onInfo?: (info: { statusCode: number; headers: IncomingHttpHeaders }) => void;
highWaterMark?: number;
}
/** undici's dispatch-controller (7.x handler API). */
export interface DispatchController {
readonly aborted: boolean;
readonly paused: boolean;
readonly reason: Error | null;
abort(reason: Error): void;
pause(): void;
resume(): void;
}
/** Dispatch handlers: the 5.x callbacks, plus 7.x's controller callbacks. */
export interface DispatchHandlers {
onConnect?(abort: (err?: Error) => void): void;
onError?(err: Error): void;
onHeaders?(
statusCode: number,
headers: Buffer[],
resume: () => void,
statusText: string,
): boolean;
onData?(chunk: Buffer): boolean;
onComplete?(trailers: string[] | null): void;
onBodySent?(chunkSize: number, totalBytesSent: number): void;
onRequestStart?(controller: DispatchController, context: unknown): void;
onResponseStart?(
controller: DispatchController,
statusCode: number,
headers: IncomingHttpHeaders,
statusMessage?: string,
): void;
onResponseData?(controller: DispatchController, chunk: Buffer): void;
onResponseEnd?(controller: DispatchController, trailers: IncomingHttpHeaders): void;
onResponseError?(controller: DispatchController, error: Error): void;
}
export type DispatchFn = (options: DispatchOptions, handler: DispatchHandlers) => boolean;
export type DispatchInterceptor = (dispatch: DispatchFn) => DispatchFn;
/** buildConnector's per-connection parameters. */
export interface ConnectorOptions {
hostname: string;
host?: string;
protocol: string;
port: string;
servername?: string;
localAddress?: string | null;
}
export type ConnectorCallback = (err: Error | null, socket: Socket | null) => void;
/** A connector: dials, reports through the callback, and returns the socket
* (undici 5 and 7 both return it although their types declare void). */
export type Connector = (options: ConnectorOptions, callback: ConnectorCallback) => Socket;
/** net/tls connect options a connector forwards (the implemented subset). */
export interface ConnectorBuildOptions {
timeout?: number | null;
maxCachedSessions?: number | null;
socketPath?: string | null;
port?: number;
rejectUnauthorized?: boolean;
ca?: string | Buffer | (string | Buffer)[];
servername?: string;
/** net.connect's resolver hook: @types/node's LookupFunction is
* assignable (method syntax; the callback takes one address or all). */
lookup?(
hostname: string,
options: unknown,
callback: (
err: NodeJS.ErrnoException | null,
address: string | LookupAddress[],
family?: number,
) => void,
): void;
keepAlive?: boolean;
keepAliveInitialDelay?: number;
noDelay?: boolean;
autoSelectFamily?: boolean;
autoSelectFamilyAttemptTimeout?: number;
}
export interface LookupAddress {
address: string;
family: number;
}
export interface ClientOptions {
keepAliveTimeout?: number | null;
keepAliveMaxTimeout?: number | null;
keepAliveTimeoutThreshold?: number | null;
pipelining?: number | null;
connect?: ConnectorBuildOptions | Connector | null;
maxHeaderSize?: number | null;
bodyTimeout?: number | null;
headersTimeout?: number | null;
strictContentLength?: boolean;
maxRequestsPerClient?: number;
maxResponseSize?: number | null;
connectTimeout?: number | null;
localAddress?: string;
/** 5.x applies `Client` interceptors; 7.x ignores the option. */
interceptors?: { Client?: readonly DispatchInterceptor[] | undefined };
}
export interface PoolOptions extends ClientOptions {
factory?(origin: URL, opts: object): Dispatcher;
connections?: number | null;
interceptors?: {
Pool?: readonly DispatchInterceptor[] | undefined;
Client?: readonly DispatchInterceptor[] | undefined;
};
}
export interface AgentOptions extends PoolOptions {
factory?(origin: string | URL, opts: object): Dispatcher;
interceptors?: {
Agent?: readonly DispatchInterceptor[] | undefined;
Pool?: readonly DispatchInterceptor[] | undefined;
Client?: readonly DispatchInterceptor[] | undefined;
};
}
/** Version-dependent behavior, selected by the importing package's installed
* undici (the 5.x or 7.x entry). */
export interface Profile {
major: number;
}
packages/compiler/shims/undici/core/util.ts
+67
View File
@@ -0,0 +1,67 @@
// Helpers from undici's lib/core/util.js.
import { InvalidArgumentError } from "./errors.ts";
import type { IncomingHttpHeaders, Profile } from "./types.ts";
/** util.parseHeaders over raw [name, value, ...] pairs: lower-cased names,
* repeated names collected into arrays. 5.x decodes values as UTF-8 and
* treats an empty first value as absent; 7.x decodes latin1. */
export function parseHeaders(raw: readonly string[], profile: Profile): IncomingHttpHeaders {
const obj: IncomingHttpHeaders = {};
for (let i = 0; i + 1 < raw.length; i += 2) {
const key = raw[i]!.toLowerCase();
const value = raw[i + 1]!;
const existing = obj[key];
const absent = profile.major >= 7 ? existing === undefined : !existing;
if (absent) {
obj[key] = value;
} else if (typeof existing === "string") {
obj[key] = [existing, value];
} else if (existing !== undefined) {
existing.push(value);
}
}
return obj;
}
/** Raw header Buffers as strings (latin1 on 7.x, UTF-8 on 5.x). */
export function rawHeaderStrings(raw: readonly Buffer[], profile: Profile): string[] {
const out: string[] = [];
for (let i = 0; i < raw.length; i++) {
// Names are ASCII tokens; values follow the version's decoding.
out.push(
i % 2 === 0 || profile.major >= 7 ? raw[i]!.toString("latin1") : raw[i]!.toString("utf8"),
);
}
return out;
}
/** util.parseOrigin: an http(s) origin URL. */
export function parseOrigin(url: string | URL, profile: Profile): URL {
const parsed = parseURL(url, profile);
if (parsed.pathname !== "/" || parsed.search !== "" || parsed.hash !== "") {
throw new InvalidArgumentError("invalid url");
}
return parsed;
}
/** util.parseURL for the string/URL forms. */
export function parseURL(url: string | URL, profile: Profile): URL {
const parsed = typeof url === "string" ? new URL(url) : url;
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
// 7.x checks only string inputs; 5.x checks every input.
if (profile.major < 7) throw new InvalidArgumentError("invalid protocol");
if (typeof url === "string") {
throw new InvalidArgumentError(
"Invalid URL protocol: the URL must start with `http:` or `https:`.",
);
}
}
return parsed;
}
/** util.parseKeepAliveTimeout: the server's Keep-Alive `timeout=<s>` hint in
* milliseconds, or null. */
export function parseKeepAliveTimeout(value: string): number | null {
const m = /timeout=(\d+)/.exec(value);
return m !== null ? parseInt(m[1]!, 10) * 1000 : null;
}
packages/compiler/shims/undici/v5.ts
+119
View File
@@ -0,0 +1,119 @@
// scriptc's built-in implementation of the `undici` package, 5.x API.
//
// Programs whose installed undici is 5.x compile this module in its place
// (see packages/compiler/src/frontend/package-shims.ts). It implements the
// HTTP/1.1 client subset over node:net / node:tls: Dispatcher, Client, Pool,
// Agent, request(), buildConnector, errors, the global dispatcher, fetch and
// the diagnostics_channel publications. Anything else undici exports is not
// declared here, so using it is a compile-time error.
import { Agent as CoreAgent } from "./core/agent.ts";
import {
getGlobalDispatcher as coreGetGlobalDispatcher,
request as coreRequest,
setGlobalDispatcher as coreSetGlobalDispatcher,
} from "./core/api.ts";
import type { TopLevelRequestOptions } from "./core/api.ts";
import { Client as CoreClient } from "./core/client.ts";
import { buildConnector as coreBuildConnector } from "./core/connector.ts";
import { Dispatcher } from "./core/dispatcher.ts";
import type { ResponseData } from "./core/dispatcher.ts";
import { undiciFetch } from "./core/fetch.ts";
import type { UndiciRequestInit } from "./core/fetch.ts";
import { Pool as CorePool, PoolStats as CorePoolStats } from "./core/pool.ts";
import type {
AgentOptions,
ClientOptions,
Connector,
ConnectorBuildOptions,
ConnectorCallback,
ConnectorOptions,
DispatchOptions,
PoolOptions,
Profile,
} from "./core/types.ts";
import type { SocketInfo as CoreSocketInfo } from "./core/errors.ts";
const profile: Profile = { major: 5 };
export { Dispatcher };
export type { DiagnosticsChannel } from "./core/diagnostics-types.ts";
export * as errors from "./core/errors-v5.ts";
export class Client extends CoreClient {
constructor(url: string | URL, options?: Client.Options) {
super(url, options, profile);
}
}
export declare namespace Client {
type Options = ClientOptions;
type SocketInfo = CoreSocketInfo;
}
export class Pool extends CorePool {
constructor(url: string | URL, options?: Pool.Options) {
super(url, options, profile);
}
__createClient(origin: URL, options: ClientOptions): Dispatcher {
return new Client(origin, options);
}
}
export declare namespace Pool {
type Options = PoolOptions;
type PoolStats = CorePoolStats;
}
export class Agent extends CoreAgent {
constructor(options?: Agent.Options) {
super(options, profile);
}
__createClient(origin: string | URL, options: PoolOptions): Dispatcher {
return new Client(origin, options);
}
__createPool(origin: string | URL, options: PoolOptions): Dispatcher {
return new Pool(origin, options);
}
}
export declare namespace Agent {
type Options = AgentOptions;
type DispatchOptions = import("./core/types.ts").DispatchOptions;
}
export function buildConnector(options?: buildConnector.BuildOptions): buildConnector.connector {
return coreBuildConnector(options ?? {});
}
export declare namespace buildConnector {
type BuildOptions = ConnectorBuildOptions;
type Options = ConnectorOptions;
type Callback = ConnectorCallback;
type connector = Connector;
}
export function setGlobalDispatcher(dispatcher: Dispatcher): void {
coreSetGlobalDispatcher(dispatcher);
}
export function getGlobalDispatcher(): Dispatcher {
return coreGetGlobalDispatcher(() => new Agent());
}
export function request(
url: string | URL,
options?: TopLevelRequestOptions,
): Promise<ResponseData> {
return coreRequest(url, options, () => new Agent(), profile);
}
export function fetch(input: string | URL, init?: UndiciRequestInit): Promise<Response> {
return undiciFetch(input, init, () => new Agent(), profile);
}
// undici's Headers is the WHATWG class the runtime implements natively.
export import Headers = globalThis.Headers;
export type { DispatchOptions };
packages/compiler/shims/undici/v7.ts
+119
View File
@@ -0,0 +1,119 @@
// scriptc's built-in implementation of the `undici` package, 7.x API.
//
// Programs whose installed undici is 7.x compile this module in its place
// (see packages/compiler/src/frontend/package-shims.ts). It implements the
// HTTP/1.1 client subset over node:net / node:tls: Dispatcher, Client, Pool,
// Agent, request(), buildConnector, errors, the global dispatcher, fetch and
// the diagnostics_channel publications. Anything else undici exports is not
// declared here, so using it is a compile-time error.
import { Agent as CoreAgent } from "./core/agent.ts";
import {
getGlobalDispatcher as coreGetGlobalDispatcher,
request as coreRequest,
setGlobalDispatcher as coreSetGlobalDispatcher,
} from "./core/api.ts";
import type { TopLevelRequestOptions } from "./core/api.ts";
import { Client as CoreClient } from "./core/client.ts";
import { buildConnector as coreBuildConnector } from "./core/connector.ts";
import { Dispatcher } from "./core/dispatcher.ts";
import type { ResponseData } from "./core/dispatcher.ts";
import { undiciFetch } from "./core/fetch.ts";
import type { UndiciRequestInit } from "./core/fetch.ts";
import { Pool as CorePool, PoolStats as CorePoolStats } from "./core/pool.ts";
import type {
AgentOptions,
ClientOptions,
Connector,
ConnectorBuildOptions,
ConnectorCallback,
ConnectorOptions,
DispatchOptions,
PoolOptions,
Profile,
} from "./core/types.ts";
import type { SocketInfo as CoreSocketInfo } from "./core/errors.ts";
const profile: Profile = { major: 7 };
export { Dispatcher };
export type { DiagnosticsChannel } from "./core/diagnostics-types.ts";
export * as errors from "./core/errors-v7.ts";
export class Client extends CoreClient {
constructor(url: string | URL, options?: Client.Options) {
super(url, options, profile);
}
}
export declare namespace Client {
type Options = ClientOptions;
type SocketInfo = CoreSocketInfo;
}
export class Pool extends CorePool {
constructor(url: string | URL, options?: Pool.Options) {
super(url, options, profile);
}
__createClient(origin: URL, options: ClientOptions): Dispatcher {
return new Client(origin, options);
}
}
export declare namespace Pool {
type Options = PoolOptions;
type PoolStats = CorePoolStats;
}
export class Agent extends CoreAgent {
constructor(options?: Agent.Options) {
super(options, profile);
}
__createClient(origin: string | URL, options: PoolOptions): Dispatcher {
return new Client(origin, options);
}
__createPool(origin: string | URL, options: PoolOptions): Dispatcher {
return new Pool(origin, options);
}
}
export declare namespace Agent {
type Options = AgentOptions;
type DispatchOptions = import("./core/types.ts").DispatchOptions;
}
export function buildConnector(options?: buildConnector.BuildOptions): buildConnector.connector {
return coreBuildConnector(options ?? {});
}
export declare namespace buildConnector {
type BuildOptions = ConnectorBuildOptions;
type Options = ConnectorOptions;
type Callback = ConnectorCallback;
type connector = Connector;
}
export function setGlobalDispatcher(dispatcher: Dispatcher): void {
coreSetGlobalDispatcher(dispatcher);
}
export function getGlobalDispatcher(): Dispatcher {
return coreGetGlobalDispatcher(() => new Agent());
}
export function request(
url: string | URL,
options?: TopLevelRequestOptions,
): Promise<ResponseData> {
return coreRequest(url, options, () => new Agent(), profile);
}
export function fetch(input: string | URL, init?: UndiciRequestInit): Promise<Response> {
return undiciFetch(input, init, () => new Agent(), profile);
}
// undici's Headers is the WHATWG class the runtime implements natively.
export import Headers = globalThis.Headers;
export type { DispatchOptions };
packages/compiler/src/backend/executable-features.ts
+5
View File
@@ -29,12 +29,15 @@ export function executableLinkFeatures(mod: IrModule): NativeLinkFeatures {
...(features.streamDyn ? { streamDyn: true } : {}),
...(features.dynForAwait ? { dynForAwait: true } : {}),
...(features.eventsOnceSignal ? { eventsOnceSignal: true } : {}),
...(features.dynBufferNum ? { dynBufferNum: true } : {}),
...(features.builtinValues ? { builtinValues: true } : {}),
events: features.processEvents,
emitter: features.emitter,
symbol: features.symbol,
bigint: features.bigint,
searchParams: features.searchParams,
qs: features.qs,
utilPromisify: features.utilPromisify,
dynBuiltinClass: features.dynBuiltinClass,
dynBuiltinStream: features.dynBuiltinStream,
parseArgs: features.parseArgs,
@@ -44,12 +47,14 @@ export function executableLinkFeatures(mod: IrModule): NativeLinkFeatures {
http2: features.http2,
dgram: features.dgram,
dnsPromises: features.dnsPromises,
dgramDyn: features.dgramDyn,
watch: features.fsWatch,
foreignFfi: hasForeignFfiCallback(mod.ffiImports ?? []),
nodeTest: features.nodeTest,
tls: features.tls,
tlsCa: features.tlsCa,
cipher: features.cipher,
ecdh: features.ecdh,
webcrypto: features.webcrypto,
perfLoop: features.perfLoop,
};
packages/compiler/src/backend/library-pack.ts
+3
View File
@@ -42,6 +42,7 @@ export async function compilePackedLibrary(
bigint: options.bigint ?? false,
searchParams: options.searchParams ?? false,
qs: false,
utilPromisify: false,
dynBuiltinClass: false,
dynBuiltinStream: false,
parseArgs: false,
@@ -51,11 +52,13 @@ export async function compilePackedLibrary(
http2: false,
dgram: false,
dnsPromises: false,
dgramDyn: false,
watch: false,
foreignFfi: false,
nodeTest: false,
tls: false,
cipher: false,
ecdh: false,
webcrypto: false,
perfLoop: false,
tlsCa: false,
packages/compiler/src/backend/llvm/array-search.ts
+3 -1
View File
@@ -63,7 +63,9 @@ export function emitArraySearchHelpers(
`define internal double @${arraySearchHelperName(kind)}(ptr %a, ptr %v) ${FN_ATTRS} {`,
`entry:`,
` %lenp = getelementptr inbounds %ScrArr, ptr %a, i32 0, i32 1`,
` %len = load ${sz}, ptr %lenp${sz === "i64" ? ", !range !{i64 0, i64 4294967296}" : ""}`,
...(sz === "i64"
? [` %len32 = load i32, ptr %lenp`, ` %len = zext i32 %len32 to i64`]
: [` %len = load i32, ptr %lenp`]),
` %fp = getelementptr inbounds %ScrArr, ptr %a, i32 0, i32 6`,
` %f32 = load i32, ptr %fp`,
widen,
packages/compiler/src/backend/llvm/classes.test.ts
+21 -9
View File
@@ -75,17 +75,25 @@ test("releases split into an inlinable fast path and an out-of-line remainder",
const llvm = emitLlvmModule(mod, { pointerBits: 64 });
const body = (name: string): string =>
new RegExp(`^define internal void @${name}(?:\\.i32)?\\([^]*?^}`, "m").exec(llvm)?.[0] ?? "";
// A hierarchy rooted in the program: a surviving release decrements and
// takes the possible-root step without loading the vtable.
// A hierarchy rooted in the program: a surviving release decrements the
// count (the low half of the rc word, which keeps the class id in its high
// half) and takes the possible-root step without loading the vtable.
const fast = body(mangleClassRelease("Node"));
expect(fast).toContain("%shared = icmp ult i64 %rcm2, -3");
expect(fast).toContain("%shared = icmp ult i32 %rcm2, -3");
expect(fast).toContain("load i32, ptr %bufp");
expect(fast).toContain(`call void @${mangleClassRelease("Node")}_slow(ptr %o)`);
expect(fast).not.toContain("%ScrVt");
// Only the last reference dispatches, from a noinline remainder.
const slow = body(`${mangleClassRelease("Node")}_slow`);
expect(slow).toMatch(/noinline/);
// The class id indexes the hierarchy's vtable array.
expect(slow).toContain("%vt.id = load i32, ptr %vt.idp");
expect(slow).toContain("ptr @sc_vta_Node, i64 0, i64 %vt.ix64");
expect(slow).toContain("getelementptr inbounds %ScrVt, ptr %vt, i64 0, i32 2");
// 32-bit targets keep the vtable word.
const wasm = emitLlvmModule(mod, { pointerBits: 32 });
expect(wasm).toContain("; class Node (vt at 1)");
expect(wasm).not.toContain("@sc_vta_Node");
});
test("ProgramLifetime hierarchies are never freed and take no reference counts", () => {
@@ -121,9 +129,11 @@ test("ProgramLifetime hierarchies are never freed and take no reference counts",
expect(body(mangleClassRetain(cls)), cls).toMatch(/entry:\n {2}ret ptr %o\n}/);
expect(body(mangleClassRelease(cls)), cls).toMatch(/entry:\n {2}ret void\n}/);
// new: no collector header (no descriptor, no scr_cyc_alloc) and the
// program-lifetime count 2^63, not SIZE_MAX (which means published).
// program-lifetime count 2^31 under the class id (Node's preorder number
// 0 + 1, Leaf's 1 + 1), not SIZE_MAX (which means published).
const nw = body(mangleClassNew(cls));
expect(nw, cls).toContain("store i64 -9223372036854775808, ptr %o");
const id = cls === "Node" ? 1 : 2;
expect(nw, cls).toContain(`store i64 ${id * 2 ** 32 + 2 ** 31}, ptr %o.vt`);
expect(nw, cls).not.toContain("store i64 -1, ptr %o");
expect(nw, cls).not.toContain("scr_cyc_alloc");
expect(llvm, cls).not.toContain(`@${mangleClassTrace(cls)}.cyctype`);
@@ -206,13 +216,15 @@ test("published ProgramLifetime instances use the published mark every other obj
// a refusal restores the program-lifetime count); untraced, so it never
// leaves a collector buffer.
expect(llvm).toContain("call void @scr_publish(ptr");
expect(llvm).toMatch(/@scr_pub_mark\(ptr %ctx, ptr %o, i1 false, i32 4\)/);
// The frozen-object guard of the store tests the count against SIZE_MAX:
// an unpublished instance (2^63) stays writable.
// Kind 4 (an object) | 0x100 (the rc word keeps a class id: the mark
// writes only the count).
expect(llvm).toMatch(/@scr_pub_mark\(ptr %ctx, ptr %o, i1 false, i32 260\)/);
// The frozen-object guard of the store tests the count against UINT32_MAX:
// an unpublished instance (2^31) stays writable.
const store =
llvm.split(/^define /m).find((fn) => fn.includes("call void @scr_throw_published_field(")) ??
"";
expect(store).toMatch(/icmp eq i64 %\S+, -1/);
expect(store).toMatch(/icmp eq i32 %\S+, -1/);
expect(store).toContain("call void @scr_publish(ptr");
// 32-bit targets use 2^31.
const llvm32 = emitLlvmModule(mod, { pointerBits: 32 });
packages/compiler/src/backend/llvm/classes.ts
+151 -27
View File
@@ -26,6 +26,7 @@ import {
mangleCtorThunk,
mangleFunction,
mangleGlobal,
mangleVtArray,
mangleVtInstance,
mangleVtStruct,
} from "../mangle.js";
@@ -81,6 +82,12 @@ export class LlClassMeta {
* member's struct carries the i32 slot word before its fields
* (classSlotWordIndex). */
threadSlot = false;
/** A member of a program-rooted hierarchy on a 64-bit target: instead of
* a vtable word, the rc word holds the class id (classIdRcWord) in its
* high 32 bits (CLASS_ID_OFFSET) and the reference count in its low 32
* bits, and the hierarchy's vtables are one array indexed by the id
* (mangleVtArray). */
idInRc = false;
constructor(readonly def: IrClassDef) {
this.root = this;
@@ -96,6 +103,7 @@ export class LlClassMeta {
export function buildClassGraph(
mod: IrModule,
fnByName: Map<string, IrFunction>,
sizeType: "i32" | "i64" = "i64",
): Map<string, LlClassMeta> {
const metaMap = new Map<string, LlClassMeta>();
for (const cls of mod.classes ?? []) {
@@ -122,6 +130,11 @@ export function buildClassGraph(
if (meta.base === null) number(meta, meta);
meta.hierarchy = meta.base !== null || meta.children.length > 0;
}
// The runtime reads the vtable word of runtime-rooted hierarchies (Error,
// EventEmitter, EventTarget, Event and the streams) and 32-bit targets
// have no spare half in the rc word: those keep the vtable word.
for (const meta of metaMap.values())
meta.idInRc = sizeType === "i64" && meta.hierarchy && meta.root.def.runtime !== true;
const slotRoots = new Set<LlClassMeta>();
for (const key of mod.threadSlotKeys ?? []) {
const meta = metaMap.get(key);
@@ -336,11 +349,12 @@ function runtimePrefixOf(meta: LlClassMeta): RuntimePrefix | null {
}
/** The GEP index after a class's header words: rc at 0, the vtable word
* at 1 on hierarchy members, then a runtime prefix (state + display name)
* at 1 on hierarchy members that do not keep their class id in the rc
* word (LlClassMeta.idInRc), then a runtime prefix (state + display name)
* on classes rooted at a runtime base, then the stream-state slot on
* stream-rooted ones. */
function headerEnd(meta: LlClassMeta): number {
if (!meta.hierarchy) return 1;
if (!meta.hierarchy || meta.idInRc) return 1;
if (streamRooted(meta)) return 5;
return runtimePrefixOf(meta) ? 4 : 2;
}
@@ -445,6 +459,69 @@ export function programLifetimeRc(sizeType: "i32" | "i64"): string {
return sizeType === "i32" ? "-2147483648" : "-9223372036854775808";
}
/** Lines that load the preorder number of the dynamic class of `obj`, an
* instance of `meta`'s hierarchy whose struct is `struct`, into `pre` (a
* `sizeType`): from the high half of the rc word (LlClassMeta.idInRc), or
* from the vtable its vtable word points at. */
export function classPreLines(
meta: LlClassMeta,
struct: string,
obj: string,
pre: string,
sizeType: "i32" | "i64",
): string[] {
if (meta.idInRc)
return [
` ${pre}.idp = getelementptr inbounds i8, ptr ${obj}, i64 ${CLASS_ID_OFFSET}`,
` ${pre}.id = load i32, ptr ${pre}.idp`,
` ${pre}.n = add i32 ${pre}.id, -1 ; the class id is the preorder number + 1`,
` ${pre} = zext i32 ${pre}.n to ${sizeType}`,
];
return [
` ${pre}.vtp = getelementptr inbounds %${struct}, ptr ${obj}, i64 0, i32 1`,
` ${pre}.vt = load ptr, ptr ${pre}.vtp`,
` ${pre}.prep = getelementptr inbounds %ScrVt, ptr ${pre}.vt, i64 0, i32 0`,
` ${pre} = load ${sizeType}, ptr ${pre}.prep`,
];
}
/** Lines that load the class id of the LlClassMeta.idInRc instance `obj`
* (`group` adds !invariant.group to the load) and point `vt` at its class's
* element of the hierarchy's vtable array. */
export function classIdVtLines(
meta: LlClassMeta,
obj: string,
vt: string,
group: string,
): string[] {
const t = `${vt}.`;
const root = meta.root;
const vtt = `%${mangleVtStruct(root.def.name)}`;
const n = root.post - root.pre + 1;
return [
` ${t}idp = getelementptr inbounds i8, ptr ${obj}, i64 ${CLASS_ID_OFFSET}`,
` ${t}id = load i32, ptr ${t}idp${group}`,
` ${t}ix = sub i32 ${t}id, ${root.pre + 1}`,
` ${t}ix64 = zext i32 ${t}ix to i64`,
` ${vt} = getelementptr inbounds [${n} x ${vtt}], ptr @${mangleVtArray(root.def.name)}, i64 0, i64 ${t}ix64`,
];
}
/** The byte offset of the class id in the rc word of an instance that
* keeps it there (LlClassMeta.idInRc): the high half of the little-endian
* 64-bit word. The count is the i32 at offset 0. */
export const CLASS_ID_OFFSET = 4;
/** The rc word an LlClassMeta.idInRc instance is created with: the class
* id, its preorder number plus 1, in the high 32 bits, and in the low 32 bits
* a count of 1, or for a program-lifetime instance the midpoint of the 32-bit
* count (as programLifetimeRc is of the 32-bit word). The id is never 0,
* 0x80000000 or 0xffffffff, the high halves of other objects' rc words, so
* the runtime can tell an object with a class id (scr_rc.h scr_rc_class_id). */
export function classIdRcWord(meta: LlClassMeta, lifetime: boolean): string {
return (BigInt(meta.pre + 1) * 2n ** 32n + (lifetime ? 2n ** 31n : 1n)).toString();
}
function undefFieldInits(
host: ClassHost,
meta: LlClassMeta,
@@ -524,17 +601,18 @@ export function emitClassShapes(
// ScrEmitter* / ScrStream* / ScrEventTargetObj* / ScrEventObj* are the
// usual pointer reinterprets.
const runtimePrefix = runtimePrefixOf(meta);
const prefix = meta.hierarchy
? streamRooted(meta)
? ["ptr", "ptr", "ptr", "ptr"]
: runtimePrefix
? ["ptr", "ptr", "ptr"]
: ["ptr"]
: [];
const prefix =
meta.hierarchy && !meta.idInRc
? streamRooted(meta)
? ["ptr", "ptr", "ptr", "ptr"]
: runtimePrefix
? ["ptr", "ptr", "ptr"]
: ["ptr"]
: [];
const members = [...prefix, ...(meta.threadSlot ? ["i32"] : []), ...fieldTys];
typeDefs.push(
`%${mangleClassStruct(cls.name)} = type { ${host.sizeType}${members.length ? ", " + members.join(", ") : ""} } ` +
`; class ${cls.name}${meta.hierarchy ? " (vt at 1)" : ""}${streamRooted(meta) ? " (ScrStream prefix at 2)" : runtimePrefix ? ` (${runtimePrefix.struct} prefix at 2)` : ""} { ${cls.fields.map((f) => llvmCommentText(f.name)).join("; ")} }`,
`; class ${cls.name}${meta.idInRc ? " (class id in rc)" : meta.hierarchy ? " (vt at 1)" : ""}${streamRooted(meta) ? " (ScrStream prefix at 2)" : runtimePrefix ? ` (${runtimePrefix.struct} prefix at 2)` : ""} { ${cls.fields.map((f) => llvmCommentText(f.name)).join("; ")} }`,
);
}
@@ -559,18 +637,40 @@ export function emitClassShapes(
// One constant vtable instance per emitted hierarchy class: interval,
// direct release, and the class's dispatch entry for every slot — the
// method functions THEMSELVES (no adapters; see the header comment).
for (const cls of emitted) {
const meta = metaMap.get(cls.name)!;
if (!meta.hierarchy) continue;
const vtInit = (meta: LlClassMeta): string => {
const entries = vtEntriesFor(meta).map(({ slot, impl }) =>
impl === null
? `ptr null` // outside the declaring subtree / fully-abstract chain
: `ptr @${host.virtualEntry?.(`%${impl.def.name}.${slot.method}`) ?? mangleFunction(`%${impl.def.name}.${slot.method}`)}`,
);
const head = `%ScrVt { ${host.sizeType} ${meta.pre}, ${host.sizeType} ${meta.post}, ptr @${mangleClassReleaseDirect(cls.name)} }`;
const head = `%ScrVt { ${host.sizeType} ${meta.pre}, ${host.sizeType} ${meta.post}, ptr @${mangleClassReleaseDirect(meta.def.name)} }`;
return `{ ${[head, ...entries].join(", ")} }`;
};
for (const cls of emitted) {
const meta = metaMap.get(cls.name)!;
if (!meta.hierarchy || meta.idInRc) continue;
defs.push(
`@${mangleVtInstance(cls.name)} = internal constant %${mangleVtStruct(meta.root.def.name)} ` +
`{ ${[head, ...entries].join(", ")} } ; class ${cls.name}`,
`${vtInit(meta)} ; class ${cls.name}`,
``,
);
}
// A hierarchy whose instances keep their class id in the rc word has
// one array of its classes' vtables, in preorder: the vtable of an
// instance is the element at its number minus the root's.
for (const name of roots) {
const root = metaMap.get(name)!;
if (!root.idInRc) continue;
const members = [...metaMap.values()]
.filter((m) => m.root === root)
.sort((a, b) => a.pre - b.pre);
const vtt = `%${mangleVtStruct(root.def.name)}`;
defs.push(
`@${mangleVtArray(root.def.name)} = internal constant [${members.length} x ${vtt}] [`,
...members.map(
(m, i) => ` ${vtt} ${vtInit(m)}${i + 1 < members.length ? "," : ""} ; class ${m.def.name}`,
),
`]`,
``,
);
}
@@ -605,6 +705,7 @@ export function emitClassShapes(
release: `@${release}`,
destroy: `${teardown}_destroy`,
traced: host.tracedShapes.has(`object:${cls.name}`),
...(meta.idInRc ? { count: "i32" as const } : {}),
},
]);
}
@@ -658,7 +759,13 @@ export function emitClassShapes(
// retain: NULL-tolerant, immortal-skip — layout-generic (rc at 0), so
// hierarchy members need no dispatch.
defs.push(...retainBody(host, mangleClassRetain(cls.name), `retain ${cls.name}`, lifetime), ``);
// An instance that keeps its class id in the rc word counts in the
// low half (CLASS_ID_OFFSET).
const count = meta.idInRc ? "i32" : host.sizeType;
defs.push(
...retainBody(host, mangleClassRetain(cls.name), `retain ${cls.name}`, lifetime, count),
``,
);
// An out-of-line teardown tears typed children down directly
// (directChildReleaseLines). It takes the chain length when it has such
@@ -714,6 +821,7 @@ export function emitClassShapes(
traced,
`release ${cls.name}`,
lifetime,
count,
),
``,
);
@@ -723,12 +831,16 @@ export function emitClassShapes(
` %isnull = icmp eq ptr %o, null`,
` br i1 %isnull, label %done, label %check`,
`check:`,
` %rc = load ${host.sizeType}, ptr %o`,
` %imm = icmp eq ${host.sizeType} %rc, -1`,
` %rc = load ${count}, ptr %o`,
` %imm = icmp eq ${count} %rc, -1`,
` br i1 %imm, label %done, label %disp`,
`disp:`,
` %vtp = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 1`,
` %vt = load ptr, ptr %vtp`,
...(meta.idInRc
? classIdVtLines(meta, "%o", "%vt", "")
: [
` %vtp = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 1`,
` %vt = load ptr, ptr %vtp`,
]),
` %relp = getelementptr inbounds %ScrVt, ptr %vt, i64 0, i32 2`,
` %rel = load ptr, ptr %relp`,
` call void %rel(ptr %o) ; the DYNAMIC class's teardown`,
@@ -743,10 +855,10 @@ export function emitClassShapes(
const reld: string[] = [
`define internal void @${mangleClassReleaseDirect(cls.name)}(ptr %o) ${FN_ATTRS} { ; direct release ${cls.name}`,
`entry:`,
` %rc = load ${host.sizeType}, ptr %o`,
` %n = sub ${host.sizeType} %rc, 1`,
` store ${host.sizeType} %n, ptr %o`,
` %dead = icmp eq ${host.sizeType} %n, 0`,
` %rc = load ${count}, ptr %o`,
` %n = sub ${count} %rc, 1`,
` store ${count} %n, ptr %o`,
` %dead = icmp eq ${count} %n, 0`,
` br i1 %dead, label %free, label %${traced ? "root" : "done"}`,
`free:`,
];
@@ -815,10 +927,22 @@ export function emitClassShapes(
host,
sizeOf,
traced && !lifetime ? { type: cycType } : null,
meta.hierarchy ? 16 : 8,
meta.hierarchy && !meta.idInRc ? 16 : 8,
),
);
nw.push(` store ${host.sizeType} ${lifetime ? programLifetimeRc(host.sizeType) : 1}, ptr %o`);
if (meta.idInRc) {
// The class id in the high half of the rc word, the count in the low
// half (classIdRcWord). It never changes while the object lives:
// retains, releases and the runtime write only the low half.
host.declare(`declare ptr @llvm.launder.invariant.group.p0(ptr)`);
nw.push(
` %o.vt = call ptr @llvm.launder.invariant.group.p0(ptr %o)`,
` store i64 ${classIdRcWord(meta, lifetime)}, ptr %o.vt ; class ${llvmCommentText(cls.name)} is ${meta.pre}`,
);
} else
nw.push(
` store ${host.sizeType} ${lifetime ? programLifetimeRc(host.sizeType) : 1}, ptr %o`,
);
if (lifetime && audit) {
// --sanitize builds: the RC audit skips its leak report when the
// program keeps program-lifetime objects (scr_console.c), and
@@ -826,7 +950,7 @@ export function emitClassShapes(
host.declare(`declare void @scr_obj_lifetime_note(ptr)`);
nw.push(` call void @scr_obj_lifetime_note(ptr %o)`);
}
if (meta.hierarchy) {
if (meta.hierarchy && !meta.idInRc) {
// The vtable word is written once, here, and never changes while the
// object lives: its store and the virtual-call loads carry
// !invariant.group. A recycled block may have held an object of
packages/compiler/src/backend/llvm/common.ts
+87 -3
View File
@@ -10,7 +10,9 @@ import type {
import type { LlvmEmitterContext } from "./expr-context.js";
/** Field numbers of %ScrArr, which mirrors scr_runtime.h's ScrArr field for
* field (its static assertions pin `cyc`, `filled` and `data`). */
* field (its static assertions pin `len`, `cap`, `cyc`, `filled` and
* `data`). `len` and `cap` are i32 on every target; the dense states end
* the allocation after the `cap` value slots at `data` (scrArrStates). */
export const SCR_ARR_FIELD = {
rc: 0,
len: 1,
@@ -21,10 +23,92 @@ export const SCR_ARR_FIELD = {
filled: 6,
ops: 7,
data: 8,
present: 9,
ext: 10,
ext: 9,
} as const;
/** Loads the i32 `len` or `cap` of a %ScrArr at `ptr` into `dest` as a
* size-type count. */
export function loadArrCount(
host: Pick<LlvmEmitterContext, "B" | "sizeType">,
dest: string,
ptr: string,
attach = "",
): void {
const B = host.B;
if (host.sizeType === "i32") {
B.line(`${dest} = load i32, ptr ${ptr}${attach}`);
return;
}
const raw = B.tmp();
B.line(`${raw} = load i32, ptr ${ptr}${attach}`);
B.line(`${dest} = zext i32 ${raw} to ${host.sizeType}`);
}
/** Stores the size-type count `value` (at most 2^32 - 1) into the i32 `len`
* or `cap` of a %ScrArr at `ptr`. */
export function storeArrCount(
host: Pick<LlvmEmitterContext, "B" | "sizeType">,
value: string,
ptr: string,
attach = "",
): void {
const B = host.B;
let narrow = value;
if (host.sizeType !== "i32") {
narrow = B.tmp();
B.line(`${narrow} = trunc ${host.sizeType} ${value} to i32`);
}
B.line(`store i32 ${narrow}, ptr ${ptr}${attach}`);
}
/** The end of the dense state bytes of a %ScrArr into `dest`
* (scr_arr_states_end): `cap` value slots of `width` bytes at `data`, then
* one state byte per slot in reverse slot order. `cap` is a size-type count. */
export function scrArrStates(
host: Pick<LlvmEmitterContext, "B" | "sizeType">,
dest: string,
data: string,
cap: string,
width: number,
): void {
host.B.line(
`${dest} = getelementptr inbounds [${width + 1} x i8], ptr ${data}, ${host.sizeType} ${cap}`,
);
}
/** The state byte of dense slot `offset` into `dest`, given the end of the
* states (scrArrStates): `end - 1 - offset`. */
export function scrArrStatePtr(
host: Pick<LlvmEmitterContext, "B" | "sizeType">,
dest: string,
end: string,
offset: string,
): void {
const B = host.B;
const back = B.tmp();
B.line(`${back} = xor ${host.sizeType} ${offset}, -1`);
B.line(`${dest} = getelementptr inbounds i8, ptr ${end}, ${host.sizeType} ${back}`);
}
/** The state byte of dense slot `offset` of a %ScrArr into `dest`, from its
* `data` and size-type `cap`: scrArrStates and scrArrStatePtr in one
* address computation, for paths that read a single state. */
export function scrArrStateAt(
host: Pick<LlvmEmitterContext, "B" | "sizeType">,
dest: string,
data: string,
cap: string,
width: number,
offset: string,
): void {
const B = host.B;
const back = B.tmp();
B.line(`${back} = xor ${host.sizeType} ${offset}, -1`);
B.line(
`${dest} = getelementptr inbounds [${width + 1} x i8], ptr ${data}, ${host.sizeType} ${cap}, ${host.sizeType} ${back}`,
);
}
/** Lines that decide whether the double `index` is an integer in
* `[0, length)`, where `length` is an unsigned `sizeType` count, and give
* that integer as an i64. `%{ok}` is the answer; `%{wide}` is meaningful
packages/compiler/src/backend/llvm/dense-array-access.test.ts
+4
View File
@@ -170,6 +170,10 @@ test.each([32, 64] as const)(
expect(fn.match(new RegExp(`icmp ult ${size} `, "g"))).toHaveLength(3);
expect(fn).toMatch(/store double 0x4004000000000000, ptr /);
expect(fn).toMatch(/zext i1 .* to i8/);
// Per push: data + (cap - len), then the state byte `width * cap - 1`
// past it; plus the boolean value slot.
expect(fn.match(/getelementptr inbounds \[8 x i8\], ptr .* -1$/gm)).toHaveLength(2);
expect(fn.match(/getelementptr inbounds \[1 x i8\], ptr .* -1$/gm)).toHaveLength(1);
expect(fn.match(/getelementptr inbounds i8, ptr /g)).toHaveLength(4);
expect(fn.match(/store i8 1, ptr /g)).toHaveLength(3);
expect(fn.match(new RegExp(`add nuw ${size} .*, 1$`, "gm"))).toHaveLength(3);
packages/compiler/src/backend/llvm/dense-array-access.ts
+102 -42
View File
@@ -17,7 +17,15 @@
* whose writes extend the bound when they continue it. */
import { isRefCounted, type IrExpr, type IrType } from "../../ir/ir.js";
import { f64Lit, SCR_ARR_FIELD, sizeToDouble } from "./common.js";
import {
f64Lit,
loadArrCount,
SCR_ARR_FIELD,
scrArrStatePtr,
scrArrStates,
sizeToDouble,
storeArrCount,
} from "./common.js";
import type { LlValue, LlvmEmitterContext } from "./expr-context.js";
import { exactInteger, widenInteger } from "./integer-values.js";
@@ -36,7 +44,8 @@ interface DenseGuard {
len: string;
/** The hole-free prefix bound loaded by the guard (i32). */
filled: string;
/** Dense value and state storage base pointers. */
/** The dense value storage base pointer, and the end of the states
* (scrArrStates). */
data: string;
states: string;
}
@@ -48,7 +57,9 @@ interface DenseGuard {
* below the hole-free prefix bound branches to `filledLabel` first; both
* that block and the returned (state-checked) dense block may use the
* guard's values. A `proven` index is known to lie below the bound (a dense
* array loop's fast copy, dense-array-loops.ts) and always takes it. */
* array loop's fast copy, dense-array-loops.ts) and always takes it. With
* `prefixOnly`, every other index branches to `slow` and no block is left
* open: only `offset` and `data` are meaningful. */
function emitDenseGuard(
host: LlvmEmitterContext,
array: string,
@@ -57,7 +68,9 @@ function emitDenseGuard(
slow: string,
requireBelowLen: boolean,
filledLabel: string,
width: number,
proven = false,
prefixOnly = false,
): DenseGuard {
const B = host.B;
const size = host.sizeType;
@@ -89,7 +102,6 @@ function emitDenseGuard(
cap = B.tmp(),
dataPtr = B.tmp(),
data = B.tmp(),
statesPtr = B.tmp(),
states = B.tmp();
B.line(
`${dataPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 ${SCR_ARR_FIELD.data}`,
@@ -97,26 +109,23 @@ function emitDenseGuard(
host.markMemoryPointer(dataPtr, "array:header");
B.line(`${data} = load ptr, ptr ${dataPtr}${host.fieldAliasAttachment(dataPtr)}`);
const loadChecked = (): void => {
B.line(`${lenPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 1`);
B.line(
`${lenPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 ${SCR_ARR_FIELD.len}`,
);
host.markMemoryPointer(lenPtr, "array:header");
loadArrCount(host, len, lenPtr, host.fieldAliasAttachment(lenPtr));
B.line(
`${len} = load ${size}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}${host.arrayLengthRange}`,
`${capPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 ${SCR_ARR_FIELD.cap}`,
);
B.line(`${capPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 2`);
host.markMemoryPointer(capPtr, "array:header");
B.line(`${cap} = load ${size}, ptr ${capPtr}${host.fieldAliasAttachment(capPtr)}`);
B.line(
`${statesPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 ${SCR_ARR_FIELD.present}`,
);
host.markMemoryPointer(statesPtr, "array:header");
B.line(`${states} = load ptr, ptr ${statesPtr}${host.fieldAliasAttachment(statesPtr)}`);
loadArrCount(host, cap, capPtr, host.fieldAliasAttachment(capPtr));
};
// Reads load the storage bases before branching: the header of a live
// array is always readable, and unconditional loads let LLVM hoist them
// out of loops whose only calls are read-only fallbacks. Writes load the
// fields that only the state-checked path needs inside it, which keeps
// them off a hot loop of prefix stores (that path stores to the header).
const eagerHeader = requireBelowLen;
const eagerHeader = requireBelowLen && !prefixOnly;
if (eagerHeader) loadChecked();
const filledPtr = B.tmp(),
filled = B.tmp(),
@@ -148,6 +157,10 @@ function emitDenseGuard(
B.line(`${t} = and i1 ${integral}, ${inFilled}`);
inFilled = t;
}
if (prefixOnly) {
B.condBr(inFilled, filledLabel, slow);
return { offset, len, filled, data, states };
}
const check = B.newLabel("arr.check");
B.condBr(inFilled, filledLabel, check);
B.startBlock(check);
@@ -171,6 +184,10 @@ function emitDenseGuard(
const dense = B.newLabel("arr.dense");
B.condBr(ok, dense, slow);
B.startBlock(dense);
// Only the state-checked block needs the states: the address is pure
// arithmetic on loaded fields, so computing it here keeps it off the
// prefix path.
scrArrStates(host, states, data, cap, width);
return { offset, len, filled, data, states };
}
@@ -186,11 +203,8 @@ function statePointer(
host: LlvmEmitterContext,
guard: Pick<DenseGuard, "offset" | "states">,
): string {
const B = host.B;
const statePtr = B.tmp();
B.line(
`${statePtr} = getelementptr inbounds i8, ptr ${guard.states}, ${host.sizeType} ${guard.offset}`,
);
const statePtr = host.B.tmp();
scrArrStatePtr(host, statePtr, guard.states, guard.offset);
host.markMemoryPointer(statePtr, "array:present");
return statePtr;
}
@@ -201,6 +215,11 @@ function slotType(acc: ArrayAccess): "i8" | "i64" {
return acc === "bool" ? "i8" : "i64";
}
/** Bytes per dense slot (scr_arr_slot_width). */
export function slotWidth(acc: ArrayAccess): number {
return acc === "bool" ? 1 : 8;
}
function valuePointer(
host: LlvmEmitterContext,
guard: Pick<DenseGuard, "offset" | "data">,
@@ -245,7 +264,7 @@ export function emitDenseSlotUnlink(
const unlink = B.newLabel("arr.unlink"),
swap = B.newLabel("arr.unlink.swap"),
done = B.newLabel("arr.unlink.done");
const guard = emitDenseGuard(host, array, index, indexExpr, done, true, unlink);
const guard = emitDenseGuard(host, array, index, indexExpr, done, true, unlink, 8);
// The state-checked dense block: leave holes and the tail to the store.
B.br(done);
B.startBlock(unlink);
@@ -290,11 +309,29 @@ export function emitDenseArrayGet(
const slow = B.newLabel("arr.get.slow"),
join = B.newLabel("arr.get.join"),
valueLabel = B.newLabel("arr.get.value");
const guard = emitDenseGuard(host, array, index, indexExpr, slow, true, valueLabel, proven);
const state = loadState(host, guard);
const present = B.tmp();
B.line(`${present} = icmp eq i8 ${state}, ${SCR_ARR_VALUE}`);
B.condBr(present, valueLabel, slow);
// The state of slot i is at data + cap * (width + 1) - 1 - i: an inline
// state check indexed by a loop's induction variable makes LLVM keep
// -1 - i as a second induction variable, so above the prefix the runtime
// getter reads the slot.
const induction = indexExpr?.kind === "varRef" && host.integerLoopBindings.has(indexExpr.localId);
const guard = emitDenseGuard(
host,
array,
index,
indexExpr,
slow,
true,
valueLabel,
slotWidth(acc),
proven,
induction,
);
if (!induction) {
const state = loadState(host, guard);
const present = B.tmp();
B.line(`${present} = icmp eq i8 ${state}, ${SCR_ARR_VALUE}`);
B.condBr(present, valueLabel, slow);
}
B.startBlock(valueLabel);
const ptr = valuePointer(host, guard, acc);
let fast: string;
@@ -360,6 +397,7 @@ export function emitDenseArrayState(
index: LlValue,
indexExpr: IrExpr | undefined,
kind: "state" | "has",
width: number,
): string {
const B = host.B;
const fn = kind === "state" ? "scr_arr_state" : "scr_arr_has";
@@ -368,7 +406,7 @@ export function emitDenseArrayState(
const slow = B.newLabel("arr.state.slow"),
join = B.newLabel("arr.state.join"),
filledLabel = B.newLabel("arr.state.filled");
const guard = emitDenseGuard(host, array, index, indexExpr, slow, true, filledLabel);
const guard = emitDenseGuard(host, array, index, indexExpr, slow, true, filledLabel, width);
const state = loadState(host, guard);
const fast = B.tmp();
if (kind === "state") B.line(`${fast} = uitofp i8 ${state} to double`);
@@ -421,7 +459,17 @@ export function emitDenseArraySet(
const slow = B.newLabel("arr.set.slow"),
join = B.newLabel("arr.set.join"),
filledLabel = B.newLabel("arr.set.filled");
const guard = emitDenseGuard(host, array, index, indexExpr, slow, false, filledLabel, proven);
const guard = emitDenseGuard(
host,
array,
index,
indexExpr,
slow,
false,
filledLabel,
slotWidth(acc),
proven,
);
const { offset, len } = guard;
const storeValue = (valuePtr: string): void => {
if (acc === "f64") {
@@ -482,9 +530,11 @@ export function emitDenseArraySet(
const nextLen = B.tmp(),
lenPtr = B.tmp();
B.line(`${nextLen} = add ${host.sizeType} ${offset}, 1`);
B.line(`${lenPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 1`);
B.line(
`${lenPtr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 ${SCR_ARR_FIELD.len}`,
);
host.markMemoryPointer(lenPtr, "array:header");
B.line(`store ${host.sizeType} ${nextLen}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}`);
storeArrCount(host, nextLen, lenPtr, host.fieldAliasAttachment(lenPtr));
B.br(storedLabel);
B.startBlock(storedLabel);
if (acc === "ref" && oldState && old) {
@@ -542,7 +592,7 @@ export function emitDenseArrayPop(
const lenPtr = header(SCR_ARR_FIELD.len),
len = B.tmp(),
none = B.tmp();
B.line(`${len} = load ${size}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}`);
loadArrCount(host, len, lenPtr, host.fieldAliasAttachment(lenPtr));
B.line(`${none} = icmp eq ${size} ${len}, 0`);
const check = B.newLabel("arr.pop.check"),
dense = B.newLabel("arr.pop.dense");
@@ -557,12 +607,15 @@ export function emitDenseArrayPop(
B.startBlock(dense);
const last = B.tmp();
B.line(`${last} = sub nuw ${size} ${len}, 1`);
// A packed array's last index is below its capacity (filled <= cap).
const dataPtr = header(SCR_ARR_FIELD.data),
data = B.tmp(),
statesPtr = header(SCR_ARR_FIELD.present),
capPtr = header(SCR_ARR_FIELD.cap),
cap = B.tmp(),
states = B.tmp();
B.line(`${data} = load ptr, ptr ${dataPtr}${host.fieldAliasAttachment(dataPtr)}`);
B.line(`${states} = load ptr, ptr ${statesPtr}${host.fieldAliasAttachment(statesPtr)}`);
loadArrCount(host, cap, capPtr, host.fieldAliasAttachment(capPtr));
scrArrStates(host, states, data, cap, slotWidth(acc));
const valuePtr = valuePointer(host, { offset: last, data }, acc);
const statePtr = statePointer(host, { offset: last, states });
const slotTy = slotType(acc);
@@ -581,7 +634,7 @@ export function emitDenseArrayPop(
B.line(`${last32} = trunc i64 ${last} to i32`);
}
B.line(`store i32 ${last32}, ptr ${filledPtr}${host.fieldAliasAttachment(filledPtr)}`);
B.line(`store ${size} ${last}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}`);
B.line(`store i32 ${last32}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}`);
return raw;
}
@@ -608,13 +661,13 @@ export function emitDenseArrayPush(
host.markMemoryPointer(ptr, "array:header");
return ptr;
};
const lenPtr = header(1),
const lenPtr = header(SCR_ARR_FIELD.len),
len = B.tmp(),
cap = B.tmp(),
room = B.tmp();
B.line(`${len} = load ${size}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}`);
const capPtr = header(2);
B.line(`${cap} = load ${size}, ptr ${capPtr}${host.fieldAliasAttachment(capPtr)}`);
loadArrCount(host, len, lenPtr, host.fieldAliasAttachment(lenPtr));
const capPtr = header(SCR_ARR_FIELD.cap);
loadArrCount(host, cap, capPtr, host.fieldAliasAttachment(capPtr));
B.line(`${room} = icmp ult ${size} ${len}, ${cap}`);
const fast = B.newLabel("arr.push.dense"),
slow = B.newLabel("arr.push.slow"),
@@ -623,11 +676,18 @@ export function emitDenseArrayPush(
B.startBlock(fast);
const dataPtr = header(SCR_ARR_FIELD.data),
data = B.tmp(),
statesPtr = header(SCR_ARR_FIELD.present),
states = B.tmp();
spare = B.tmp(),
near = B.tmp(),
statePtr = B.tmp();
B.line(`${data} = load ptr, ptr ${dataPtr}${host.fieldAliasAttachment(dataPtr)}`);
B.line(`${states} = load ptr, ptr ${statesPtr}${host.fieldAliasAttachment(statesPtr)}`);
const statePtr = statePointer(host, { offset: len, states });
// The state of slot `len` is data + width * cap + (cap - len) - 1. With
// `cap - len` formed first, the store addresses it from one register sum.
B.line(`${spare} = sub nuw ${size} ${cap}, ${len}`);
B.line(`${near} = getelementptr inbounds i8, ptr ${data}, ${size} ${spare}`);
B.line(
`${statePtr} = getelementptr inbounds [${slotWidth(acc)} x i8], ptr ${near}, ${size} ${cap}, ${size} -1`,
);
host.markMemoryPointer(statePtr, "array:present");
const valuePtr = valuePointer(host, { offset: len, data }, acc);
if (acc === "f64") {
B.line(`store double ${value}, ptr ${valuePtr}${host.fieldAliasAttachment(valuePtr)}`);
@@ -642,7 +702,7 @@ export function emitDenseArrayPush(
const next = B.tmp(),
fastValue = B.tmp();
B.line(`${next} = add nuw ${size} ${len}, 1`);
B.line(`store ${size} ${next}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}`);
storeArrCount(host, next, lenPtr, host.fieldAliasAttachment(lenPtr));
// An append to a hole-free array keeps it hole-free.
const filledPtr = header(SCR_ARR_FIELD.filled),
filled = B.tmp(),
packages/compiler/src/backend/llvm/dense-array-loops.ts
+3 -5
View File
@@ -22,7 +22,7 @@ import type { IntegerRange } from "../../ir/integer-ranges.js";
import { preservesArrayLength } from "../../ir/array-length-stability.js";
import { everyExprChild, everyStmtChild } from "../../ir/traverse.js";
import type { LlvmEmitterContext } from "./expr-context.js";
import { SCR_ARR_FIELD } from "./common.js";
import { loadArrCount, SCR_ARR_FIELD } from "./common.js";
export const INT32_MAX = 2147483647;
@@ -187,12 +187,10 @@ export function emitDenseLoopGuard(host: LlvmEmitterContext, loop: DenseArrayLoo
return ptr;
};
const receiver = host.emitStableReceiver(loop.length, []);
const lenPtr = header(receiver.name, 1),
const lenPtr = header(receiver.name, SCR_ARR_FIELD.len),
len = B.tmp(),
small = B.tmp();
B.line(
`${len} = load ${size}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}${host.arrayLengthRange}`,
);
loadArrCount(host, len, lenPtr, host.fieldAliasAttachment(lenPtr));
B.line(`${small} = icmp ule ${size} ${len}, ${INT32_MAX}`);
let valid = small;
for (const [id, ref] of loop.arrays) {
packages/compiler/src/backend/llvm/dyn.ts
+66 -7
View File
@@ -39,6 +39,7 @@ import {
isDirentShape,
isDynTypedRefType,
isRefCounted,
isUnitType,
typeKey,
} from "../../ir/ir.js";
import { dynDesc, streamTypedRefEligible } from "../../ir/analysis.js";
@@ -85,6 +86,7 @@ export const DYN_HANDLE_TAG_NUM: Record<string, number> = {
child: 16,
fileHandle: 17,
dir: 44,
dgramSocket: 62,
cryptoHash: 52,
cryptoHmac: 53,
};
@@ -95,6 +97,15 @@ export const DYN_HANDLE_TAG_NUM: Record<string, number> = {
const DYN_HANDLE_BOX_FN: Record<string, string> = {
cryptoHash: "scr_crypto_hash_to_dyn",
cryptoHmac: "scr_crypto_hash_to_dyn",
dgramSocket: "scr_dgram_to_dyn",
};
/** Handle kinds whose box holds a unit-owned wrapper rather than the
* handle itself (a dgram socket's box carries its untyped listeners):
* the unit's own function unwraps it, with the tag check and TypeError
* of scr_dyn_handle_unbox. */
const DYN_HANDLE_UNBOX_FN: Record<string, string> = {
dgramSocket: "scr_dgram_from_dyn",
};
export const DYN_KIND = {
@@ -697,6 +708,46 @@ export class LlDyn {
/** Emit the standard in-helper pending check: on a pending exception
* run `cleanup` and return `dummy`. */
/** The failure block of a union dynCheck whose value matched no arm. A
* value of the one container kind exactly one arm can take (an object
* for the union's only record arm, an array for its only array arm,
* beside unit and scalar arms) runs that arm's builder, which throws the
* deepest failing path and the reason there (`expected string at
* $.order.owner.id, got undefined`) instead of the union's own outer
* path. Other values, and an arm builder that unexpectedly succeeds,
* fall through to the union's message. Only the failure path changes. */
private unionArmFailure(B: BlockBuilder, def: IrUnionDef, kind: string): void {
const scalar = (arm: IrType): boolean =>
isUnitType(arm) ||
arm.kind === "f64" ||
arm.kind === "bool" ||
arm.kind === "string" ||
arm.kind === "bigint";
const containers = def.arms.filter((arm) => !scalar(arm));
const arm = containers.length === 1 ? containers[0]! : null;
const container =
arm?.kind === "array"
? DYN_KIND.ARR
: arm?.kind === "record" && !this.host.recordsById.get(arm.shapeId)?.tuple
? DYN_KIND.OBJ
: null;
if (!arm || container === null) return;
const host = this.host;
const deep = B.newLabel("dcu.deep");
const outer = B.newLabel("dcu.outer");
const same = B.tmp();
B.line(`${same} = icmp eq i32 ${kind}, ${container}`);
B.condBr(same, deep, outer);
B.startBlock(deep);
const built = B.tmp();
B.line(`${built} = call ptr @${this.dynCheckHelper(arm)}(ptr %d, ptr %path)`);
this.pendingBail(B, "dcu.deep", () => {}, "ptr null");
const rc = vAdapters(host, arm);
B.line(`call void ${rc.release}(ptr ${built})`);
B.br(outer);
B.startBlock(outer);
}
private pendingBail(B: BlockBuilder, hint: string, cleanup: () => void, dummy: string): void {
const p = B.tmp();
for (const line of this.host.pendingTestLines(p)) B.line(line.trimStart());
@@ -1631,10 +1682,10 @@ export class LlDyn {
: []),
]
: [
{ index: 1, type: host.sizeType, name: "length" },
{ index: 2, type: host.sizeType, name: "capacity" },
// The dense states follow the data slots: they move with data and capacity.
{ index: SCR_ARR_FIELD.len, type: "i32", name: "length" },
{ index: SCR_ARR_FIELD.cap, type: "i32", name: "capacity" },
{ index: SCR_ARR_FIELD.data, type: "ptr" as const, name: "data" },
{ index: SCR_ARR_FIELD.present, type: "ptr" as const, name: "presence" },
{ index: SCR_ARR_FIELD.filled, type: "i32" as const, name: "filled" },
];
members.forEach((member) => {
@@ -2291,6 +2342,7 @@ export class LlDyn {
B.line(`call void @scr_dyn_release_v(ptr ${materialized})`);
B.terminate(`ret ptr ${checked}`);
B.startBlock(lFail);
this.unionArmFailure(B, def, kind);
B.line(`call void @scr_dyn_check_fail(ptr %path, ptr ${want}, ptr %d)`);
B.terminate(`ret ptr null`);
break;
@@ -2359,6 +2411,7 @@ export class LlDyn {
);
B.terminate(`ret ptr ${checked}`);
B.startBlock(lFail);
this.unionArmFailure(B, def, kind);
B.line(`call void @scr_dyn_check_fail(ptr %path, ptr ${want}, ptr %d)`);
B.terminate(`ret ptr null`);
break;
@@ -2449,11 +2502,17 @@ export class LlDyn {
// TypeError on any other kind or tag).
const h = DYN_HANDLE_KINDS.get(t.kind);
if (h) {
host.declare(`declare ptr @scr_dyn_handle_unbox(ptr, i32, ptr, ptr)`);
const r = B.tmp();
B.line(
`${r} = call ptr @scr_dyn_handle_unbox(ptr %d, i32 ${DYN_HANDLE_TAG_NUM[t.kind]}, ptr %path, ptr ${want})`,
);
const unbox = DYN_HANDLE_UNBOX_FN[t.kind];
if (unbox) {
host.declare(`declare ptr @${unbox}(ptr, ptr, ptr)`);
B.line(`${r} = call ptr @${unbox}(ptr %d, ptr %path, ptr ${want})`);
} else {
host.declare(`declare ptr @scr_dyn_handle_unbox(ptr, i32, ptr, ptr)`);
B.line(
`${r} = call ptr @scr_dyn_handle_unbox(ptr %d, i32 ${DYN_HANDLE_TAG_NUM[t.kind]}, ptr %path, ptr ${want})`,
);
}
B.terminate(`ret ptr ${r}`);
break;
}
packages/compiler/src/backend/llvm/emit-function.ts
+5 -2
View File
@@ -27,7 +27,7 @@ import { findMapEntryFusions } from "./map-entry-fusion.js";
import { findLocalStackUnions } from "./stack-unions.js";
import { findLocalStackObjects, findReassignedStackObjects } from "./stack-objects.js";
import { findLocalUnionStorage } from "./local-union-storage.js";
import { INT32_RANGE, analyzeIntegerRanges } from "../../ir/integer-ranges.js";
import { INT32_RANGE, analyzeIntegerRangeFacts } from "../../ir/integer-ranges.js";
import { findBytesBounds } from "./bytes-bounds.js";
import { findInt32Locals } from "./int32-locals.js";
import { findIntegerViews } from "./integer-views.js";
@@ -59,6 +59,7 @@ export function emitFunction(emitter: LlEmitter, fn: IrFunction): string {
const B = new BlockBuilder();
emitter.B = B;
emitter.currentFunctionName = fn.name;
emitter.currentTier = fn.tier;
// Fiber switches restore the active exception cell before control
// returns to a synchronous frame (emitPendingCheck).
B.excCellInvariant = fn.async !== true && fn.generator === undefined;
@@ -220,7 +221,9 @@ export function emitFunction(emitter: LlEmitter, fn: IrFunction): string {
emitter.localUnionStorage = new Map();
emitter.walkBorrows = emitter.walkBorrowsByFunction.get(fn.name) ?? new Set();
const slotFacts = emitter.int32Slots.facts(fn.name);
emitter.integerRanges = analyzeIntegerRanges(numericFn, slotFacts);
const rangeFacts = analyzeIntegerRangeFacts(numericFn, slotFacts);
emitter.integerRanges = rangeFacts.ranges;
emitter.integerLoopRanges = rangeFacts.loopRanges;
emitter.int32Elements = slotFacts.elements ?? null;
emitter.bytesBounds = findBytesBounds(numericFn, emitter.integerRanges);
// Debug builds keep every number local a double for the debugger's
packages/compiler/src/backend/llvm/emit-module.ts
+29 -16
View File
@@ -1,6 +1,7 @@
/* LlEmitter's module-level output: module parts, helper definitions, library declarations and FFI callback definitions. */
import { emitWasmModules } from "./wasm-modules.js";
import { type LlEmitter, ffiCallbackDummyLl, llStrBytes } from "./emitter.js";
import { JS_IC_WORDS } from "./js-ic.js";
import {
mangleAsyncSpawn,
mangleClassObj,
@@ -436,14 +437,15 @@ export function emitModuleParts(emitter: LlEmitter): string[] {
`%ScrClosure = type { ${emitter.sizeType}, ptr, ${emitter.sizeType}, ptr, i32, i32, ptr, ptr }`,
`%ScrFfiTable = type { ptr, ${emitter.sizeType}, ${emitter.sizeType}, ptr, i8, ptr, ptr, ${emitter.sizeType}, ${emitter.sizeType}, ${emitter.sizeType}, ptr, ptr }`,
`%ScrRegex = type { ${emitter.sizeType}, ptr, ptr, ptr, double, ptr, ptr, ptr }`,
// ScrArr mirrors scr_runtime.h field-for-field: { rc, len, cap, elem,
// cyc, spare, filled, ops, data, present, ext } (field numbers in
// SCR_ARR_FIELD), where filled bounds the hole-free prefix, ops points at the element type's constant RC descriptor
// ScrArr mirrors scr_runtime.h field-for-field: { rc, len (i32), cap
// (i32), elem, cyc, spare, filled, ops, data, ext } (field numbers in
// SCR_ARR_FIELD), where the dense states follow the cap value slots at
// data, filled bounds the hole-free prefix, ops points at the element type's constant RC descriptor
// (%ScrArrOps, SCR_ELEM_REF only) and ext at the out-of-line sparse,
// property and metadata storage (NULL until first used). Live dynamic
// stream commits swap its dense, presence and out-of-line storage while
// preserving the target object's identity.
`%ScrArr = type { ${emitter.sizeType}, ${emitter.sizeType}, ${emitter.sizeType}, i8, i8, i16, i32, ptr, ptr, ptr, ptr }`,
`%ScrArr = type { ${emitter.sizeType}, i32, i32, i8, i8, i16, i32, ptr, ptr, ptr }`,
`%ScrArrOps = type { ptr, ptr, ptr }`,
// The ScrMap prefix { rc, key_kind, val_kind, traced } — the inline
// RC fast paths test the traced byte (shapes.ts).
@@ -558,6 +560,8 @@ export function emitModuleParts(emitter: LlEmitter): string[] {
out.push(
`@sc_dyn_read_ic_${site} = internal ${tl}global { ptr, ${emitter.sizeType}, ${emitter.sizeType}, ptr } zeroinitializer`,
);
for (let site = 0; site < emitter.jsIcSiteCount; site++)
out.push(`@sc_js_ic_${site} = internal ${tl}global [${JS_IC_WORDS} x i64] zeroinitializer`);
}
for (const table of emitter.constantNumericTables.values()) {
const n = table.values.length;
@@ -684,17 +688,23 @@ export function emitModuleParts(emitter: LlEmitter): string[] {
if (emitter.regexInstances.size > 0) out.push(``);
for (const code of regexMatchers.values()) out.push(...code.lines, ``);
for (const [, inst] of emitter.templateStringsInstances) {
// One immortal ScrArr per tagged-template site: a [N x ptr] data
// global of interned cooked-string literals, and the ScrArr header
// over it (rc == SIZE_MAX, len == cap == filled, SCR_ELEM_STR = 2,
// no REF entry points). Every dense slot is present; reads retain
// immortal strings — a no-op.
// One immortal ScrArr per tagged-template site: a data global of
// interned cooked-string literals followed by their states (the
// runtime finds them before data + cap * 9), and the ScrArr header over
// it (rc == SIZE_MAX, len == cap == filled, SCR_ELEM_STR = 2, no REF
// entry points). Every dense slot is present; reads retain immortal
// strings — a no-op. Slots are 8 bytes on every target: on 32-bit
// targets each holds the pointer in its low half and zero in its high
// half, so the states start where the runtime looks for them.
const n = inst.slots.length;
const present = n === 0 ? "zeroinitializer" : `[ ${inst.slots.map(() => "i8 1").join(", ")} ]`;
const wide = emitter.sizeType === "i64";
const slot = wide ? "ptr" : "<{ ptr, i32 }>";
const slotValue = (s: string): string =>
wide ? `ptr ${s}` : `<{ ptr, i32 }> <{ ptr ${s}, i32 0 }>`;
out.push(
`@${inst.sym}_data = internal constant [${n} x ptr] [ ${inst.slots.map((s) => `ptr ${s}`).join(", ")} ]`,
`@${inst.sym}_present = internal constant [${n} x i8] ${present}`,
`@${inst.sym} = internal global %ScrArr { ${emitter.sizeType} -1, ${emitter.sizeType} ${n}, ${emitter.sizeType} ${n}, i8 2, i8 0, i16 0, i32 ${n}, ptr null, ptr @${inst.sym}_data, ptr @${inst.sym}_present, ptr null }`,
`@${inst.sym}_data = internal constant <{ [${n} x ${slot}], [${n} x i8] }> <{ [${n} x ${slot}] [ ${inst.slots.map(slotValue).join(", ")} ], [${n} x i8] ${present} }>, align 8`,
`@${inst.sym} = internal global %ScrArr { ${emitter.sizeType} -1, i32 ${n}, i32 ${n}, i8 2, i8 0, i16 0, i32 ${n}, ptr null, ptr @${inst.sym}_data, ptr null }`,
);
}
if (emitter.templateStringsInstances.size > 0) out.push(``);
@@ -1333,14 +1343,17 @@ export function helperDefs(emitter: LlEmitter): string[] {
// of the capacity word (SCR_STR_ASCII_BIT, the low half's sign bit on
// little-endian targets) makes UTF-16 units bytes. Unproven strings and
// anything but an in-range index take the runtime's UTF-16 mapping.
// A scan's receiver reads inline too where runtime state is ordinary
// global storage (ScrStrScan; SCR_TL makes it thread-local otherwise).
const scanInline = emitter.mod.workers !== true && emitter.mod.lib?.threadInstances !== true;
// A scan's receiver reads inline too (ScrStrScan): from ordinary global
// storage, or, in worker executables, from the thread's static TLS block,
// one thread-pointer-relative load (see executableTls).
const scanInline =
emitter.executableTls ||
(emitter.mod.workers !== true && emitter.mod.lib?.threadInstances !== true);
let scanDeclared = false;
for (const helper of STRING_READ_HELPERS) {
if (!emitter.decls.has(helper.decl)) continue;
if (scanInline && !scanDeclared) {
defs.push(stringScanDecl(emitter.sizeType), ``);
defs.push(stringScanDecl(emitter.sizeType, emitter.executableTls), ``);
scanDeclared = true;
}
defs.push(...helper.define(emitter.sizeType, FN_ATTRS, scanInline), ``);

Some files were not shown because too many files have changed in this diff Show More