diff --git a/AGENTS.md b/AGENTS.md index 4d66cb42f..b94c90067 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,6 +48,12 @@ Corpus programs are differential tests against Node: every program runs under No Before opening a PR that adds or changes corpus, diagnostic, or frontend fixtures, run `pnpm test:ts7 --baselines-only`. This fast completeness check uses the same fixture discovery as the parity suite; it does not type-check or replace behavioral parity. Generate missing preflight/order baselines, review their diagnostics and module order, and run the affected `order-parity.test.ts` batches before pushing. Keep existing baseline entries unchanged unless the corresponding behavior intentionally changes. The full `pnpm test:ts7` command also checks completeness before starting its sweep. +A change that must not cost good code anything, such as a compatibility fix that replaces a refusal, proves it with the zero-cost gate. The gate compiles the benchmark workloads, the self-hosted compiler, and the TypeScript corpus with two built checkouts. It requires byte-identical IR, LLVM, and release executables, and compile time within tolerance. `--allow=` names the change's pattern signature: programs whose sources match may differ, and the PR lists them. Any other difference fails the gate, which names the first differing function. The gate also enforces the two-tier runtime separation: no executable may gain symbols from `DYNAMIC_TIER_PREFIXES` in `scripts/zero-cost-compare.mjs`, and fully typed benchmarks may link none; extend that list as dynamic-tier units land. `--quick` checks only the benchmarks; `--cachegrind` adds per-function instruction counts for executables that differ. Run it in a Sandbox with both checkouts prepared: + +```bash +node scripts/zero-cost-gate.mjs --baseline=/vercel/sandbox/prev --candidate=. --allow='\barguments\b' +``` + Test location follows scope: - Co-locate white-box unit tests with implementation files under diff --git a/docs/content/docs/cli.mdx b/docs/content/docs/cli.mdx index a6f75c6d3..bbe1da3c6 100644 --- a/docs/content/docs/cli.mdx +++ b/docs/content/docs/cli.mdx @@ -119,7 +119,7 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
On Linux, when the program calls process.exit(), hand the release of its memory to a helper process so the parent sees the exit sooner. The kernel otherwise frees the whole address space before the exit is reported, which takes tens of milliseconds per gigabyte. Opt in only where the trade-offs are acceptable: for up to about 100 ms after the exit, the helper keeps the executable mapped (overwriting the binary in place fails with "text file busy"), its memory stays charged to the cgroup, and the teardown CPU time no longer appears in the parent's resource usage. Valid only for executable builds; it has no effect on other platforms.
--npm-static <pkg[,pkg…]|auto>
-
Name npm packages to compile even if they don't ship their own type declarations (repeatable). auto uses the default automatic selection. Packages that can't compile produce diagnostics. See npm Dependencies.
+
Name npm packages to compile even if they don't ship their own type declarations (repeatable). A trailing ..., as in fs-extra..., also names every package that package depends on. auto uses the default automatic selection. Packages that can't compile produce diagnostics. See npm Dependencies.
--provenance-sources
Experimental. Compile npm dependencies from the source at the commit their provenance attestation names. Packages without a usable attestation compile from their installed files.
@@ -127,6 +127,12 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
--external-types <specifier=file.d.ts>
Coverage only. Map an exact bare module specifier to a local declaration file supplied by an embedder. Repeat the option for multiple modules. The declaration supplies checker types so application coverage can continue; runtime imports and values remain explicit SC1010 blockers. Relative paths resolve from the current working directory. Accepted files end in .d.ts, .d.mts, or .d.cts.
+
--tier <auto|static-only>
+
The tier policy for code that has no static lowering. auto is the default. static-only guarantees that the program contains no dynamic-tier demotions: every such construct stays a compile error in TypeScript and a deferred runtime error in JavaScript, which is how both modes behave today. Use it in CI to keep a program fully static. Library mode is always static-only. See Dynamic tier.
+ +
--explain-dynamic[=<name>]
+
Coverage only. After the report, list the functions that already operate on values typed only at runtime and the bindings, fields, functions, classes and modules that unsupported constructs would move to the dynamic tier, each with the diagnostic behind it. With a name such as x, Point, or Point.x, show only the matching units. See Dynamic tier.
+
--sanitize
Build an executable with AddressSanitizer and a reference-count audit at exit, the same checks scriptc's own test suite uses. Requires a C compiler and is available only for host builds. Not supported with --emit=asm or --emit=obj.
diff --git a/docs/content/docs/coverage.mdx b/docs/content/docs/coverage.mdx index 2d26e684b..805d7f612 100644 --- a/docs/content/docs/coverage.mdx +++ b/docs/content/docs/coverage.mdx @@ -36,6 +36,7 @@ When a program has unsupported operations, the report adds a group for each root statements analyzedExecutable statements in application code, excluding dependency implementation code. compile staticallyStatements that compile to native instructions. + in dynamic tierOf those, the statements in functions that operate on values typed only at runtime, such as any values and untyped JavaScript. Shown when there are any. See Dynamic tier. blockersUnsupported operations with counts, diagnostic codes, reasons, locations, and hints. in unreached codeBlockers in code the entry never reaches. A build never compiles that code, so these cannot fail it. differs from NodeSites that compile but can behave differently from Node.js. See Node divergences. @@ -51,6 +52,16 @@ If a function's signature can't compile, the statements in its body count as not + + +## Dynamic tier + +Code that operates on values whose types are known only at runtime, such as `any` values, the result of an untyped `JSON.parse`, or untyped JavaScript, compiles into dynamic-tier functions. They are native code too, but they check types and look up properties at runtime, so they are slower than code with static types. The report's `in dynamic tier` line counts their statements. + +`--explain-dynamic` lists those functions and the operation that makes each one dynamic. It also lists the units an unsupported construct would move to the dynamic tier: the binding, field, class, function, or module that has no static representation for the construct, with the diagnostic behind each one. These are the constructs the dynamic tier is planned to compile; today they remain blockers in TypeScript and deferred runtime errors in JavaScript. Add a name, such as `--explain-dynamic=Point`, to show only the matching units. With `--print=diagnostics`, the document gains a `tiers` object with the same information. + +`--tier=static-only` keeps every unsupported construct a blocker instead of moving it to the dynamic tier; code that already operates on runtime-typed values compiles as before. Today `auto`, the default, behaves the same way. + ## Diagnostic codes Coverage and `scriptc build` report the same diagnostic codes. Common ones include: @@ -160,7 +171,7 @@ For editors, CI annotations, and other tools, `scriptc coverage --print=diagnost stats - Coverage only, absent after a preflight failure: the report's statement counts. statementsDynamic is always 0 and remains for schema compatibility. + Coverage only, absent after a preflight failure: the report's statement counts. statementsStatic counts every statement that compiles to native code; statementsDynamic is the part of those in dynamic-tier functions. category diff --git a/docs/content/docs/dependencies.mdx b/docs/content/docs/dependencies.mdx index 5ecc144c6..3c6958841 100644 --- a/docs/content/docs/dependencies.mdx +++ b/docs/content/docs/dependencies.mdx @@ -46,6 +46,8 @@ Your callback and the commander implementation both compile to native code. No e scriptc finds packages in `node_modules` with Node.js's resolution rules, including the export conditions in `package.json`, and compiles their dependencies too. TypeScript workspace packages compile as part of your program, including `.js` import specifiers that resolve to `.ts` sources. +Type-only imports follow Node.js's type stripping. `import type` and `export type … from` statements are erased, so a package or module named only by them supplies types and isn't compiled or loaded; it can even be a package with no JavaScript, such as `type-fest`. A clause whose specifiers are all marked `type`, such as `import { type Options } from "./config.ts"`, still loads its module, as it does under Node.js. + ## Which packages compile scriptc compiles a package automatically when it: @@ -53,7 +55,7 @@ scriptc compiles a package automatically when it: - ships its own type declarations, and - ships readable, unminified JavaScript. -A package that doesn't meet these requirements, or that uses something scriptc can't compile, gets a package-specific diagnostic that explains why. The usual fixes are to switch to a version or alternative package that compiles, or to replace the call with code you write yourself. +A package that doesn't meet these requirements, or that uses something scriptc can't compile, gets a package-specific diagnostic that explains why. The usual fixes are to switch to a version or alternative package that compiles, or to replace the call with code you write yourself. A package whose declarations come from a separate `@types/` package doesn't ship its own; its diagnostic names the package and the `--npm-static` option that compiles it anyway. @@ -67,7 +69,7 @@ Some packages rely on separately installed declarations, such as an `@types/` pa $ scriptc build app.ts --npm-static=three -o app >/dev/null ``` -Naming a package skips the declaration requirement above, but the package still has to compile. `--npm-static=auto` uses the default automatic selection. The option accepts a comma-separated list and can be repeated. +Naming a package skips the declaration requirement above, but the package still has to compile. The option selects only the packages it names: a dependency of a named package that doesn't ship its own declarations is still refused. Add a trailing `...` to select a package together with every package it depends on, for example `--npm-static=fs-extra...`. `--npm-static=auto` uses the default automatic selection. The option accepts a comma-separated list and can be repeated. ## CommonJS packages diff --git a/docs/content/docs/limitations.mdx b/docs/content/docs/limitations.mdx index 544bd2b2d..6f21e2ce3 100644 --- a/docs/content/docs/limitations.mdx +++ b/docs/content/docs/limitations.mdx @@ -76,6 +76,8 @@ title: "Limitations" ../limitations/builtin-apis/legacy-url-api.mdx +../limitations/builtin-apis/deprecated-modules.mdx + ../limitations/builtin-apis/filesystem-flags.mdx ../limitations/builtin-apis/copy.mdx diff --git a/docs/content/limitations/builtin-apis/deprecated-modules.mdx b/docs/content/limitations/builtin-apis/deprecated-modules.mdx new file mode 100644 index 000000000..bb630b8c3 --- /dev/null +++ b/docs/content/limitations/builtin-apis/deprecated-modules.mdx @@ -0,0 +1 @@ +- **Deprecated `punycode` and `constants` modules:** `punycode.encode()`, `decode()`, `toASCII()`, `toUnicode()`, `ucs2.decode()`, `ucs2.encode()`, and `version` compile, with the `DEP0040` warning Node.js reports when the module loads. A `require("punycode")` made from inside `node_modules` doesn't warn; a `require()` used inside a larger expression, such as `require("punycode").toASCII(name)`, doesn't warn either. The `punycode.ucs2` object itself is unsupported as a value. The `constants` module is the frozen object Node.js builds from `os.constants` and `fs.constants`, plus the `crypto.constants` of Node.js 24.22.0. diff --git a/docs/content/limitations/builtin-apis/test-runner.mdx b/docs/content/limitations/builtin-apis/test-runner.mdx index e1baac963..6583a2873 100644 --- a/docs/content/limitations/builtin-apis/test-runner.mdx +++ b/docs/content/limitations/builtin-apis/test-runner.mdx @@ -2,3 +2,4 @@ - **Snapshots:** `t.assert.snapshot` and `t.assert.fileSnapshot` compile, with Node's default serializer, `JSON.stringify(value, null, 2)`, or an `options.serializers` array literal. A compiled test reads the snapshot file Node writes beside the entry file, `.snapshot`, at the entry file's path when the program was compiled. Running the binary with Node's flag `--test-update-snapshots` as its first argument writes the snapshot files instead; as in Node, the flag is kept out of `process.argv` and appears in `process.execArgv`. Snapshot files are read in the form Node writes them, `` exports[`key`] = `value`; `` statements. The `snapshot.setDefaultSnapshotSerializers` and `snapshot.setResolveSnapshotPath` module functions throw `SC2020`. - **TestContext values:** `t.filePath` is the entry file's path when the program was compiled. `t.attempt` is `0`, since a test runs once. `t.passed` is `false` while the body runs and the test's outcome in its `afterEach` and `t.after` hooks. `t.workerId` is `Number(process.env.NODE_TEST_WORKER_ID) || undefined`, as in Node. `t.tags` is `[]`, since the `tags` test option throws `SC2020`; the array is not frozen as Node's is. - **Suite and hook contexts:** a `describe` body may take its `SuiteContext`, which has `name`, `fullName`, `filePath`, `attempt`, `passed` and `diagnostic`. Hooks may take their context, as in Node: the suite's for `before` and `after`, the test's for `beforeEach`, `afterEach`, `t.before` and `t.after`. The `SuiteContext` `signal` and `log` members, and hook `done` callbacks, throw `SC2020`. +- **Module-level hooks:** `before` hooks outside any suite run at their `before()` call, and the first top-level test waits for the async ones; `after` hooks outside any suite run after the last top-level test. Both get the root's context, named ``. As in Node, a throwing root `before` fails each top-level test with its error, without running its body, and cancels the tests and suites inside each top-level suite. A throwing root `after` reports the entry file as a failed result, and the exit code is unchanged. diff --git a/docs/src/generated/node-v24-compatibility-meta.json b/docs/src/generated/node-v24-compatibility-meta.json index a64169933..f03a39e85 100644 --- a/docs/src/generated/node-v24-compatibility-meta.json +++ b/docs/src/generated/node-v24-compatibility-meta.json @@ -2,6 +2,6 @@ "schemaVersion": 3, "nodeVersion": "24.22.0", "nodeCommit": "f14cd3aa7bf9ca8df2cc5b8bb14beb0f335b1d93", - "artifactVersion": "a610bdd1ae56e1c4cd38", + "artifactVersion": "4421a7b808191ac7f410", "rowCount": 3880 } diff --git a/docs/src/generated/node-v24-compatibility.json b/docs/src/generated/node-v24-compatibility.json index d4b6839e7..7a9e0e2a2 100644 --- a/docs/src/generated/node-v24-compatibility.json +++ b/docs/src/generated/node-v24-compatibility.json @@ -584,9 +584,9 @@ "apiEntries": 9, "static": { "supported": 0, - "partial": 0, + "partial": 8, "refused": 0, - "not-implemented": 9, + "not-implemented": 1, "by-design": 0, "unreviewed": 0, "not-applicable": 0 @@ -58489,7 +58489,7 @@ "inherited": false }, "static": { - "status": "not-implemented", + "status": "partial", "detail": "Derived from the exact API rows in this chapter.", "verification": "derived" } @@ -58512,9 +58512,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { @@ -58535,9 +58535,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { @@ -58558,9 +58558,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { @@ -58581,9 +58581,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { @@ -58627,9 +58627,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { @@ -58650,9 +58650,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { @@ -58673,9 +58673,9 @@ "inherited": true }, "static": { - "status": "not-implemented", - "detail": "No static compiler lowering is registered for this API yet.", - "verification": "registry-gap" + "status": "partial", + "detail": "Implemented for the call shapes accepted by the compiler lowering.", + "verification": "test-backed" } }, { diff --git a/internal/compatibility/generated/node-v24-backlog.json b/internal/compatibility/generated/node-v24-backlog.json index f36c865cf..0fba7781e 100644 --- a/internal/compatibility/generated/node-v24-backlog.json +++ b/internal/compatibility/generated/node-v24-backlog.json @@ -7,8 +7,8 @@ "tierItemCount": 3456, "tiers": { "static": { - "audit-partial": 1155, - "verify-gap": 2174, + "audit-partial": 1162, + "verify-gap": 2167, "replace-refusal": 40, "classify": 2, "implement": 85 @@ -58559,12 +58559,16 @@ "priority": "low", "tiers": { "static": { - "status": "not-implemented", - "action": "verify-gap", - "verification": "registry-gap", - "confidence": "low", - "source": "compiler-unmatched:punycode", - "tests": [] + "status": "partial", + "action": "audit-partial", + "verification": "test-backed", + "confidence": "high", + "source": "surface-manifest:node-builtin.punycode.decode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] } } }, @@ -58584,12 +58588,16 @@ "priority": "low", "tiers": { "static": { - "status": "not-implemented", - "action": "verify-gap", - "verification": "registry-gap", - "confidence": "low", - "source": "compiler-unmatched:punycode", - "tests": [] + "status": "partial", + "action": "audit-partial", + "verification": "test-backed", + "confidence": "high", + "source": "surface-manifest:node-builtin.punycode.encode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] } } }, @@ -58609,12 +58617,16 @@ "priority": "low", "tiers": { "static": { - "status": "not-implemented", - "action": "verify-gap", - "verification": "registry-gap", - "confidence": "low", - "source": "compiler-unmatched:punycode", - "tests": [] + "status": "partial", + "action": "audit-partial", + "verification": "test-backed", + "confidence": "high", + "source": "surface-manifest:node-builtin.punycode.toASCII", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] } } }, @@ -58634,12 +58646,16 @@ "priority": "low", "tiers": { "static": { - "status": "not-implemented", - "action": "verify-gap", - "verification": "registry-gap", - "confidence": "low", - "source": "compiler-unmatched:punycode", - "tests": [] + "status": "partial", + "action": "audit-partial", + "verification": "test-backed", + "confidence": "high", + "source": "surface-manifest:node-builtin.punycode.toUnicode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] } } }, @@ -58684,12 +58700,16 @@ "priority": "low", "tiers": { "static": { - "status": "not-implemented", - "action": "verify-gap", - "verification": "registry-gap", - "confidence": "low", - "source": "compiler-unmatched:punycode", - "tests": [] + "status": "partial", + "action": "audit-partial", + "verification": "test-backed", + "confidence": "high", + "source": "surface-manifest:node-builtin.punycode.ucs2.decode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] } } }, @@ -58709,12 +58729,16 @@ "priority": "low", "tiers": { "static": { - "status": "not-implemented", - "action": "verify-gap", - "verification": "registry-gap", - "confidence": "low", - "source": "compiler-unmatched:punycode", - "tests": [] + "status": "partial", + "action": "audit-partial", + "verification": "test-backed", + "confidence": "high", + "source": "surface-manifest:node-builtin.punycode.ucs2.encode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] } } }, @@ -58734,12 +58758,16 @@ "priority": "low", "tiers": { "static": { - "status": "not-implemented", - "action": "verify-gap", - "verification": "registry-gap", - "confidence": "low", - "source": "compiler-unmatched:punycode", - "tests": [] + "status": "partial", + "action": "audit-partial", + "verification": "test-backed", + "confidence": "high", + "source": "surface-manifest:node-builtin.punycode.version", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] } } }, diff --git a/internal/compatibility/generated/node-v24-internal.json b/internal/compatibility/generated/node-v24-internal.json index 514282a0a..b755a95e8 100644 --- a/internal/compatibility/generated/node-v24-internal.json +++ b/internal/compatibility/generated/node-v24-internal.json @@ -611,9 +611,9 @@ "apiEntries": 9, "static": { "supported": 0, - "partial": 0, + "partial": 8, "refused": 0, - "not-implemented": 9, + "not-implemented": 1, "by-design": 0, "unreviewed": 0, "not-applicable": 0 @@ -81945,8 +81945,13 @@ "inherited": false }, "static": { - "status": "not-implemented", - "evidence": "derived:descendants" + "status": "partial", + "evidence": "derived:descendants", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "chapter" }, @@ -81968,8 +81973,13 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:punycode" + "status": "partial", + "evidence": "surface-manifest:node-builtin.punycode.decode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "exact" }, @@ -81991,8 +82001,13 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:punycode" + "status": "partial", + "evidence": "surface-manifest:node-builtin.punycode.encode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "exact" }, @@ -82014,8 +82029,13 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:punycode" + "status": "partial", + "evidence": "surface-manifest:node-builtin.punycode.toASCII", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "exact" }, @@ -82037,8 +82057,13 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:punycode" + "status": "partial", + "evidence": "surface-manifest:node-builtin.punycode.toUnicode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "exact" }, @@ -82083,8 +82108,13 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:punycode" + "status": "partial", + "evidence": "surface-manifest:node-builtin.punycode.ucs2.decode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "exact" }, @@ -82106,8 +82136,13 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:punycode" + "status": "partial", + "evidence": "surface-manifest:node-builtin.punycode.ucs2.encode", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "exact" }, @@ -82129,8 +82164,13 @@ "inherited": true }, "static": { - "status": "not-implemented", - "evidence": "compiler-unmatched:punycode" + "status": "partial", + "evidence": "surface-manifest:node-builtin.punycode.version", + "tests": [ + "tests/corpus/6122-punycode-conversions.ts", + "tests/corpus/6124-punycode-require-warnings.cjs", + "tests/corpus/6126-punycode-dependency/main.cjs" + ] }, "anchorSource": "chapter" }, diff --git a/internal/compatibility/static-support.json b/internal/compatibility/static-support.json index 45a946925..0adeb9882 100644 --- a/internal/compatibility/static-support.json +++ b/internal/compatibility/static-support.json @@ -147,6 +147,7 @@ "node-builtin.process.rows": ["tests/corpus/2992-stream-rows.ts", "tests/harness/terminal-geometry.test.ts"], "node-builtin.process.columns": ["tests/corpus/1448-stream-columns.ts", "tests/harness/terminal-geometry.test.ts"], "node-builtin.process.exitCode": ["tests/corpus/5809-process-exit-and-rejections.cjs", "tests/corpus/2941-process-exit-code.ts", "tests/corpus/2942-process-exit-code-override.ts", "tests/corpus/2943-process-exit-code-invalid.ts", "tests/corpus/2944-process-exit-code-wrap.ts"], + "node-builtin.punycode": ["tests/corpus/6122-punycode-conversions.ts", "tests/corpus/6124-punycode-require-warnings.cjs", "tests/corpus/6126-punycode-dependency/main.cjs"], "node-builtin.querystring": ["tests/corpus/957-builtins-namespace.ts", "tests/corpus/2917-builtin-callable-widths.ts"], "node-builtin.readline": ["tests/corpus/1475-readline-closed-stdin.ts"], "node-builtin.stream": ["tests/corpus/1685-stream-readable-basics.ts", "tests/corpus/1746-stream-for-await.ts", "tests/corpus/1814-stream-pipeline.cjs", "tests/corpus/2629-stream-consumers.ts", "tests/corpus/2861-readable-iterator-close.ts"], diff --git a/packages/compiler/ambient/scriptc-node-fallback.d.ts b/packages/compiler/ambient/scriptc-node-fallback.d.ts index d72ca2dcd..cc5de935c 100644 --- a/packages/compiler/ambient/scriptc-node-fallback.d.ts +++ b/packages/compiler/ambient/scriptc-node-fallback.d.ts @@ -3762,6 +3762,39 @@ declare module "node:querystring" { export * from "querystring"; } +/* node:punycode (deprecated, DEP0040) — punycode.js 2.1.0: decode and + * toUnicode throw its RangeErrors on malformed input; ucs2 converts + * between strings and UTF-16 code point arrays. */ +declare module "punycode" { + export function decode(string: string): string; + export function encode(string: string): string; + export function toUnicode(domain: string): string; + export function toASCII(domain: string): string; + export namespace ucs2 { + function decode(string: string): number[]; + function encode(codePoints: readonly number[]): string; + } + export const version: string; +} +declare module "node:punycode" { + export * from "punycode"; +} + +/* node:constants (deprecated, DEP0008) — one frozen object: os.constants' + * dlopen, errno, priority and signals groups, fs.constants, and + * crypto.constants. A platform's missing constant reads as undefined. */ +declare module "constants" { + const constants: { readonly [name: string]: number } & { + readonly defaultCoreCipherList: string; + readonly defaultCipherList: string; + }; + export = constants; +} +declare module "node:constants" { + import constants = require("constants"); + export = constants; +} + /* node:readline — the question/close slice: createInterface over exactly * { input: process.stdin, output: process.stdout }, question(query, cb) * (the query writes to stdout, the callback gets the next line's text), diff --git a/packages/compiler/src/backend/llvm/dense-array-access.ts b/packages/compiler/src/backend/llvm/dense-array-access.ts index 183bb8239..f19073b75 100644 --- a/packages/compiler/src/backend/llvm/dense-array-access.ts +++ b/packages/compiler/src/backend/llvm/dense-array-access.ts @@ -47,7 +47,8 @@ interface DenseGuard { * then extends the length over hole states exactly the same way). An index * below the hole-free prefix bound branches to `filledLabel` first; both * that block and the returned (state-checked) dense block may use the - * guard's values. */ + * guard's values. A `proven` index is known to lie below the bound (a dense + * array loop's fast copy, dense-array-loops.ts) and always takes it. */ function emitDenseGuard( host: LlvmEmitterContext, array: string, @@ -56,6 +57,7 @@ function emitDenseGuard( slow: string, requireBelowLen: boolean, filledLabel: string, + proven = false, ): DenseGuard { const B = host.B; const size = host.sizeType; @@ -137,8 +139,10 @@ function emitDenseGuard( B.line(`${offset} = trunc i64 ${wide} to i32`); } // The bound never exceeds the length or the dense capacity. - let inFilled = B.tmp(); - B.line(`${inFilled} = icmp ult i64 ${wide}, ${filledWide}`); + // The proof covers the counter's integer value only. + const below = proven && integral === null; + let inFilled = below ? "true" : B.tmp(); + if (!below) B.line(`${inFilled} = icmp ult i64 ${wide}, ${filledWide}`); if (integral) { const t = B.tmp(); B.line(`${t} = and i1 ${integral}, ${inFilled}`); @@ -272,6 +276,7 @@ export function emitDenseArrayGet( element: IrType, getter: string, retain: boolean, + proven = false, ): string { const B = host.B; const ty = acc === "f64" ? "double" : acc === "bool" ? "i1" : "ptr"; @@ -285,7 +290,7 @@ export function emitDenseArrayGet( const slow = B.newLabel("arr.get.slow"), join = B.newLabel("arr.get.join"), valueLabel = B.newLabel("arr.get.value"); - const guard = emitDenseGuard(host, array, index, indexExpr, slow, true, valueLabel); + const guard = emitDenseGuard(host, array, index, indexExpr, slow, true, valueLabel, proven); const state = loadState(host, guard); const present = B.tmp(); B.line(`${present} = icmp eq i8 ${state}, ${SCR_ARR_VALUE}`); @@ -331,6 +336,7 @@ export function emitDenseArrayNumber( array: string, index: LlValue, indexExpr: IrExpr | undefined, + proven = false, ): string { return emitDenseArrayGet( host, @@ -341,6 +347,7 @@ export function emitDenseArrayNumber( { kind: "f64" }, "scr_arr_get_number", false, + proven, ); } @@ -399,6 +406,7 @@ export function emitDenseArraySet( acc: ArrayAccess, element: IrType, value: string, + proven = false, ): void { const B = host.B; const ty = acc === "f64" ? "double" : acc === "bool" ? "i1" : "ptr"; @@ -413,7 +421,7 @@ export function emitDenseArraySet( const slow = B.newLabel("arr.set.slow"), join = B.newLabel("arr.set.join"), filledLabel = B.newLabel("arr.set.filled"); - const guard = emitDenseGuard(host, array, index, indexExpr, slow, false, filledLabel); + const guard = emitDenseGuard(host, array, index, indexExpr, slow, false, filledLabel, proven); const { offset, len } = guard; const storeValue = (valuePtr: string): void => { if (acc === "f64") { diff --git a/packages/compiler/src/backend/llvm/dense-array-loops.test.ts b/packages/compiler/src/backend/llvm/dense-array-loops.test.ts new file mode 100644 index 000000000..405ad67e7 --- /dev/null +++ b/packages/compiler/src/backend/llvm/dense-array-loops.test.ts @@ -0,0 +1,147 @@ +import { expect, test } from "vitest"; +import { BOOL, F64, arrayOf, type IrExpr, type IrLocal, type IrStmt } from "../../ir/ir.js"; +import { + countedDenseShape, + matchDenseArrayLoop, + type DenseLoopShape, +} from "./dense-array-loops.js"; + +const loc = { file: "dense-array-loops.ts", start: 0, end: 0 }; +const ARR = arrayOf(F64); +const num = (value: number): IrExpr => ({ kind: "numLit", value, type: F64, loc }); +const ref = (localId: string): IrExpr => ({ kind: "varRef", localId, type: F64, loc }); +const arr = (localId: string): IrExpr & { kind: "varRef" } => ({ + kind: "varRef", + localId, + type: ARR, + loc, +}); +const method = ( + receiver: string, + name: Extract["method"], + args: IrExpr[] = [], +): IrExpr => ({ + kind: "arrIntrinsic", + method: name, + receiver: arr(receiver), + args, + type: F64, + loc, +}); +const read = (receiver: string, index = ref("i")): IrExpr => method(receiver, "getNumber", [index]); +const write = (receiver: string, value: IrExpr, index = ref("i")): IrStmt => ({ + kind: "arraySet", + arr: arr(receiver), + index, + value, + loc, +}); +const locals = new Map([ + ["i", { id: "i", name: "i", type: F64, mutable: true }], + ["a", { id: "a", name: "a", type: ARR, mutable: false }], + ["b", { id: "b", name: "b", type: ARR, mutable: true }], + ["c", { id: "c", name: "c", type: ARR, mutable: false }], +]); + +/** for (let i = 0; i < a.length; i++) body */ +function loop(body: IrStmt[]): IrStmt & { kind: "for" } { + return { + kind: "for", + loc, + init: { kind: "varDecl", localId: "i", init: num(0), loc }, + cond: { kind: "bin", op: "<", left: ref("i"), right: method("a", "length"), type: BOOL, loc }, + update: { + kind: "assign", + localId: "i", + value: { kind: "bin", op: "+", left: ref("i"), right: num(1), type: F64, loc }, + loc, + }, + body, + }; +} +const shape: DenseLoopShape = { + counter: "i", + length: arr("a"), + range: { min: 0, max: 4294967294 }, +}; +const match = (body: IrStmt[], initialized = new Set(["a", "b"])) => + matchDenseArrayLoop(loop(body), shape, locals, new Set(), initialized); + +test("arrays indexed at the counter of a length-preserving loop get an int32 fast copy", () => { + const scale = write("b", { + kind: "bin", + op: "*", + left: read("a"), + right: num(2), + type: F64, + loc, + }); + const dense = match([scale]); + expect([...dense!.arrays.keys()].sort()).toEqual(["a", "b"]); + expect(dense!.range).toEqual({ min: 0, max: 2147483646 }); + // A binding not definitely assigned before the loop is not read by the guard. + expect([...match([scale], new Set())!.arrays.keys()]).toEqual(["a"]); +}); + +test("loops that may resize, rebind or move storage keep the ordinary copy only", () => { + expect(match([{ kind: "exprStmt", expr: method("a", "push", [num(1)]), loc }])).toBeNull(); + expect(match([write("a", num(0), ref("unknown"))])).toBeNull(); + expect( + match([ + { kind: "exprStmt", expr: { kind: "call", callee: "f", args: [], type: F64, loc }, loc }, + ]), + ).toBeNull(); + // A nested loop would be duplicated too. + expect(match([{ kind: "exprStmt", expr: read("a"), loc }, loop([])])).toBeNull(); + // No access at the counter: nothing to prove. + expect(match([{ kind: "exprStmt", expr: read("a", num(0)), loc }])).toBeNull(); + // Copying the bounding array into another binding lets it escape. + expect( + match([ + { kind: "exprStmt", expr: read("a"), loc }, + { kind: "assign", localId: "b", value: arr("a"), loc }, + ]), + ).toBeNull(); + // A rebound array is not covered, but the bounding array still is. + const rebound = match([ + { kind: "exprStmt", expr: read("b"), loc }, + { kind: "exprStmt", expr: read("a"), loc }, + { kind: "assign", localId: "b", value: arr("c"), loc }, + ]); + expect([...rebound!.arrays.keys()]).toEqual(["a"]); +}); + +test("counted shapes need a non-negative counter strictly below the length", () => { + const ascending = (limit: IrExpr, inclusive: boolean, min = 0) => + countedDenseShape(loop([]), { + localId: "i", + start: num(min), + limit, + step: 1, + inclusive, + range: { min, max: 100 }, + }); + const minus = (offset: number): IrExpr => ({ + kind: "bin", + op: "-", + left: method("a", "length"), + right: num(offset), + type: F64, + loc, + }); + expect(ascending(method("a", "length"), false)?.length.localId).toBe("a"); + expect(ascending(minus(1), true)?.length.localId).toBe("a"); + expect(ascending(method("a", "length"), true)).toBeNull(); + expect(ascending(minus(1), false, -1)).toBeNull(); + const descending = (offset: number) => + countedDenseShape(loop([]), { + localId: "i", + start: minus(offset), + limit: num(0), + step: -1, + inclusive: true, + range: { min: 0, max: 100 }, + }); + expect(descending(1)?.length.localId).toBe("a"); + expect(descending(0)).toBeNull(); +}); diff --git a/packages/compiler/src/backend/llvm/dense-array-loops.ts b/packages/compiler/src/backend/llvm/dense-array-loops.ts new file mode 100644 index 000000000..271a5ddf8 --- /dev/null +++ b/packages/compiler/src/backend/llvm/dense-array-loops.ts @@ -0,0 +1,215 @@ +/* Dense array loops: a versioned copy of an index loop over an array whose + * length the loop never changes. + * + * `for (let i = 0; i < a.length; i++) b[i] = a[i] * 2` reads its bound from + * `a`, and every element access at the counter normally tests the index + * against the array's hole-free prefix bound (`filled`, see + * dense-array-access.ts) before it touches the slot. When the loop provably + * never changes `a.length` (ir/array-length-stability.ts) and never moves or + * cuts the storage of the arrays it indexes, one test before the loop covers + * every pass: if `a.length` is at most each indexed array's `filled` bound, + * every counter value the loop can reach (at least 0, below `a.length`) + * names a present value in dense storage. The fast copy then reads and + * replaces those slots directly, which leaves LLVM a loop without + * per-element branches (vectorizable where the body allows it). The same + * test bounds `a.length` by 2^31 - 1, so the fast copy's counter is an int32. + * Arrays longer than that are sparse in practice (a dense one would need + * 16 GB of slots) and take the ordinary copy, as does every array with a + * hole, present undefined or sparse storage below `a.length`. */ + +import type { IrExpr, IrLocal, IrStmt } from "../../ir/ir.js"; +import type { IntegerRange } from "../../ir/integer-ranges.js"; +import { preservesArrayLength } from "../../ir/array-length-stability.js"; +import { everyExprChild, everyStmtChild } from "../../ir/traverse.js"; +import type { LlvmEmitterContext } from "./expr-context.js"; +import { SCR_ARR_FIELD } from "./common.js"; + +export const INT32_MAX = 2147483647; + +export interface DenseArrayLoop { + /** The loop's counter binding. */ + counter: string; + /** The array whose length bounds the counter. */ + length: IrExpr & { kind: "varRef" }; + /** Arrays whose accesses at the counter are proven to be inside the + * hole-free prefix in the fast copy, with a read of each binding. */ + arrays: ReadonlyMap; + /** The counter's range on every pass of the fast copy. */ + range: IntegerRange; +} + +/** The fast copy duplicates the loop; keep the duplicated code small. */ +const MAX_LOOP_NODES = 120; + +/** How a matched integer loop bounds its counter below `length.length`. */ +export interface DenseLoopShape { + counter: string; + length: IrExpr & { kind: "varRef" }; + /** The counter's proven range on every pass of the ordinary loop; the + * fast copy keeps its minimum, which must be at least 0. */ + range: IntegerRange; +} + +/** `length.length - c` (c an integer literal, 0 when absent). */ +function lengthOffset(e: IrExpr): { receiver: IrExpr & { kind: "varRef" }; offset: number } | null { + let offset = 0; + if ( + e.kind === "bin" && + e.op === "-" && + e.right.kind === "numLit" && + Number.isSafeInteger(e.right.value) + ) { + offset = e.right.value; + e = e.left; + } + return e.kind === "arrIntrinsic" && + e.method === "length" && + e.args.length === 0 && + e.receiver.kind === "varRef" && + e.receiver.type.kind === "array" + ? { receiver: e.receiver, offset } + : null; +} + +/** The array bounding a counted loop's counter below its length: an + * ascending loop compared below `a.length - c`, or a descending one starting + * at `a.length - c` (c >= 1). */ +export function countedDenseShape( + loop: IrStmt & { kind: "for" }, + counted: { + localId: string; + start: IrExpr; + limit: IrExpr; + step: number; + inclusive: boolean; + range: IntegerRange; + }, +): DenseLoopShape | null { + if (counted.range.min < 0) return null; + if (counted.step > 0) { + const bound = lengthOffset(counted.limit); + if (!bound || bound.offset < (counted.inclusive ? 1 : 0)) return null; + return { counter: counted.localId, length: bound.receiver, range: counted.range }; + } + // The start is evaluated once, before the loop; the counter only falls. + const start = lengthOffset(counted.start); + if (!start || start.offset < 1 || loop.init?.kind !== "varDecl") return null; + return { counter: counted.localId, length: start.receiver, range: counted.range }; +} + +/** Match the fast copy of a loop of the given shape. `initialized` holds the + * bindings definitely assigned when the loop starts; `unaliased` the local + * arrays no other binding holds. */ +export function matchDenseArrayLoop( + loop: IrStmt & { kind: "for" }, + shape: DenseLoopShape, + locals: ReadonlyMap, + unaliased: ReadonlySet, + initialized: ReadonlySet, +): DenseArrayLoop | null { + const { counter, length } = shape; + if (shape.range.min < 0 || !loop.cond) return null; + let nodes = 0, + nested = false; + const candidates = new Map(); + // Arrays written, resized, or indexed other than at the counter by an + // element write. + const disturbed = new Set(); + const access = (arr: IrExpr, index: IrExpr): void => { + if (arr.kind === "varRef" && index.kind === "varRef" && index.localId === counter) + candidates.set(arr.localId, arr); + }; + const expr = (e: IrExpr): boolean => { + if (++nodes > MAX_LOOP_NODES) return false; + if (e.kind === "assignExpr" || e.kind === "incDec") disturbed.add(e.localId); + if (e.kind === "arrayGet") access(e.arr, e.index); + if (e.kind === "arrIntrinsic") { + if (e.method === "getNumber" && e.args[0]) access(e.receiver, e.args[0]); + else if (e.method !== "length" && e.receiver.kind === "varRef") + disturbed.add(e.receiver.localId); + } + return everyExprChild(e, expr, stmt); + }; + const stmt = (s: IrStmt): boolean => { + if (++nodes > MAX_LOOP_NODES) return false; + if (s.kind === "for" || s.kind === "while" || s.kind === "doWhile" || s.kind === "forOf") + nested = true; + if (s.kind === "assign" || s.kind === "varDecl") disturbed.add(s.localId); + if (s.kind === "arraySet") { + access(s.arr, s.index); + if (s.arr.kind === "varRef" && (s.index.kind !== "varRef" || s.index.localId !== counter)) + disturbed.add(s.arr.localId); + } + return everyStmtChild(s, expr, stmt); + }; + if (!loop.body.every(stmt) || !expr(loop.cond) || nested) return null; + if ( + !preservesArrayLength( + [loop.cond, ...loop.body], + { receiver: length.localId, inRange: counter }, + locals, + unaliased, + ) + ) + return null; + const arrays = new Map(); + for (const [id, ref] of candidates) { + const local = locals.get(id); + if ( + !local || + local.type.kind !== "array" || + local.boxed || + local.tdz || + disturbed.has(id) || + (id !== length.localId && !initialized.has(id)) + ) + continue; + arrays.set(id, ref); + } + if (arrays.size === 0) return null; + return { + counter, + length, + arrays, + range: { min: shape.range.min, max: Math.min(shape.range.max, INT32_MAX - 1) }, + }; +} + +/** The fast copy's entry test: the bounding length is at most 2^31 - 1 and + * at most every indexed array's hole-free prefix bound. */ +export function emitDenseLoopGuard(host: LlvmEmitterContext, loop: DenseArrayLoop): string { + const B = host.B; + const size = host.sizeType; + const header = (array: string, field: number): string => { + const ptr = B.tmp(); + B.line(`${ptr} = getelementptr inbounds %ScrArr, ptr ${array}, i32 0, i32 ${field}`); + host.markMemoryPointer(ptr, "array:header"); + return ptr; + }; + const receiver = host.emitStableReceiver(loop.length, []); + const lenPtr = header(receiver.name, 1), + len = B.tmp(), + small = B.tmp(); + B.line( + `${len} = load ${size}, ptr ${lenPtr}${host.fieldAliasAttachment(lenPtr)}${host.arrayLengthRange}`, + ); + B.line(`${small} = icmp ule ${size} ${len}, ${INT32_MAX}`); + let valid = small; + for (const [id, ref] of loop.arrays) { + const array = id === loop.length.localId ? receiver : host.emitStableReceiver(ref, []); + const filledPtr = header(array.name, SCR_ARR_FIELD.filled), + filled = B.tmp(), + covered = B.tmp(), + both = B.tmp(); + B.line(`${filled} = load i32, ptr ${filledPtr}${host.fieldAliasAttachment(filledPtr)}`); + let bound = filled; + if (size === "i64") { + bound = B.tmp(); + B.line(`${bound} = zext i32 ${filled} to i64`); + } + B.line(`${covered} = icmp ule ${size} ${len}, ${bound}`); + B.line(`${both} = and i1 ${valid}, ${covered}`); + valid = both; + } + return valid; +} diff --git a/packages/compiler/src/backend/llvm/dyn.ts b/packages/compiler/src/backend/llvm/dyn.ts index bb35e70c2..86e97a374 100644 --- a/packages/compiler/src/backend/llvm/dyn.ts +++ b/packages/compiler/src/backend/llvm/dyn.ts @@ -1217,12 +1217,13 @@ export class LlDyn { `llvm emitter bug: JSON schema of unknown shape ${t.shapeId}`, ); const struct = mangleRecordStruct(t.shapeId); - const index = new Map(shape.fields.map((f, i) => [f.name, i + 1])); + const base = host.nullableUnions.fieldBase(t.shapeId); + const index = new Map(shape.fields.map((f, i) => [f.name, i + base])); // The runtime names kept-source keys by this index (record-sources.ts). const order = recordSourceFieldOrder(shape); const fieldTy = `{ ptr, ${S}, ${S}, ptr }`; const entries = order.map((fieldName) => { - const field = shape.fields[index.get(fieldName)! - 1]!; + const field = shape.fields[index.get(fieldName)! - base]!; const sub = this.jsonSchema(field.type); const offset = `ptrtoint (ptr getelementptr (%${struct}, ptr null, i64 0, i32 ${index.get(fieldName)}) to ${S})`; return `${fieldTy} { ptr ${host.cstr(fieldName)}, ${S} ${Buffer.byteLength(fieldName, "utf8")}, ${S} ${offset}, ptr @${sub} }`; @@ -1240,7 +1241,7 @@ export class LlDyn { const overflow = shape.indexValue ? { elem: `@${this.jsonSchema(shape.indexValue)}`, - offset: `ptrtoint (ptr getelementptr (%${struct}, ptr null, i64 0, i32 ${shape.fields.length + 1}) to ${S})`, + offset: `ptrtoint (ptr getelementptr (%${struct}, ptr null, i64 0, i32 ${shape.fields.length + base}) to ${S})`, } : { elem: "null", offset: "0" }; body = `${S} 3, ${S} ${entries.length}, ptr ${fields}, ptr ${overflow.elem}, ptr @${mangleRecordNew(t.shapeId)}, ptr null, ptr ${vAdapters(host, t).release}, ptr null, ${S} ${overflow.offset}`; @@ -1547,18 +1548,19 @@ export class LlDyn { B.line(`call void @scr_arr_outline(ptr ${cached})`); B.line(`call void @scr_arr_outline(ptr ${checked})`); } + const base = t.kind === "record" ? host.nullableUnions.fieldBase(t.shapeId) : 1; const members = t.kind === "record" ? [ ...shape!.fields.map((field, index) => ({ - index: index + 1, + index: index + base, type: llFieldType(field.type, host.inlineUnions.record(field.type)), name: field.name, })), ...(shape!.indexValue ? [ { - index: shape!.fields.length + 1, + index: shape!.fields.length + base, type: "ptr" as const, name: "[key: string] overflow", }, @@ -1819,7 +1821,8 @@ export class LlDyn { `llvm emitter bug: dynCheck of unknown shape ${t.shapeId}`, ); const struct = mangleRecordStruct(t.shapeId); - const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + 1])); + const base = host.nullableUnions.fieldBase(t.shapeId); + const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + base])); const releaseR = (): void => { B.line(`call void ${releaseSym(host, t)}(ptr %r0)`); }; @@ -1996,7 +1999,7 @@ export class LlDyn { const ovfp = B.tmp(); const ovf = B.tmp(); B.line( - `${ovfp} = getelementptr inbounds %${struct}, ptr %r0, i64 0, i32 ${shape.fields.length + 1}`, + `${ovfp} = getelementptr inbounds %${struct}, ptr %r0, i64 0, i32 ${shape.fields.length + base}`, ); B.line(`${ovf} = load ptr, ptr ${ovfp} ; overflow map`); if (shape.fields.length === 0) { @@ -2578,7 +2581,8 @@ export class LlDyn { if (!shape) throw new InternalCompilerError(`llvm emitter bug: to-dyn of unknown shape ${t.shapeId}`); const struct = mangleRecordStruct(t.shapeId); - const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + 1])); + const base = host.nullableUnions.fieldBase(t.shapeId); + const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + base])); const loadFieldOf = (fname: string, ft: IrType): string => { const p = B.tmp(); B.line( @@ -2702,7 +2706,7 @@ export class LlDyn { const ovfp = B.tmp(); const ovf = B.tmp(); B.line( - `${ovfp} = getelementptr inbounds %${struct}, ptr %v, i64 0, i32 ${shape.fields.length + 1}`, + `${ovfp} = getelementptr inbounds %${struct}, ptr %v, i64 0, i32 ${shape.fields.length + base}`, ); B.line(`${ovf} = load ptr, ptr ${ovfp} ; overflow map`); const ks = B.tmp(); diff --git a/packages/compiler/src/backend/llvm/emitter.ts b/packages/compiler/src/backend/llvm/emitter.ts index d5d52926a..f0c4b1cf0 100644 --- a/packages/compiler/src/backend/llvm/emitter.ts +++ b/packages/compiler/src/backend/llvm/emitter.ts @@ -56,6 +56,13 @@ import { matchIntegerBytesForLoop, matchIntegerCountedForLoop, } from "../../ir/integer-loops.js"; +import { findUnaliasedLocalArrays } from "../../ir/array-length-stability.js"; +import { + countedDenseShape, + emitDenseLoopGuard, + matchDenseArrayLoop, + type DenseArrayLoop, +} from "./dense-array-loops.js"; import { emitCountedLoopLimit } from "./counted-loops.js"; import { emitSafeIntegerCheck, @@ -230,7 +237,10 @@ import { findAppendedArrayLiterals } from "./appended-array-literals.js"; import { AmbientReceiverReaders, findConstantCallbacks } from "./constant-callbacks.js"; import { findScalarStringSlices, + findScratchStringSlices, + findLentStringParameters, emitStringSliceSnapshot, + type LentStringParameters, type StringSliceSnapshot, } from "./string-slices.js"; import { LazyCaptures } from "./lazy-captures.js"; @@ -384,6 +394,7 @@ import { elemAccess, emitInlineRcHelpers, emitNullableRetainWrappers, + emitShapeUnionHelpers, FN_ATTRS, inlineRcDecls, type InlineRcMode, @@ -624,6 +635,10 @@ export class LlEmitter { private loopArrayBorrows: ReadonlySet = new Set(); private scalarStringSlices = new Map(); readonly stringSlices = new Map(); + /** Slices written into stack buffers (findScratchStringSlices). */ + scratchStringSlices: ReadonlySet = new Set(); + /** String parameters a caller may lend scratch slices to. */ + private lentStringParameters: LentStringParameters = new Map(); private localStackUnions = new Map(); private localStackObjects = new Map(); private reassignedStackObjects = new Set(); @@ -675,6 +690,21 @@ export class LlEmitter { return this.sizeType === "i64" ? ", !range !{i64 0, i64 4294967296}" : ""; } + /** Whether an element access at `index` of `arr` lies in the hole-free + * prefix: the counter of a dense array loop's fast copy indexing one of + * the arrays its entry test covered. */ + provenDenseIndex(arr: IrExpr, index: IrExpr): boolean { + const loop = this.denseLoop; + return ( + loop !== null && + arr.kind === "varRef" && + loop.arrays.has(arr.localId) && + index.kind === "varRef" && + index.localId === loop.counter && + this.integerLoopBindings.has(loop.counter) + ); + } + fieldAliasAttachment(ptr: string): string { const tag = this.fieldPointerTags.get(ptr); return tag === undefined ? "" : `, !tbaa !${tag}`; @@ -1031,6 +1061,14 @@ export class LlEmitter { * immortal from construction (classes.ts programLifetimeClass). */ readonly programLifetime: ReadonlySet; private numericLocals = new Map(); + /** Local arrays of the current function that no other binding can hold + * (ir/array-length-stability.ts). */ + private unaliasedArrays: ReadonlySet = new Set(); + /** Dense array loops (dense-array-loops.ts) run in this function. */ + private denseLoopsEnabled = false; + private denseLoopEntries: ReturnType = new Map(); + /** The fast copy of a dense array loop being emitted. */ + private denseLoop: DenseArrayLoop | null = null; private captureIds = new Set(); /** Active induction bindings retain their width independently of size_t. */ integerLoopBindings = new Map< @@ -1143,7 +1181,7 @@ export class LlEmitter { options: LlvmTargetOptions, ) { this.stackTraces = observesStacks(mod); - this.nullableUnions = new NullableUnions(mod); + this.nullableUnions = new NullableUnions(mod, options.debugSources === undefined); this.mockTimers = moduleUsesMockTimers(mod); this.debug = options.debugSources === undefined @@ -1245,8 +1283,10 @@ export class LlEmitter { // Optional array reads also cover nullable unions: emitLocalArrayRead // then yields the (borrowed or owned) element pointer, not a stack box. // Arrays of such a union are read the same way. - this.optionalArrayReads = new OptionalArrayReads(this.fnByName, this.unionsById, (id) => - this.nullableUnions.has(id), + this.optionalArrayReads = new OptionalArrayReads( + this.fnByName, + this.unionsById, + (id) => this.nullableUnions.single(id) !== null, ); this.unionSources = { unions: this.boxedUnionsById, @@ -1299,7 +1339,8 @@ export class LlEmitter { (type) => this.plainReference(type) && type.kind !== "string" && - (type.kind !== "union" || this.nullableUnions.get(type.unionId)?.arm.kind !== "string"), + (type.kind !== "union" || + this.nullableUnions.single(type.unionId)?.arm.kind !== "string"), this.walkBorrowsByFunction.get(fn.name) ?? new Set(), ); if (rebound.size === 0) continue; @@ -1325,6 +1366,8 @@ export class LlEmitter { return shape !== undefined && !recordDynKeepsIdentity(shape); }, ); + this.lentStringParameters = + this.debug === null ? findLentStringParameters(this.fnByName, this.callLifetimes) : new Map(); this.constantCallbacks = findConstantCallbacks(mod, this.callLifetimes); // Integer clones signal a stack overflow through their result, which // needs the single-threaded guard; worker programs also poll for @@ -1350,6 +1393,27 @@ export class LlEmitter { const traced = computeTraced(mod); this.tracedShapes = traced.shapes; this.tracedUnions = traced.unions; + // A shape union's value is whichever record it holds, and the collector + // visits it as that record: when one arm carries a collector header, + // every arm must (nullable-unions.ts). Unions that share an arm pass a + // header on to each other's arms, so repeat until no union adds one. + const shapeUnions = this.nullableUnions.allShapeUnions(); + for (let changed = true; changed;) { + changed = false; + for (const union of shapeUnions) { + if ( + !this.tracedUnions.has(union.unionId) && + !union.arms.some((arm) => this.tracedShapes.has(`record:${arm.shapeId}`)) + ) + continue; + for (const arm of union.arms) { + const key = `record:${arm.shapeId}`; + if (this.tracedShapes.has(key)) continue; + this.tracedShapes.add(key); + changed = true; + } + } + } this.shapeHost = { sizeType: this.sizeType, cycleBufOffset: this.cycleBufOffset, @@ -3690,6 +3754,7 @@ export class LlEmitter { return [ ...late, ...emitNullableRetainWrappers(this.shapeHost), + ...emitShapeUnionHelpers(this.shapeHost), ...emitInlineRcHelpers(this.shapeHost), ]; } @@ -5857,7 +5922,11 @@ export class LlEmitter { const nullable = this.nullableUnions.get(unionId); if (nullable) { // The owned payload IS the nullable union's value. - if (tag !== nullable.refTag) + if ( + nullable.repr === "single" + ? tag !== nullable.refTag + : !nullable.arms.some((arm) => arm.tag === tag) + ) throw new InternalCompilerError( `llvm emitter bug: payload wrap into unit arm ${tag} of ${unionId}`, ); @@ -5962,7 +6031,7 @@ export class LlEmitter { const index = shape.fields.findIndex((entry) => entry.name === field); if (index < 0) throw new InternalCompilerError(`llvm emitter bug: missing union field ${field}`); - layout = "record:"; + layout = this.nullableUnions.fieldBase(arm.shapeId) === 2 ? "record:tagged;" : "record:"; for (let i = 0; i <= index; i++) { const type = shape.fields[i]!.type; layout += `${llFieldType(type, this.inlineUnions.record(type))};`; @@ -5998,7 +6067,7 @@ export class LlEmitter { ); const p = this.B.tmp(); this.B.line( - `${p} = getelementptr inbounds %${mangleRecordStruct(shapeId)}, ptr ${objName}, i64 0, i32 ${idx + 1}`, + `${p} = getelementptr inbounds %${mangleRecordStruct(shapeId)}, ptr ${objName}, i64 0, i32 ${idx + this.nullableUnions.fieldBase(shapeId)}`, ); const type = shape.fields[idx]!.type; return { ptr: p, type, inline: this.inlineUnions.record(type) }; @@ -6012,7 +6081,7 @@ export class LlEmitter { const p = this.B.tmp(); const v = this.B.tmp(); this.B.line( - `${p} = getelementptr inbounds %${mangleRecordStruct(shapeId)}, ptr ${objName}, i64 0, i32 ${shape.fields.length + 1}`, + `${p} = getelementptr inbounds %${mangleRecordStruct(shapeId)}, ptr ${objName}, i64 0, i32 ${shape.fields.length + this.nullableUnions.fieldBase(shapeId)}`, ); this.B.line(`${v} = load ptr, ptr ${p}`); return v; @@ -6675,6 +6744,7 @@ export class LlEmitter { const initializerBindings = this.initializerBindings.get(fn.name) ?? []; const numericFn = withInitializerBindings(fn, initializerBindings); this.numericLocals = new Map(numericFn.locals.map((l) => [l.id, l])); + this.unaliasedArrays = findUnaliasedLocalArrays(fn); this.borrowedParameters.clear(); this.referenceWrites?.enter(fn); this.reboundOwnerSlots.clear(); @@ -6713,6 +6783,12 @@ export class LlEmitter { this.finalFieldBorrows = !fn.async && !fn.generator && constructorClass(fn.name) === null && this.finalFields.size > 0; this.byteWindowsEnabled = this.countedLoopsEnabled; + this.denseLoopsEnabled = this.countedLoopsEnabled; + this.denseLoop = null; + this.denseLoopEntries = + this.denseLoopsEnabled && numericFn.locals.some((l) => l.type.kind === "array") + ? findInitializedByteLoopBindings(fn) + : new Map(); this.byteWindowEntries = this.byteWindowsEnabled && numericFn.locals.some((l) => l.type.kind === "bytes") ? findInitializedByteLoopBindings(fn) @@ -6750,6 +6826,15 @@ export class LlEmitter { ); this.scalarStringSlices = this.debug === null ? findScalarStringSlices(fn, this.callLifetimes) : new Map(); + this.scratchStringSlices = + this.debug === null + ? findScratchStringSlices( + fn, + this.callLifetimes, + this.scalarStringSlices, + this.lentStringParameters, + ) + : new Set(); this.mapReadLifetimes = findMapReadLifetimes(fn, this.boxedUnionsById, this.callLifetimes); this.readNumberMapWrites = findReadNumberMapWrites(fn); this.mapEntryFusion = this.debug === null ? findMapEntryFusions(fn) : null; @@ -8211,7 +8296,16 @@ export class LlEmitter { const acc = elemAccess(s.arr.type.elem); this.emitPublishedArrayGuard(arr.name, s.arr.type, 0, idx.name); if (acc === "ref") this.moveTemp(v); - emitDenseArraySet(this, arr.name, idx, s.index, acc, s.arr.type.elem, v.name); + emitDenseArraySet( + this, + arr.name, + idx, + s.index, + acc, + s.arr.type.elem, + v.name, + this.provenDenseIndex(s.arr, s.index), + ); break; } case "arraySetLength": { @@ -8658,7 +8752,12 @@ export class LlEmitter { (this.debug === null ? matchIntegerBytesForLoop(s, this.numericLocals) : null); const countedLoop = !integerLoop && this.countedLoopsEnabled - ? matchIntegerCountedForLoop(s, this.numericLocals, this.integerRanges) + ? matchIntegerCountedForLoop( + s, + this.numericLocals, + this.integerRanges, + this.unaliasedArrays, + ) : null; const window = countedLoop && this.byteWindowsEnabled @@ -8671,6 +8770,49 @@ export class LlEmitter { this.byteWindowEntries.get(s) ?? new Set(), ) : null; + const denseShape = !this.denseLoopsEnabled + ? null + : integerArrayLoop + ? { + counter: integerArrayLoop.localId, + length: integerArrayLoop.limitReceiver as IrExpr & { kind: "varRef" }, + range: { min: 0, max: 4294967294 }, + } + : countedLoop && !countedLoop.guarded + ? countedDenseShape(s, countedLoop) + : null; + const dense = + denseShape && !window + ? matchDenseArrayLoop( + s, + denseShape, + this.numericLocals, + this.unaliasedArrays, + this.denseLoopEntries.get(s) ?? new Set(), + ) + : null; + if (dense) { + const valid = emitDenseLoopGuard(this, dense); + const fast = B.newLabel("dense.loop.fast"), + slow = B.newLabel("dense.loop.slow"), + done = B.newLabel("dense.loop.done"); + B.condBr(valid, fast, slow); + this.denseLoopsEnabled = false; + try { + B.startBlock(slow); + this.emitStmt(s); + B.br(done); + B.startBlock(fast); + this.denseLoop = dense; + this.emitStmt(s); + B.br(done); + } finally { + this.denseLoop = null; + this.denseLoopsEnabled = true; + } + B.startBlock(done); + break; + } if (window) { const valid = emitByteWindowGuard(this, window); const fast = B.newLabel("bytes.window.fast"), @@ -8773,7 +8915,10 @@ export class LlEmitter { this.integerLoopBindings.set(integerLoop.localId, { slot: integerSlot, type: this.sizeType, - range: { min: 0, max: integerArrayLoop ? 4294967294 : Number.MAX_SAFE_INTEGER - 1 }, + range: + this.denseLoop?.counter === integerLoop.localId + ? this.denseLoop.range + : { min: 0, max: integerArrayLoop ? 4294967294 : Number.MAX_SAFE_INTEGER - 1 }, }); } else if (countedLoop) { integerSlot = B.slot(); @@ -8788,7 +8933,10 @@ export class LlEmitter { slot: integerSlot, type: "i64", signed: true, - range: countedLoop.range, + range: + this.denseLoop?.counter === countedLoop.localId + ? this.denseLoop.range + : countedLoop.range, }); countedLimit = emitCountedLoopLimit(this, countedLoop); } else if (s.init) { @@ -10131,7 +10279,12 @@ export class LlEmitter { unionsById: this.unionsById, recordsById: this.recordsById, inlineUnions: this.inlineUnions, - nullableArm: (t) => this.nullableUnions.of(t)?.arm ?? null, + nullableArm: (t) => { + const nullable = this.nullableUnions.of(t); + return nullable?.repr === "single" ? nullable.arm : null; + }, + shapeUnion: (t) => this.nullableUnions.of(t)?.repr === "shapes", + fieldBase: (shapeId) => this.nullableUnions.fieldBase(shapeId), classInfo: (className) => { const meta = this.classMetaOf(className); return { @@ -10852,7 +11005,11 @@ export class LlEmitter { const nullable = this.nullableUnions.get(e.unionId); if (!nullable) return null; if (isUnitType(e.value.type)) return e.value.kind === "unitLit" ? "unit" : null; - return e.tag === nullable.refTag && isRefCounted(e.value.type) ? "ref" : null; + const refArm = + nullable.repr === "single" + ? e.tag === nullable.refTag + : nullable.arms.some((arm) => arm.tag === e.tag); + return refArm && isRefCounted(e.value.type) ? "ref" : null; } materializeSplitLocal(localId: string): void { diff --git a/packages/compiler/src/backend/llvm/expr-callbacks.ts b/packages/compiler/src/backend/llvm/expr-callbacks.ts index e5560f14f..d71df333a 100644 --- a/packages/compiler/src/backend/llvm/expr-callbacks.ts +++ b/packages/compiler/src/backend/llvm/expr-callbacks.ts @@ -182,6 +182,7 @@ export function genResultThunkFor( sym = mangleGenResThunk(host.resolveThunks.size); host.resolveThunks.set(key, sym); const shape = host.recordsById.get(recT.shapeId); + const base = host.nullableUnions.fieldBase(recT.shapeId); const valueT = shape?.fields.find((f) => f.name === "value")?.type; if (!shape || !valueT) throw new InternalCompilerError("llvm emitter bug: genResume record lacks its value field"); @@ -242,7 +243,7 @@ export function genResultThunkFor( ` %r = call ptr @${mangleRecordNew(recT.shapeId)}()`, ` %d = call zeroext i1 @scr_gen_done(ptr %g)`, ` %dz = zext i1 %d to i8`, - ` %dp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${doneIdxD + 1}`, + ` %dp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${doneIdxD + base}`, ` store i8 %dz, ptr %dp`, ` %has = call zeroext i1 @scr_gen_out_has(ptr %g)`, ` br i1 %has, label %take, label %undefv`, @@ -259,7 +260,7 @@ export function genResultThunkFor( ` br label %join`, `join:`, ` %v = load ptr, ptr %vslot`, - ` %vp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${valueIdxD + 1}`, + ` %vp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${valueIdxD + base}`, ` store ptr %v, ptr %vp`, ` ret ptr %r`, `}`, @@ -293,7 +294,7 @@ export function genResultThunkFor( ` %r = call ptr @${mangleRecordNew(recT.shapeId)}()`, ` %d = call zeroext i1 @scr_gen_done(ptr %g)`, ` %dz = zext i1 %d to i8`, - ` %dp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${doneIdx + 1}`, + ` %dp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${doneIdx + base}`, ` store i8 %dz, ptr %dp`, ` br i1 %d, label %doneb, label %susp`, ]; @@ -410,7 +411,7 @@ export function genResultThunkFor( ` br label %join`, `join:`, ` %v = load ptr, ptr %vslot`, - ` %vp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${valueIdx + 1}`, + ` %vp = getelementptr inbounds %${mangleRecordStruct(recT.shapeId)}, ptr %r, i64 0, i32 ${valueIdx + base}`, // An inline value field copies the box's bits and drops the box. ...(host.inlineUnions.record(valueT) ? [ @@ -1092,7 +1093,8 @@ export function settledStoreFor( return { tag, arm }; }; const fieldIndex = (arm: IrType & { kind: "record" }, field: string): number => - host.recordsById.get(arm.shapeId)!.fields.findIndex((f) => f.name === field) + 1; + host.recordsById.get(arm.shapeId)!.fields.findIndex((f) => f.name === field) + + host.nullableUnions.fieldBase(arm.shapeId); const fulfilled = armOf("value"); const rejected = armOf("reason"); const valueT = host.recordsById diff --git a/packages/compiler/src/backend/llvm/expr-containers.ts b/packages/compiler/src/backend/llvm/expr-containers.ts index 9c77690f8..22a0b4df5 100644 --- a/packages/compiler/src/backend/llvm/expr-containers.ts +++ b/packages/compiler/src/backend/llvm/expr-containers.ts @@ -42,7 +42,7 @@ import { } from "./map-number-lookup.js"; import { emitBorrowedInput } from "./borrowed-inputs.js"; import { emitFusedMapSet, rememberMapEntry } from "./map-entry-fusion.js"; -import { emitStringSliceRead } from "./string-slices.js"; +import { emitScratchStringSlice, emitStringSliceRead } from "./string-slices.js"; import { STRING_CHAR_CODE_AT_DECL, STRING_CHAR_CODE_AT_INT_MARKER, @@ -395,6 +395,7 @@ export function emitStrIntrinsic( false, ); case "slice": + if (host.scratchStringSlices.has(e)) return emitScratchStringSlice(host, e, r, args); return call( "scr_str_slice", "ptr (ptr, double, double)", @@ -403,6 +404,7 @@ export function emitStrIntrinsic( true, ); case "substring": + if (host.scratchStringSlices.has(e)) return emitScratchStringSlice(host, e, r, args); return call( "scr_str_substring", "ptr (ptr, double, double)", @@ -575,7 +577,16 @@ export function emitArrIntrinsic( ? host.constantNumericTables.get(e.receiver.localId) : undefined; if (!table && elem.kind === "f64") - return { name: emitDenseArrayNumber(host, r.name, index, e.args[0]), type: e.type }; + return { + name: emitDenseArrayNumber( + host, + r.name, + index, + e.args[0], + host.provenDenseIndex(e.receiver, e.args[0]!), + ), + type: e.type, + }; host.declare(`declare double @scr_arr_get_number(ptr, double)`); const t = B.tmp(); B.line( diff --git a/packages/compiler/src/backend/llvm/expr-control.ts b/packages/compiler/src/backend/llvm/expr-control.ts index 298ea0199..fd4bc8787 100644 --- a/packages/compiler/src/backend/llvm/expr-control.ts +++ b/packages/compiler/src/backend/llvm/expr-control.ts @@ -330,7 +330,7 @@ export function emitControlExpr( ); B.line(`store ${ty} ${t}, ptr ${slot}`); if (host.mayThrow.has(e.retag)) host.emitPendingCheck(); - } else if (host.nullableUnions.has(e.left.type.unionId)) { + } else if (host.nullableUnions.single(e.left.type.unionId)) { // The owned nullable pointer IS the owned arm value. B.line(`store ${ty} ${l.name}, ptr ${slot}`); } else { @@ -440,7 +440,7 @@ export function emitControlExpr( host.emitBranchInto(slot, e.right); B.br(lj); B.startBlock(lv); - if (typeEquals(e.type, e.left.type) || host.nullableUnions.has(def.id)) { + if (typeEquals(e.type, e.left.type) || host.nullableUnions.single(def.id)) { // Pass-through, or a nullable pointer that IS its owned arm value. B.line(`store ${ty} ${l.name}, ptr ${slot}`); } else { diff --git a/packages/compiler/src/backend/llvm/expr-primitives.ts b/packages/compiler/src/backend/llvm/expr-primitives.ts index aaae4c35c..e237f8f13 100644 --- a/packages/compiler/src/backend/llvm/expr-primitives.ts +++ b/packages/compiler/src/backend/llvm/expr-primitives.ts @@ -64,16 +64,25 @@ export function emitLiteralExpr( const integer = B.tmp(); const number = B.tmp(); B.line(`${integer} = load ${integerSlot.type}, ptr ${integerSlot.slot}`); - // A 64-bit induction value is a safe integer, so its sign bit is - // clear and the signed conversion is exact: one cvtsi2sd instead of - // x86-64's multi-instruction unsigned 64-bit conversion. - const signedConversion = integerSlot.signed === true || integerSlot.type === "i64"; - B.line( - `${number} = ${signedConversion ? "sitofp" : "uitofp"} ${integerSlot.type} ${integer} to double`, - ); const uint32 = integerSlot.type === "i32" ? integer : B.tmp(); if (integerSlot.type !== "i32") B.line(`${uint32} = trunc ${integerSlot.type} ${integer} to i32`); + // An int32 counter (a dense array loop's) converts from its low half, + // so LLVM folds a later `fptosi` of the double (an inlined callee's + // index) back to the integer. Otherwise a 64-bit induction value is + // a safe integer, so its sign bit is clear and the signed conversion + // is exact: one cvtsi2sd instead of x86-64's multi-instruction + // unsigned 64-bit conversion. (A uint32 counter would fold through + // `uitofp i32` too, but the zero extension it adds on x86-64 cost + // more in loops that never convert the double than it saved.) + const range = integerSlot.range; + const signedConversion = integerSlot.signed === true || integerSlot.type === "i64"; + if (range && range.min >= -2147483648 && range.max <= 2147483647) + B.line(`${number} = sitofp i32 ${uint32} to double`); + else + B.line( + `${number} = ${signedConversion ? "sitofp" : "uitofp"} ${integerSlot.type} ${integer} to double`, + ); return { name: number, type: e.type, @@ -851,6 +860,7 @@ export function emitContainerExpr( elem, `scr_arr_get_${acc}`, true, + host.provenDenseIndex(e.arr, e.index), ); return host.own({ name: t, type: e.type }); } diff --git a/packages/compiler/src/backend/llvm/expr-records.ts b/packages/compiler/src/backend/llvm/expr-records.ts index fc9df6daf..3318ae774 100644 --- a/packages/compiler/src/backend/llvm/expr-records.ts +++ b/packages/compiler/src/backend/llvm/expr-records.ts @@ -62,9 +62,17 @@ export function emitUnionRetype( const from = host.nullableUnions.get(e.fromId); const to = host.nullableUnions.get(e.unionId); if (from && to) { - // Equal encodings make the pointer itself the target value. - if (from.refTag === to.refTag && from.unitTag === to.unitTag && from.nullTag === to.nullTag) - return host.own({ name: value.name, type: e.type }); + // Equal encodings make the pointer itself the target value. A shape + // union derives its record arms from the record, so only the unit + // encodings must agree. + const same = + from.repr === "single" && to.repr === "single" + ? from.refTag === to.refTag && from.unitTag === to.unitTag && from.nullTag === to.nullTag + : from.repr === "shapes" && + to.repr === "shapes" && + from.unitTag === to.unitTag && + from.nullTag === to.nullTag; + if (same) return host.own({ name: value.name, type: e.type }); return host.own({ name: rebuildRetype(host, value.name, e), type: e.type }); } if (from || to) return host.own({ name: rebuildRetype(host, value.name, e), type: e.type }); diff --git a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts index 55fa363cd..76ef35358 100644 --- a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts +++ b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts @@ -500,16 +500,17 @@ export function streamTypedRefCommitAdapter( ` br i1 %missing, label %done, label %swap`, `swap:`, ]; + const base = host.nullableUnions.fieldBase(t.shapeId); const members = [ ...shape.fields.map((field, index) => ({ - index: index + 1, + index: index + base, type: llFieldType(field.type, host.inlineUnions.record(field.type)), name: field.name, })), ...(shape.indexValue ? [ { - index: shape.fields.length + 1, + index: shape.fields.length + base, type: "ptr" as const, name: "[key: string] overflow", }, @@ -943,7 +944,7 @@ export function streamTypedRefMaterializeAdapter( B.line(`${out} = call ptr @scr_dyn_new_arr()`); const fields = [...shape.fields].sort((a, b) => Number(a.name) - Number(b.name)); for (const field of fields) { - const index = shape.fields.indexOf(field) + 1; + const index = shape.fields.indexOf(field) + host.nullableUnions.fieldBase(t.shapeId); const fieldPtr = B.tmp(); B.line( `${fieldPtr} = getelementptr inbounds %${mangleRecordStruct(t.shapeId)}, ptr %p, i64 0, i32 ${index}`, @@ -980,7 +981,7 @@ export function streamTypedRefMaterializeAdapter( liveRecordOps(host, t, snapshot, fields); } for (const field of fields) { - const index = shape.fields.indexOf(field) + 1; + const index = shape.fields.indexOf(field) + host.nullableUnions.fieldBase(t.shapeId); const fieldPtr = B.tmp(); B.line( `${fieldPtr} = getelementptr inbounds %${mangleRecordStruct(t.shapeId)}, ptr %p, i64 0, i32 ${index}`, @@ -1202,7 +1203,7 @@ function liveRecordOps( const names = `${snapshot}_names`; const ops = `${snapshot}_rops`; const loadField = (B: BlockBuilder, field: IrRecordShape["fields"][number]): string => { - const index = shape.fields.indexOf(field) + 1; + const index = shape.fields.indexOf(field) + host.nullableUnions.fieldBase(t.shapeId); const fieldPtr = B.tmp(); B.line(`${fieldPtr} = getelementptr inbounds %${struct}, ptr %p, i64 0, i32 ${index}`); return loadStructField( diff --git a/packages/compiler/src/backend/llvm/lib-datagram.ts b/packages/compiler/src/backend/llvm/lib-datagram.ts index c97744f64..3fae845e2 100644 --- a/packages/compiler/src/backend/llvm/lib-datagram.ts +++ b/packages/compiler/src/backend/llvm/lib-datagram.ts @@ -29,7 +29,7 @@ function messageAdapter(host: LlvmEmitterContext, type: IrType): string { const isString = field === "address" || field === "family"; if (isString) lines.push(` %${field}.owned = call ptr @scr_str_retain_v(ptr %${field})`); lines.push( - ` %${field}.slot = getelementptr inbounds %${mangleRecordStruct(type.shapeId)}, ptr %record, i64 0, i32 ${index + 1}`, + ` %${field}.slot = getelementptr inbounds %${mangleRecordStruct(type.shapeId)}, ptr %record, i64 0, i32 ${index + host.nullableUnions.fieldBase(type.shapeId)}`, ` store ${isString ? "ptr" : "double"} %${field}${isString ? ".owned" : ""}, ptr %${field}.slot`, ); } @@ -169,7 +169,7 @@ export function emitDatagramLibCall(host: LlvmEmitterContext, e: LibCallExpr): L ); const slot = host.B.tmp(); host.B.line( - `${slot} = getelementptr inbounds %${mangleRecordStruct(type.shapeId)}, ptr ${record}, i64 0, i32 ${index + 1}`, + `${slot} = getelementptr inbounds %${mangleRecordStruct(type.shapeId)}, ptr ${record}, i64 0, i32 ${index + host.nullableUnions.fieldBase(type.shapeId)}`, ); host.B.line(`store ${ty} ${value}, ptr ${slot}`); } diff --git a/packages/compiler/src/backend/llvm/lib-dispatch.ts b/packages/compiler/src/backend/llvm/lib-dispatch.ts index fdfe723d2..68d943ea1 100644 --- a/packages/compiler/src/backend/llvm/lib-dispatch.ts +++ b/packages/compiler/src/backend/llvm/lib-dispatch.ts @@ -774,6 +774,8 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue { case "blob": case "record": case "disposable": + case "punycode": + case "constants": return host.emitGenericLibCall(e); case "test": return emitTestLibCall(host, e); diff --git a/packages/compiler/src/backend/llvm/lib-filesystem.ts b/packages/compiler/src/backend/llvm/lib-filesystem.ts index 0e7d02a15..74278789c 100644 --- a/packages/compiler/src/backend/llvm/lib-filesystem.ts +++ b/packages/compiler/src/backend/llvm/lib-filesystem.ts @@ -912,11 +912,8 @@ export function emitPathUrlLibCall(host: LlvmEmitterContext, e: LibCallExpr): Ll scopeid.inline, ); } - const rc = vAdapters(host.shapeHost, t); - const rowU = B.tmp(); - B.line( - `${rowU} = call ptr @scr_union_new_ref(i32 ${tag}, ptr ${r}, ptr ${rc.retain}, ptr ${rc.release}, ptr ${traceArg(host.shapeHost, t)})`, - ); + // The row (+1) moves into its union arm (a shape union is the row). + const rowU = host.unionNewOwned(infoT.unionId, tag, { name: r, type: t }); B.line(`store ptr ${rowU}, ptr ${rowSlot}`); B.br(lRow); }; diff --git a/packages/compiler/src/backend/llvm/lib-lookup.ts b/packages/compiler/src/backend/llvm/lib-lookup.ts index 9fc23cb94..a6e76e4e0 100644 --- a/packages/compiler/src/backend/llvm/lib-lookup.ts +++ b/packages/compiler/src/backend/llvm/lib-lookup.ts @@ -65,7 +65,7 @@ export function emitLookupConnect(host: LlvmEmitterContext, e: LibCallExpr): LlV " br i1 %more, label %body, label %finish", "body:", " %record = call ptr @scr_arr_get_ref(ptr %addresses, double %index)", - ` %addressp = getelementptr inbounds %${struct}, ptr %record, i64 0, i32 ${field + 1}`, + ` %addressp = getelementptr inbounds %${struct}, ptr %record, i64 0, i32 ${field + host.nullableUnions.fieldBase(addresses.elem.shapeId)}`, " %address = load ptr, ptr %addressp", " %owned.address = call ptr @scr_str_retain_v(ptr %address)", " call double @scr_arr_push_ref(ptr %ips, ptr %owned.address)", diff --git a/packages/compiler/src/backend/llvm/lib-network.ts b/packages/compiler/src/backend/llvm/lib-network.ts index 3ac9fe050..322479a85 100644 --- a/packages/compiler/src/backend/llvm/lib-network.ts +++ b/packages/compiler/src/backend/llvm/lib-network.ts @@ -168,7 +168,7 @@ export function emitNetworkHttpLibCall(host: LlvmEmitterContext, e: LibCallExpr) const i = shape.fields.findIndex((f) => f.name === name); if (i < 0) throw new InternalCompilerError(`llvm emitter bug: net.serverAddress record lacks ${name}`); - return i + 1; + return i + host.nullableUnions.fieldBase(recT.shapeId); }; const store = (name: string, ty: string, v: string): void => { const p = B.tmp(); diff --git a/packages/compiler/src/backend/llvm/lib-shared.ts b/packages/compiler/src/backend/llvm/lib-shared.ts index 02bf37192..6b2f739a8 100644 --- a/packages/compiler/src/backend/llvm/lib-shared.ts +++ b/packages/compiler/src/backend/llvm/lib-shared.ts @@ -194,6 +194,7 @@ export const LIB_FN_SYMS: Record = { "os.cpusModel": "scr_os_cpus_model", "os.cpusField": "scr_os_cpus_field", "os.constants": "scr_os_constants", + "constants.object": "scr_node_constants", "os.release": "scr_os_release", "os.userName": "scr_os_user_name", "os.userShell": "scr_os_user_shell", @@ -638,6 +639,13 @@ export const LIB_FN_SYMS: Record = { "url.legacyFormat": "scr_url_legacy_format", "url.legacyResolve": "scr_url_legacy_resolve", "url.domainToASCII": "scr_url_domain_to_ascii", + "punycode.decode": "scr_punycode_decode", + "punycode.encode": "scr_punycode_encode", + "punycode.toASCII": "scr_punycode_to_ascii", + "punycode.toUnicode": "scr_punycode_to_unicode", + "punycode.ucs2Decode": "scr_punycode_ucs2_decode", + "punycode.ucs2Encode": "scr_str_from_code_point", + "punycode.warn": "scr_punycode_warn", "url.domainToUnicode": "scr_url_domain_to_unicode", "url.pathToFileURL": "scr_url_from_path", "url.pathToFileURLPlatform": "scr_url_from_path_platform", diff --git a/packages/compiler/src/backend/llvm/nullable-unions.test.ts b/packages/compiler/src/backend/llvm/nullable-unions.test.ts new file mode 100644 index 000000000..46bd688e7 --- /dev/null +++ b/packages/compiler/src/backend/llvm/nullable-unions.test.ts @@ -0,0 +1,83 @@ +import { expect, test } from "vitest"; +import { + F64, + NULL_T, + UNDEFINED_T, + type IrModule, + type IrType, + type IrUnionDef, +} from "../../ir/ir.js"; +import { NullableUnions } from "./nullable-unions.js"; + +const loc = { file: "nullable-unions.ts", start: 0, end: 0 }; +const a: IrType = { kind: "record", shapeId: "a" }; +const b: IrType = { kind: "record", shapeId: "b" }; +const c: IrType = { kind: "record", shapeId: "c" }; + +function module(unions: IrUnionDef[], extra: Partial = {}): IrModule { + return { + irVersion: 16, + sourceFile: loc.file, + entry: "main", + records: ["a", "b", "c"].map((id) => ({ id, fields: [{ name: id, type: F64 }] })), + unions, + functions: [], + ...extra, + }; +} + +test("unions of two or more records become shape unions", () => { + const unions = new NullableUnions( + module([ + { id: "ab", arms: [a, b] }, + { id: "abu", arms: [a, UNDEFINED_T, b] }, + { id: "abnu", arms: [a, b, NULL_T, UNDEFINED_T] }, + { id: "af", arms: [a, F64] }, + { id: "au", arms: [a, UNDEFINED_T] }, + ]), + ); + expect(unions.shapes("ab")).toMatchObject({ unitTag: -1, nullTag: -1 }); + expect(unions.shapes("abu")).toMatchObject({ unitTag: 1, nullTag: -1 }); + expect(unions.shapes("abnu")).toMatchObject({ unitTag: 3, nullTag: 2 }); + expect(unions.shapes("abnu")?.arms).toEqual([ + { tag: 0, shapeId: "a" }, + { tag: 1, shapeId: "b" }, + ]); + // A scalar arm keeps the box; one record arm is a single-arm nullable. + expect(unions.get("af")).toBeNull(); + expect(unions.single("au")).not.toBeNull(); + expect(unions.shapes("au")).toBeNull(); + // Tagged shapes number from 2 and move their fields after the word. + expect([unions.shapeNumber("a"), unions.shapeNumber("b"), unions.shapeNumber("c")]).toEqual([ + 2, 3, 0, + ]); + expect([unions.fieldBase("a"), unions.fieldBase("c")]).toEqual([2, 1]); + expect(unions.boxedUnions(new Map([["af", { id: "af", arms: [a, F64] }]])).size).toBe(1); +}); + +test("records of runtime-inspected unions keep every union over them boxed", () => { + const set: IrType = { kind: "set", elem: { kind: "union", unionId: "ab" } }; + const unions = new NullableUnions( + module( + [ + { id: "ab", arms: [a, b] }, + { id: "abu", arms: [a, b, UNDEFINED_T] }, + { id: "bc", arms: [b, c] }, + { id: "ac", arms: [a, c] }, + ], + { globals: [{ id: "%g.s", name: "s", type: set, mutable: false }] }, + ), + ); + expect(unions.get("ab")).toBeNull(); + expect(unions.get("abu")).toBeNull(); + expect(unions.get("bc")).toBeNull(); + expect(unions.get("ac")).toBeNull(); + expect(unions.shapeNumber("c")).toBe(0); +}); + +test("library, lifetime-region and debug builds keep record unions boxed", () => { + const unions = [{ id: "ab", arms: [a, b] }]; + expect(new NullableUnions(module(unions), false).get("ab")).toBeNull(); + expect(new NullableUnions(module(unions, { lifetimeRegions: true })).get("ab")).toBeNull(); + expect(new NullableUnions(module(unions)).shapes("ab")).not.toBeNull(); +}); diff --git a/packages/compiler/src/backend/llvm/nullable-unions.ts b/packages/compiler/src/backend/llvm/nullable-unions.ts index b1032f9d4..8729e9649 100644 --- a/packages/compiler/src/backend/llvm/nullable-unions.ts +++ b/packages/compiler/src/backend/llvm/nullable-unions.ts @@ -17,6 +17,7 @@ import { everyModuleNode } from "../../ir/traverse.js"; * every container, field, capture, parameter, and return slot holds the * plain pointer. Every other union keeps the tagged `%ScrUnion` box. */ export interface NullableUnion { + repr: "single"; unionId: string; refTag: number; /** The unit arm NULL encodes. */ @@ -24,7 +25,57 @@ export interface NullableUnion { /** The `null` arm of a two-unit-arm union, encoded by NULLABLE_NULL; * -1 when the union has one unit arm. */ nullTag: number; - arm: NullableRefArm; + /** One of NullableRefArm. Typed as the plain IrType so the self-hosted + * frontend keeps IrType's structurally identical arms one record shape + * (a PointerUnion arm holding the narrowed intersection would split them). */ + arm: IrType; +} + +/** A union of two or more record arms and at most two unit arms (`A | B`, + * `A | B | undefined`, discriminated unions of object literals) whose + * VALUES are the record pointer itself, like a NullableUnion: NULL is the + * unit arm (`undefined` when both exist, NULLABLE_NULL is then `null`). + * Every record shape that is an arm of such a union is a TAGGED shape: its + * struct carries the shape's module-wide number in a hidden `i32` word + * right after the reference count (struct index 1), stored by every + * allocation, so the arm of a pointer is that word mapped through the + * union's arm table (the per-union `@sc_stag.` helper). Records share + * one retain; release and trace dispatch on the arm. */ +export interface ShapeUnion { + repr: "shapes"; + unionId: string; + /** The unit arm NULL encodes; -1 when the union has no unit arm. */ + unitTag: number; + /** The `null` arm of a two-unit-arm union, encoded by NULLABLE_NULL; + * -1 otherwise. */ + nullTag: number; + /** The record arms in tag order. */ + arms: readonly { tag: number; shapeId: string }[]; +} + +/** A union whose values are plain pointers (no `%ScrUnion` box). */ +export type PointerUnion = NullableUnion | ShapeUnion; + +/** The struct index of a tagged record shape's hidden shape-number word. */ +export const SHAPE_WORD_INDEX = 1; + +/** The LLVM type through which any tagged record's shape word is read + * (the reference count, then the word). */ +export function shapeWordHeader(sizeType: "i32" | "i64"): string { + return `{ ${sizeType}, i32 }`; +} + +/** The per-union helper mapping a shape-union value to its tag. */ +export function shapeTagFn(unionId: string): string { + return `@sc_stag.${unionId.replace(/[^A-Za-z0-9_.]/g, "_")}`; +} + +/** The per-union release and trace dispatchers of a shape union. */ +export function shapeReleaseFn(unionId: string): string { + return `@sc_srelease.${unionId.replace(/[^A-Za-z0-9_.]/g, "_")}`; +} +export function shapeTraceFn(unionId: string): string { + return `@sc_strace.${unionId.replace(/[^A-Za-z0-9_.]/g, "_")}`; } /** The module-scope immortal sentinel standing for `null` in a @@ -48,44 +99,98 @@ export type NullableRefArm = | (IrType & { kind: "array" | "map" | "set" }); /** The arm is a JS object: always truthy, compared by identity. */ -export function isObjectArm(nullable: NullableUnion): boolean { - return nullable.arm.kind !== "string"; +export function isObjectArm(nullable: PointerUnion): boolean { + return nullable.repr === "shapes" || nullable.arm.kind !== "string"; } /** The representation decision for every union of one module. It is a * pure function of the module, so serialized-IR and in-memory emission * agree. */ export class NullableUnions { - private readonly byId = new Map(); + private readonly byId = new Map(); + /** Tagged record shapes (arms of a shape union) by their module-wide + * shape number (2 and up: 0 and 1 are the unit encodings' pseudo + * numbers, never stored). */ + private readonly shapeNumbers = new Map(); /** NULL-skipping retain wrappers requested so far: symbol → inner retain * call target. The emitter defines each once. */ readonly retainWrappers = new Map(); /** Some nullable union encodes `null` with NULLABLE_NULL. */ readonly usesNullSentinel: boolean = false; - constructor(mod: IrModule) { + /** `shapes`: allow shape unions (debug builds keep boxes, whose + * debug-info layouts describe every arm). */ + constructor(mod: IrModule, shapes = true) { const emitted = new Set(); for (const cls of mod.classes ?? []) if (!cls.runtime) emitted.add(cls.name); const records = new Set(); for (const shape of mod.records ?? []) records.add(shape.id); const boxed = boxedUnionIds(mod); + // Shape unions stay out of library builds (their host ABI marshals + // boxes) and of programs with lifetime regions (whose walks report a + // reference slot by its type's release entry point). + const allowShapes = shapes && mod.lib === undefined && mod.lifetimeRegions !== true; + // A record that is an arm of a boxed union never becomes a shape + // union's arm: conversions between unions over the same records then + // keep one representation. + const boxedRecords = new Set(); + for (const def of mod.unions ?? []) + if (boxed.has(def.id)) + for (const arm of def.arms) if (arm.kind === "record") boxedRecords.add(arm.shapeId); for (const def of mod.unions ?? []) { if (boxed.has(def.id)) continue; - const nullable = nullableShape(def, emitted, records); + const nullable = + nullableShape(def, emitted, records) ?? + (allowShapes ? shapeUnion(def, records, boxedRecords) : null); if (!nullable) continue; this.byId.set(def.id, nullable); if (nullable.nullTag >= 0) this.usesNullSentinel = true; + if (nullable.repr === "shapes") + for (const arm of nullable.arms) + if (!this.shapeNumbers.has(arm.shapeId)) + this.shapeNumbers.set(arm.shapeId, this.shapeNumbers.size + 2); } } - get(unionId: string): NullableUnion | null { + get(unionId: string): PointerUnion | null { return this.byId.get(unionId) ?? null; } - of(type: IrType): NullableUnion | null { + of(type: IrType): PointerUnion | null { return type.kind === "union" ? this.get(type.unionId) : null; } + /** The single-reference-arm form only (shape unions answer null). */ + single(unionId: string): NullableUnion | null { + const nullable = this.byId.get(unionId); + return nullable?.repr === "single" ? nullable : null; + } + + /** The shape-union form only. */ + shapes(unionId: string): ShapeUnion | null { + const nullable = this.byId.get(unionId); + return nullable?.repr === "shapes" ? nullable : null; + } + + /** Every shape union, in module order. */ + allShapeUnions(): ShapeUnion[] { + const out: ShapeUnion[] = []; + for (const nullable of this.byId.values()) if (nullable.repr === "shapes") out.push(nullable); + return out; + } + + /** The module-wide number a tagged record shape stores in its shape + * word, or 0 for shapes that carry none. */ + shapeNumber(shapeId: string): number { + return this.shapeNumbers.get(shapeId) ?? 0; + } + + /** The struct index of a record shape's first declared field: 2 for + * tagged shapes (after the shape word), 1 otherwise. */ + fieldBase(shapeId: string): number { + return this.shapeNumbers.has(shapeId) ? 2 : 1; + } + has(unionId: string): boolean { return this.byId.has(unionId); } @@ -94,10 +199,11 @@ export class NullableUnions { return this.byId.size; } - /** The reference arm of every nullable union, by union id. */ + /** The reference arm of every single-arm nullable union, by union id. */ armTypes(): Map { const result = new Map(); - for (const [id, nullable] of this.byId) result.set(id, nullable.arm); + for (const [id, nullable] of this.byId) + if (nullable.repr === "single") result.set(id, nullable.arm); return result; } @@ -129,7 +235,7 @@ function nullableShape( if (unitTag < 0 || nullTag < 0) return null; } const arm = def.arms[refTag]!; - const base = { unionId: def.id, refTag, unitTag, nullTag }; + const base = { repr: "single" as const, unionId: def.id, refTag, unitTag, nullTag }; if (arm.kind === "object" && emittedClasses.has(arm.className)) return { ...base, arm: { kind: "object", className: arm.className } }; if (arm.kind === "record" && records.has(arm.shapeId)) @@ -140,6 +246,34 @@ function nullableShape( return null; } +/** Two or more emitted record arms and at most two unit arms (both: + * undefined and null). */ +function shapeUnion( + def: IrUnionDef, + records: ReadonlySet, + boxedRecords: ReadonlySet, +): ShapeUnion | null { + const units = def.arms.filter(isUnitType).length; + if (units > 2 || def.arms.length - units < 2) return null; + const arms: { tag: number; shapeId: string }[] = []; + for (let tag = 0; tag < def.arms.length; tag++) { + const arm = def.arms[tag]!; + if (isUnitType(arm)) continue; + if (arm.kind !== "record" || !records.has(arm.shapeId) || boxedRecords.has(arm.shapeId)) + return null; + arms.push({ tag, shapeId: arm.shapeId }); + } + let unitTag = -1, + nullTag = -1; + if (units === 1) unitTag = def.arms.findIndex(isUnitType); + else if (units === 2) { + unitTag = def.arms.findIndex((arm) => arm.kind === "undefinedT"); + nullTag = def.arms.findIndex((arm) => arm.kind === "nullT"); + if (unitTag < 0 || nullTag < 0) return null; + } + return { repr: "shapes", unionId: def.id, unitTag, nullTag, arms }; +} + /** Unions the runtime itself inspects as tagged boxes keep that * representation: Map keys and Set elements hash and compare union keys by * tag and payload inside the C map. */ diff --git a/packages/compiler/src/backend/llvm/publish.ts b/packages/compiler/src/backend/llvm/publish.ts index c884339af..4f807424b 100644 --- a/packages/compiler/src/backend/llvm/publish.ts +++ b/packages/compiler/src/backend/llvm/publish.ts @@ -25,6 +25,7 @@ import type { IrClassDef, IrRecordShape, IrType, IrUnionDef, SrcLoc } from "../. import { mapOf, STRING, typeKey } from "../../ir/ir.js"; import { mangleClassStruct, mangleRecordStruct } from "../mangle.js"; import { FN_ATTRS } from "./shapes.js"; +import { shapeTagFn } from "./nullable-unions.js"; import type { InlineUnionFields } from "./inline-unions.js"; import { LlvmUnsupportedError } from "./unsupported.js"; @@ -44,6 +45,10 @@ export interface PublishHost { readonly inlineUnions: InlineUnionFields; /** The nullable-pointer representation of a union, when it has one. */ nullableArm(t: IrType): IrType | null; + /** The union's values are tagged record pointers (nullable-unions.ts). */ + shapeUnion(t: IrType): boolean; + /** The struct index of a record shape's first field. */ + fieldBase(shapeId: string): number; /** Class layout: hierarchy membership, preorder interval, field slots * (with their storage types), runtime roots. */ classInfo(className: string): PublishClassInfo; @@ -210,6 +215,19 @@ export class PublishWalkers { if (def.arms.some((arm) => scalarRefusal(arm))) throw new LlvmUnsupportedError(`publish of ${typeKey(t)}`, loc); const live = arms.flatMap((a) => (a.fn === null ? [] : [{ tag: a.tag, fn: a.fn }])); + if (this.host.shapeUnion(t)) { + // The value is the record itself (or a unit encoding): no box to + // mark; the arm's walker marks and walks the record. + const lines = [head, `entry:`]; + lines.push( + ` %tag = call i32 ${shapeTagFn(t.unionId)}(ptr %o)`, + ` switch i32 %tag, label %done [ ${live.map((a) => `i32 ${a.tag}, label %arm${a.tag}`).join(" ")} ]`, + ); + for (const a of live) + lines.push(`arm${a.tag}:`, ` call void ${a.fn}(ptr %o, ptr %ctx)`, ` br label %done`); + lines.push(`done:`, ` ret void`, `}`); + return lines; + } const lines = [ head, `entry:`, @@ -240,17 +258,18 @@ export class PublishWalkers { const shape = this.host.recordsById.get(t.shapeId); if (!shape) throw new LlvmUnsupportedError(`publish of an unknown record ${t.shapeId}`, loc); + const base = this.host.fieldBase(shape.id); const members = [ ...shape.fields.flatMap((f, i) => this.host.inlineUnions.record(f.type) ? [] - : [{ name: f.name, index: i + 1, type: f.type }], + : [{ name: f.name, index: i + base, type: f.type }], ), ...(shape.indexValue ? [ { name: "[key: string]", - index: shape.fields.length + 1, + index: shape.fields.length + base, type: mapOf(STRING, shape.indexValue), }, ] diff --git a/packages/compiler/src/backend/llvm/regions.ts b/packages/compiler/src/backend/llvm/regions.ts index 41f67c0cb..b789c644d 100644 --- a/packages/compiler/src/backend/llvm/regions.ts +++ b/packages/compiler/src/backend/llvm/regions.ts @@ -101,8 +101,8 @@ export class LifetimeRegions { /** Can a value of type `t` hold (or own something that holds) a * reference to a ProgramLifetime instance the walk can explain? */ private reaches(t: IrType): boolean { - const arm = this.host.nullableUnions.of(t)?.arm; - if (arm !== undefined) return this.reaches(arm); + const nullable = this.host.nullableUnions.of(t); + if (nullable?.repr === "single") return this.reaches(nullable.arm); switch (t.kind) { case "object": return this.isLifetime(t.className) || this.reach.get(`object:${t.className}`) === true; @@ -287,7 +287,8 @@ export class LifetimeRegions { * ProgramLifetime instance). */ private reportLines(struct: string, f: Field, i: number): string[] { if (!isRefCounted(f.type) || !this.reaches(f.type)) return []; - const arm = this.host.nullableUnions.of(f.type)?.arm ?? f.type; + const nullable = this.host.nullableUnions.of(f.type); + const arm = nullable?.repr === "single" ? nullable.arm : f.type; const load = [ ` %f${i} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${f.index}`, ` %v${i} = load ptr, ptr %f${i}`, @@ -411,7 +412,7 @@ export class LifetimeRegions { ...this.reportBody( `define internal void ${walk}(ptr %o, ptr %w) ${FN_ATTRS} { ; region walk record ${shape.id}`, mangleRecordStruct(shape.id), - rcMembers(shape, this.host.inlineUnions), + rcMembers(shape, this.host.inlineUnions, this.host.nullableUnions.fieldBase(shape.id)), ), ); rows.push(`{ ptr, ptr } { ptr @${mangleRecordRelease(shape.id)}, ptr ${walk} }`); diff --git a/packages/compiler/src/backend/llvm/shapes.ts b/packages/compiler/src/backend/llvm/shapes.ts index 23565eff4..162b36425 100644 --- a/packages/compiler/src/backend/llvm/shapes.ts +++ b/packages/compiler/src/backend/llvm/shapes.ts @@ -31,7 +31,16 @@ import { import { cycTypeDescriptor, emitObjectAlloc, emitObjectFree } from "./alloc.js"; import type { BlockBuilder } from "./blocks.js"; import { SCR_ARR_FIELD, llvmCommentText } from "./common.js"; -import type { NullableUnions } from "./nullable-unions.js"; +import { + NULLABLE_NULL, + SHAPE_WORD_INDEX, + shapeReleaseFn, + shapeTagFn, + shapeTraceFn, + shapeWordHeader, + type NullableUnions, + type ShapeUnion, +} from "./nullable-unions.js"; import { INLINE_UNION_FIELD, loadInlineUnion, @@ -129,6 +138,7 @@ export { computeTraced } from "../cycle-analysis.js"; * `_v`-shaped. */ export function vAdapters(host: ShapeHost, t: IrType): { retain: string; release: string } { const nullable = host.nullableUnions.of(t); + if (nullable?.repr === "shapes") return shapeUnionRc(nullable); if (nullable) { const arm = vAdapters(host, nullable.arm); return { retain: nullableRetainSym(host, nullable.arm), release: arm.release }; @@ -184,6 +194,7 @@ export function vAdapters(host: ShapeHost, t: IrType): { retain: string; release * Families with a mirrored fast path call the inline helper instead. */ export function retainSym(host: ShapeHost, t: IrType): string { const nullable = host.nullableUnions.of(t); + if (nullable?.repr === "shapes") return shapeUnionRc(nullable).retain; if (nullable) return nullableRetainSym(host, nullable.arm); const stem = runtimeRcStem(t); if (host.rcHelpers !== null && stem !== null && inlineRcFamily(host, stem, "retain") !== null) { @@ -195,6 +206,99 @@ export function retainSym(host: ShapeHost, t: IrType): string { return vAdapters(host, t).retain; } +/** A shape union's RC entry points: every record retain is the same + * NULL- and immortal-skipping increment (the null sentinel is immortal), + * so the first arm's serves; release dispatches on the arm. */ +function shapeUnionRc(union: ShapeUnion): { retain: string; release: string } { + return { + retain: `@${mangleRecordRetain(union.arms[0]!.shapeId)}`, + release: shapeReleaseFn(union.unionId), + }; +} + +/** The tag, release and trace helpers of every shape union + * (nullable-unions.ts). The tag helper maps a value to its arm: NULL and + * the null sentinel are the unit arms, anything else a tagged record whose + * shape word names its arm (the switch folds to a lookup table). */ +export function emitShapeUnionHelpers(host: ShapeHost): string[] { + const out: string[] = []; + const S = host.sizeType; + for (const union of host.nullableUnions.allShapeUnions()) { + const header = shapeWordHeader(S); + const tagFn = shapeTagFn(union.unionId); + const cases = union.arms + .map((arm) => `i32 ${host.nullableUnions.shapeNumber(arm.shapeId)}, label %a${arm.tag}`) + .join(" "); + out.push( + `define internal i32 ${tagFn}(ptr %u) alwaysinline ${FN_ATTRS} { ; shape union tag`, + `entry:`, + ` %isnull = icmp eq ptr %u, null`, + ` br i1 %isnull, label %unit, label %ref`, + `unit:`, + ` ret i32 ${union.unitTag}`, + `ref:`, + ); + if (union.nullTag >= 0) + out.push( + ` %isnullarm = icmp eq ptr %u, ${NULLABLE_NULL}`, + ` br i1 %isnullarm, label %nullarm, label %rec`, + `nullarm:`, + ` ret i32 ${union.nullTag}`, + `rec:`, + ); + out.push( + ` %wp = getelementptr inbounds ${header}, ptr %u, i32 0, i32 ${SHAPE_WORD_INDEX}`, + ` %w = load i32, ptr %wp`, + ` switch i32 %w, label %bad [ ${cases} ]`, + ...union.arms.flatMap((arm) => [`a${arm.tag}:`, ` ret i32 ${arm.tag}`]), + `bad:`, + ` ret i32 -1`, + `}`, + ``, + ); + // release: unit encodings are NULL or immortal; each arm releases + // through its own (NULL- and immortal-tolerant) helper. + out.push( + `define internal void ${shapeReleaseFn(union.unionId)}(ptr %u) ${FN_ATTRS} { ; shape union release`, + `entry:`, + ` %t = call i32 ${tagFn}(ptr %u)`, + ` switch i32 %t, label %done [ ${union.arms.map((arm) => `i32 ${arm.tag}, label %a${arm.tag}`).join(" ")} ]`, + ...union.arms.flatMap((arm) => [ + `a${arm.tag}:`, + ` call void @${mangleRecordRelease(arm.shapeId)}(ptr %u)`, + ` br label %done`, + ]), + `done:`, + ` ret void`, + `}`, + ``, + ); + if (shapeUnionTraced(host, union)) + out.push( + `define internal void ${shapeTraceFn(union.unionId)}(ptr %u, ptr %visit, ptr %ctx) ${FN_ATTRS} { ; shape union trace`, + `entry:`, + ` %t = call i32 ${tagFn}(ptr %u)`, + ` switch i32 %t, label %done [ ${union.arms.map((arm) => `i32 ${arm.tag}, label %a${arm.tag}`).join(" ")} ]`, + ...union.arms.flatMap((arm) => [ + `a${arm.tag}:`, + ` call void @${mangleRecordTrace(arm.shapeId)}(ptr %u, ptr %visit, ptr %ctx)`, + ` br label %done`, + ]), + `done:`, + ` ret void`, + `}`, + ``, + ); + } + return out; +} + +/** A shape union's arms carry collector headers (all or none: the emitter + * traces every arm of a union with one traced arm). */ +function shapeUnionTraced(host: ShapeHost, union: ShapeUnion): boolean { + return union.arms.some((arm) => host.tracedShapes.has(`record:${arm.shapeId}`)); +} + /** The retain entry point of a nullable union's value: the arm's own when * it tolerates NULL (emitted class and record helpers), otherwise a * NULL-skipping wrapper around it (the null sentinel and literals are @@ -233,6 +337,7 @@ export function emitNullableRetainWrappers(host: ShapeHost): string[] { * Families with a mirrored fast path call the inline helper instead. */ export function releaseSym(host: ShapeHost, t: IrType): string { const nullable = host.nullableUnions.of(t); + if (nullable?.repr === "shapes") return shapeUnionRc(nullable).release; if (nullable) return releaseSym(host, nullable.arm); const stem = runtimeRcStem(t); if (host.rcHelpers !== null && stem !== null && inlineRcFamily(host, stem, "release") !== null) { @@ -643,6 +748,8 @@ export function traceAdapter(host: ShapeHost, t: IrType): string | null { return "@scr_promise_trace_v"; case "union": { const nullable = host.nullableUnions.get(t.unionId); + if (nullable?.repr === "shapes") + return shapeUnionTraced(host, nullable) ? shapeTraceFn(nullable.unionId) : null; if (nullable) return traceAdapter(host, nullable.arm); if (!host.tracedUnions.has(t.unionId)) return null; host.declare(`declare void @scr_union_trace_v(ptr, ptr, ptr)`); @@ -945,15 +1052,17 @@ export function mapValKindNum(value: IrType): number { export function rcMembers( shape: IrRecordShape, inline: InlineUnionFields, + /** The struct index of the first field (2 on tagged shapes). */ + base = 1, ): { index: number; type: IrType; name: string }[] { return [ ...shape.fields.flatMap((f, i) => - inline.record(f.type) ? [] : [{ index: i + 1, type: f.type, name: f.name }], + inline.record(f.type) ? [] : [{ index: i + base, type: f.type, name: f.name }], ), ...(shape.indexValue ? [ { - index: shape.fields.length + 1, + index: shape.fields.length + base, type: mapOf(STRING, shape.indexValue), name: "[key: string] overflow", }, @@ -1107,7 +1216,12 @@ export function directDestroyTarget( } function directDestroyKey(host: ShapeHost, type: IrType): string | null { - const t = host.nullableUnions.of(type)?.arm ?? type; + const nullable = host.nullableUnions.of(type); + let t = type; + if (nullable !== null) { + if (nullable.repr === "shapes") return null; + t = nullable.arm; + } if (t.kind === "record") return `record:${t.shapeId}`; return t.kind === "object" ? `object:${t.className}` : null; } @@ -1338,8 +1452,12 @@ export function emitRecordShapes( for (const shape of records) { const struct = mangleRecordStruct(shape.id); - const fieldTys = shape.fields.map((f) => llFieldType(f.type, host.inlineUnions.record(f.type))); + const fieldTys: string[] = shape.fields.map((f) => + llFieldType(f.type, host.inlineUnions.record(f.type)), + ); if (shape.indexValue) fieldTys.push("ptr"); // the overflow ScrMap * + // A tagged shape (an arm of a shape union) carries its shape number. + if (host.nullableUnions.shapeNumber(shape.id) !== 0) fieldTys.unshift("i32"); typeDefs.push( `%${struct} = type { ${host.sizeType}${fieldTys.length ? ", " + fieldTys.join(", ") : ""} } ` + `; record ${shape.id} { ${shape.fields.map((f) => llvmCommentText(f.name)).join("; ")}${shape.indexValue ? "; [key: string]" : ""} }`, @@ -1352,7 +1470,7 @@ export function emitRecordShapes( const candidates: [string, DirectDestroy][] = []; const memberTypes: IrType[] = []; for (const shape of records) { - const members = rcMembers(shape, host.inlineUnions); + const members = rcMembers(shape, host.inlineUnions, host.nullableUnions.fieldBase(shape.id)); for (const m of members) memberTypes.push(m.type); if (!members.some((m) => isRefCounted(m.type) && needsBoundedRelease(m.type))) continue; const release = mangleRecordRelease(shape.id); @@ -1370,7 +1488,9 @@ export function emitRecordShapes( for (const shape of records) { const struct = mangleRecordStruct(shape.id); const traced = host.tracedShapes.has(`record:${shape.id}`); - const members = rcMembers(shape, host.inlineUnions); + const base = host.nullableUnions.fieldBase(shape.id); + const shapeNumber = host.nullableUnions.shapeNumber(shape.id); + const members = rcMembers(shape, host.inlineUnions, base); const refMembers = members.filter((m) => isRefCounted(m.type)); const sizeOf = `ptrtoint (ptr getelementptr (%${struct}, ptr null, i32 1) to ${host.sizeType})`; @@ -1439,6 +1559,11 @@ export function emitRecordShapes( ); nw.push(...emitObjectAlloc(host, sizeOf, traced ? { type: cycType } : null)); nw.push(` store ${host.sizeType} 1, ptr %o`); + if (shapeNumber !== 0) + nw.push( + ` %shp = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${SHAPE_WORD_INDEX}`, + ` store i32 ${shapeNumber}, ptr %shp ; shape number`, + ); if (shape.indexValue) { // The overflow map (string-keyed): value handling is type-directed // exactly like shapes.ts's overflowNewC. @@ -1449,7 +1574,7 @@ export function emitRecordShapes( const trace = valKind === 2 ? traceArg(host, v) : "null"; nw.push( ` %ovf = call ptr @scr_map_new(i32 1, i32 ${valKind}, ptr ${host.mapOps(["null", "null", "null", rc.retain, rc.release, trace, "null"])})`, - ` %ovfp = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${shape.fields.length + 1}`, + ` %ovfp = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${shape.fields.length + base}`, ` store ptr %ovf, ptr %ovfp`, ); } @@ -1465,7 +1590,7 @@ export function emitRecordShapes( ]; let i = 0; for (const field of shape.fields) { - const index = i + 1; + const index = i + base; const inline = host.inlineUnions.record(field.type); const fieldTy = llFieldType(field.type, inline); clone.push( diff --git a/packages/compiler/src/backend/llvm/stack-objects.ts b/packages/compiler/src/backend/llvm/stack-objects.ts index e9f437f14..0d65d9456 100644 --- a/packages/compiler/src/backend/llvm/stack-objects.ts +++ b/packages/compiler/src/backend/llvm/stack-objects.ts @@ -21,6 +21,7 @@ import type { CallLifetimes } from "./call-lifetimes.js"; import { emitCallExpr, emitConstruction } from "./expr-calls.js"; import type { LlValue, LlvmEmitterContext } from "./expr-context.js"; import { emitRecordLitFields } from "./expr-primitives.js"; +import { SHAPE_WORD_INDEX } from "./nullable-unions.js"; import { llFieldType } from "./shapes.js"; export type StackObjectSource = IrExpr & { kind: "recordLit" | "recordClone" | "new" | "call" }; @@ -196,12 +197,13 @@ export function stackObjectFields( const cached = host.stackObjectFieldPtrs.get(slot); if (cached) return cached; const struct = stackObjectStruct(type); + const base = host.nullableUnions.fieldBase(type.shapeId); const fields: StackObjectField[] = []; host.recordShape(type.shapeId).fields.forEach((field, index) => { if (!isRefCounted(field.type) || host.inlineUnions.record(field.type)) return; const ptr = host.B.slot(); host.B.entryAllocas.push( - `${ptr} = getelementptr inbounds ${struct}, ptr ${slot}, i64 0, i32 ${index + 1}`, + `${ptr} = getelementptr inbounds ${struct}, ptr ${slot}, i64 0, i32 ${index + base}`, ); fields.push({ name: field.name, ptr, type: field.type }); }); @@ -296,6 +298,17 @@ export function emitStackObject( } } B.line(`store ${host.sizeType} -1, ptr ${slot}`); + if (type.kind === "record") { + // A tagged shape's word names it to every shape union's tag read. + const shapeNumber = host.nullableUnions.shapeNumber(type.shapeId); + if (shapeNumber !== 0) { + const p = B.tmp(); + B.line( + `${p} = getelementptr inbounds ${struct}, ptr ${slot}, i64 0, i32 ${SHAPE_WORD_INDEX}`, + ); + B.line(`store i32 ${shapeNumber}, ptr ${p}`); + } + } if (e.kind === "call") { // The entry writes the result into the slot it receives last. emitCallExpr( diff --git a/packages/compiler/src/backend/llvm/string-slices.test.ts b/packages/compiler/src/backend/llvm/string-slices.test.ts index 53cca6417..b2656e560 100644 --- a/packages/compiler/src/backend/llvm/string-slices.test.ts +++ b/packages/compiler/src/backend/llvm/string-slices.test.ts @@ -1,7 +1,11 @@ import { expect, test } from "vitest"; -import { F64, STRING, VOID, type IrExpr, type IrFunction, type IrStmt } from "../../ir/ir.js"; +import { BOOL, F64, STRING, VOID, type IrExpr, type IrFunction, type IrStmt } from "../../ir/ir.js"; import { analyzeCallLifetimes } from "./call-lifetimes.js"; -import { findScalarStringSlices } from "./string-slices.js"; +import { + findLentStringParameters, + findScalarStringSlices, + findScratchStringSlices, +} from "./string-slices.js"; const loc = { file: "slices.ts", start: 0, end: 0 }; const ref = (id: string): IrExpr => ({ kind: "varRef", localId: id, type: STRING, loc }); function fixture(): IrFunction { @@ -95,3 +99,146 @@ test.each(["escape", "capture", "write", "other-consumer", "suspend"])( expect(facts(fn).size).toBe(0); }, ); + +const slice = (receiver: string, start: number, end: number): IrExpr => ({ + kind: "strIntrinsic", + method: "slice", + receiver: ref(receiver), + args: [ + { kind: "numLit", value: start, type: F64, loc }, + { kind: "numLit", value: end, type: F64, loc }, + ], + type: STRING, + loc, +}); +const equals = (left: IrExpr): IrExpr => ({ + kind: "strEq", + negated: false, + left, + right: { kind: "strLit", value: "GET", type: STRING, loc }, + type: BOOL, + loc, +}); +const parse = (value: IrExpr): IrExpr => ({ + kind: "libCall", + fn: "num.fromString", + args: [value], + type: F64, + loc, +}); +function lending(body: IrStmt[]): IrFunction { + return { + name: "scan", + params: [{ localId: "source", name: "source", type: STRING }], + locals: [ + { id: "source", name: "source", type: STRING, mutable: false }, + { id: "word", name: "word", type: STRING, mutable: false }, + ], + returnType: VOID, + loc, + body, + }; +} +function scratch(fn: IrFunction, extra: IrFunction[] = []): Set { + const functions = new Map([fn, ...extra].map((f) => [f.name, f])); + const lifetimes = analyzeCallLifetimes(functions); + return findScratchStringSlices( + fn, + lifetimes, + findScalarStringSlices(fn, lifetimes), + findLentStringParameters(functions, lifetimes), + ); +} +test("slices read only by lending consumers use a stack buffer", () => { + const inline = slice("source", 0, 3); + const parsed = slice("source", 4, 6); + const init = slice("source", 1, 5); + const fn = lending([ + { kind: "exprStmt", expr: equals(inline), loc }, + { kind: "exprStmt", expr: parse(parsed), loc }, + { kind: "varDecl", localId: "word", init, loc }, + { kind: "exprStmt", expr: equals(ref("word")), loc }, + { kind: "exprStmt", expr: parse(ref("word")), loc }, + ]); + expect(scratch(fn)).toEqual(new Set([inline, parsed, init])); +}); +test.each(["return", "concat", "slice-of-slice", "capture", "async"])( + "a slice local that may be kept (%s) stays owned", + (reason) => { + const init = slice("source", 1, 5); + const fn = lending([ + { kind: "varDecl", localId: "word", init, loc }, + { kind: "exprStmt", expr: equals(ref("word")), loc }, + ]); + if (reason === "return") fn.body.push({ kind: "return", value: ref("word"), loc }); + if (reason === "concat") + fn.body.push({ + kind: "exprStmt", + expr: { + kind: "strConcat", + left: { kind: "strLit", value: "", type: STRING, loc }, + right: ref("word"), + type: STRING, + loc, + }, + loc, + }); + if (reason === "slice-of-slice") + fn.body.push({ kind: "exprStmt", expr: equals(slice("word", 0, 2)), loc }); + if (reason === "capture") fn.locals[1]!.boxed = true; + if (reason === "async") fn.async = true; + expect(scratch(fn).has(init)).toBe(false); + }, +); +function helper(name: string, body: IrStmt[]): IrFunction { + return { + name, + params: [{ localId: "text", name: "text", type: STRING }], + locals: [{ id: "text", name: "text", type: STRING, mutable: false }], + returnType: BOOL, + loc, + body, + }; +} +const callOf = (callee: string, arg: IrExpr): IrExpr => ({ + kind: "call", + callee, + args: [arg], + type: BOOL, + loc, +}); +test("helper parameters read only by lending consumers accept stack slices", () => { + const recursive = helper("isGet", [ + { + kind: "return", + value: { + kind: "logical", + op: "||", + left: equals(ref("text")), + right: callOf("isGet", ref("text")), + type: BOOL, + loc, + }, + loc, + }, + ]); + const keeps = helper("keeps", [{ kind: "return", value: callOf("stores", ref("text")), loc }]); + const stores = helper("stores", [ + { + kind: "exprStmt", + expr: { kind: "arrayLit", elems: [ref("text")], type: { kind: "array", elem: STRING }, loc }, + loc, + }, + { kind: "return", value: { kind: "boolLit", value: true, type: BOOL, loc }, loc }, + ]); + const lent = slice("source", 0, 3); + const kept = slice("source", 0, 4); + const fn = lending([ + { kind: "exprStmt", expr: callOf("isGet", lent), loc }, + { kind: "exprStmt", expr: callOf("keeps", kept), loc }, + ]); + const functions = [recursive, keeps, stores]; + expect(scratch(fn, functions)).toEqual(new Set([lent])); + const all = new Map([fn, ...functions].map((f) => [f.name, f])); + expect([...findLentStringParameters(all, analyzeCallLifetimes(all)).keys()]).toEqual(["isGet"]); +}); diff --git a/packages/compiler/src/backend/llvm/string-slices.ts b/packages/compiler/src/backend/llvm/string-slices.ts index 71a26e2f3..ae73f1ea1 100644 --- a/packages/compiler/src/backend/llvm/string-slices.ts +++ b/packages/compiler/src/backend/llvm/string-slices.ts @@ -1,4 +1,9 @@ -import { type IrExpr, type IrFunction, type IrStmt } from "../../ir/ir.js"; +import { + type IrExpr, + type IrFunction, + type IrStmt, + type IrStrIntrinsicMethod, +} from "../../ir/ir.js"; import { everyExprChild, everyStmtChild, everyStmtList } from "../../ir/traverse.js"; import { F64_INF, f64Lit, sizeToDouble } from "./common.js"; import type { CallLifetimes } from "./call-lifetimes.js"; @@ -6,6 +11,12 @@ import type { LlValue, LlvmEmitterContext } from "./expr-context.js"; type Slice = IrExpr & { kind: "strIntrinsic" }; +function isSlice(value: IrExpr): value is Slice { + return ( + value.kind === "strIntrinsic" && (value.method === "slice" || value.method === "substring") + ); +} + /** A local used exclusively for scalar indexing needs no copied bytes. * Snapshot the receiver and normalized bounds at its initializer, including * side effects in the bounds. Escapes, captures, writes, suspension, and all @@ -111,3 +122,234 @@ export function emitStringSliceRead(host: LlvmEmitterContext, value: Slice): LlV ); return { name: result, type: value.type }; } + +/** Bytes of the stack buffer behind one scratch slice, header included. */ +export const SCRATCH_SLICE_BYTES = 96; + +/** String methods that read a lent receiver or argument only during their + * runtime call and never return it. Methods returning a string can return + * their unchanged receiver, so they stay on the owned path. */ +const LENDING_METHODS: ReadonlySet = new Set([ + "length", + "charCodeAt", + "indexOf", + "includes", + "startsWith", + "endsWith", +]); + +/** Numeric parsers read their text during the call. */ +const LENDING_LIB_FNS: ReadonlySet = new Set([ + "num.fromString", + "num.parseInt", + "num.parseFloat", +]); + +/** Typed string-keyed collection operations: reads probe with the key, and + * an insertion stores scr_str_scratch_keep's copy (scr_map_append). */ +const LENDING_KEY_METHODS: ReadonlySet = new Set(["get", "has", "set", "add", "delete"]); + +function stringKeyed(value: IrExpr & { kind: "mapIntrinsic" | "setIntrinsic" }): boolean { + const receiver = value.receiver.type; + if (receiver.kind === "map") + return receiver.key.kind === "string" && receiver.value.kind !== "dyn"; + if (receiver.kind === "set") return receiver.elem.kind === "string"; + return false; +} + +/** String parameters, by function and index, that the body reads only in + * lending positions (findLentStringParameters). */ +export type LentStringParameters = ReadonlyMap>; + +/** The operands `value` reads only while it runs, keeping no reference. */ +function lentOperands(value: IrExpr, parameters: LentStringParameters): readonly IrExpr[] { + switch (value.kind) { + case "strEq": + case "strCmp": + return [value.left, value.right]; + case "strIntrinsic": + return LENDING_METHODS.has(value.method) ? [value.receiver, ...value.args] : []; + case "libCall": + return LENDING_LIB_FNS.has(value.fn) ? value.args.slice(0, 1) : []; + case "mapIntrinsic": + case "setIntrinsic": + return LENDING_KEY_METHODS.has(value.method) && stringKeyed(value) + ? value.args.slice(0, 1) + : []; + case "call": { + const lent = parameters.get(value.callee); + return lent ? value.args.filter((_, index) => lent.has(index)) : []; + } + default: + return []; + } +} + +interface LendingScan { + /** Slice operands of lending consumers, excluding any node also reached + * in an owning position. */ + scratch: Set; + /** Candidates read anywhere other than a lending position. */ + invalid: Set; + /** Direct callees, for the parameter fixpoint. */ + callees: Set; +} + +/** One pass over `fn`: which candidate bindings only lending consumers read + * (a slice initializer of a candidate local is not itself a read), and + * which slice operands those consumers lend. A switch discriminant is only + * compared with the case tests. */ +function scanLending( + fn: IrFunction, + candidates: ReadonlySet, + initializers: ReadonlyMap, + parameters: LentStringParameters, +): LendingScan { + const scan: LendingScan = { scratch: new Set(), invalid: new Set(), callees: new Set() }; + const owned = new Set(); + function lent(value: IrExpr): boolean { + if (value.kind === "varRef" && candidates.has(value.localId)) return true; + if (!isSlice(value)) return expr(value); + scan.scratch.add(value); + return everyExprChild(value, expr, stmt); + } + function expr(value: IrExpr): boolean { + if (isSlice(value)) owned.add(value); + if (value.kind === "varRef" && candidates.has(value.localId)) scan.invalid.add(value.localId); + if (value.kind === "closure" || value.kind === "classRef") + for (const id of value.captures ?? []) scan.invalid.add(id); + if (value.kind === "call") scan.callees.add(value.callee); + const operands = lentOperands(value, parameters); + if (operands.length === 0) return everyExprChild(value, expr, stmt); + return everyExprChild( + value, + (child) => (operands.includes(child) ? lent(child) : expr(child)), + stmt, + ); + } + function stmt(value: IrStmt): boolean { + // A candidate's single declaration initializes it with its slice. + if (value.kind === "varDecl" && value.init !== null && initializers.has(value.localId)) + return everyExprChild(value.init, expr, stmt); + if (value.kind === "switch" && value.disc.type.kind === "string") { + lent(value.disc); + for (const entry of value.cases) { + if (entry.test) expr(entry.test); + entry.body.forEach(stmt); + } + return true; + } + return everyStmtChild(value, expr, stmt); + } + fn.body.forEach(stmt); + for (const node of owned) scan.scratch.delete(node); + return scan; +} + +/** String parameters that every body reads only in lending positions, + * including lent arguments of other such parameters: a caller may pass a + * scratch slice to them. Start from every borrowed (unchanged) string + * parameter of a synchronous function and drop violations until nothing changes, so + * recursion through a lent parameter stays lent. A coroutine can keep its + * parameters past the caller's frame, so it never lends. */ +export function findLentStringParameters( + functions: ReadonlyMap, + lifetimes: CallLifetimes, +): Map> { + const lent = new Map>(); + for (const fn of functions.values()) { + if (fn.async || fn.generator) continue; + // Borrowed parameters are the unchanged ones nothing redeclares. + const stable = lifetimes.borrowed.get(fn.name); + const indexes = new Set(); + fn.params.forEach((param, index) => { + if (param.type.kind === "string" && stable?.has(index)) indexes.add(index); + }); + if (indexes.size > 0) lent.set(fn.name, indexes); + } + const callers = new Map>(); + const pending = new Set(lent.keys()); + while (pending.size > 0) { + const name = pending.values().next().value as string; + pending.delete(name); + const indexes = lent.get(name); + const fn = functions.get(name); + if (!indexes || !fn) continue; + const ids = new Set([...indexes].map((index) => fn.params[index]!.localId)); + const scan = scanLending(fn, ids, new Map(), lent); + for (const callee of scan.callees) { + let set = callers.get(callee); + if (!set) callers.set(callee, (set = new Set())); + set.add(name); + } + let changed = false; + for (const index of [...indexes]) { + if (!scan.invalid.has(fn.params[index]!.localId)) continue; + indexes.delete(index); + changed = true; + } + if (!changed) continue; + if (indexes.size === 0) lent.delete(name); + const affected = callers.get(name); + if (affected) for (const caller of affected) if (lent.has(caller)) pending.add(caller); + } + return lent; +} + +/** Slices whose every reader borrows them are written into a stack buffer + * instead of a heap allocation (scr_str_slice_scratch): a slice operand of + * a lending consumer, and the initializer of a stable local read only by + * lending consumers. The copy is ASCII, bounded, and immortal for retain and + * release, so it neither pins its source nor needs to be freed; longer or + * non-ASCII results keep the ordinary owned string. Each site owns its + * buffer: a site runs again only after its previous value is dead, since + * nothing can keep it. Coroutine frames do not keep stack buffers across a + * suspension, so they are left alone. */ +export function findScratchStringSlices( + fn: IrFunction, + lifetimes: CallLifetimes, + scalar: ReadonlyMap, + parameters: LentStringParameters = new Map(), +): Set { + if (fn.async || fn.generator) return new Set(); + const bindings = lifetimes.bindings.get(fn.name); + const locals = new Map(); + everyStmtList(fn.body, { + expr: () => true, + stmt: (stmt) => { + if ( + stmt.kind === "varDecl" && + bindings?.has(stmt.localId) && + !scalar.has(stmt.localId) && + stmt.init !== null && + isSlice(stmt.init) + ) + locals.set(stmt.localId, stmt.init); + return true; + }, + }); + const scan = scanLending(fn, new Set(locals.keys()), locals, parameters); + for (const [id, init] of locals) if (!scan.invalid.has(id)) scan.scratch.add(init); + return scan.scratch; +} + +/** Write a scratch slice into its own stack buffer. The result is owned: + * releasing the scratch copy does nothing, and a heap fallback is freed. */ +export function emitScratchStringSlice( + host: LlvmEmitterContext, + slice: Slice, + receiver: LlValue, + args: readonly LlValue[], +): LlValue { + const B = host.B; + const buffer = B.slot(); + B.entryAllocas.push(`${buffer} = alloca [${SCRATCH_SLICE_BYTES} x i8], align 8`); + host.declare( + `declare ptr @scr_str_slice_scratch(ptr, double, double, i1 zeroext, ptr, ${host.sizeType})`, + ); + const result = B.tmp(); + B.line( + `${result} = call ptr @scr_str_slice_scratch(ptr ${receiver.name}, double ${args[0]?.name ?? f64Lit(0)}, double ${args[1]?.name ?? F64_INF}, i1 ${slice.method === "substring" ? "true" : "false"}, ptr ${buffer}, ${host.sizeType} ${SCRATCH_SLICE_BYTES})`, + ); + return host.own({ name: result, type: slice.type }); +} diff --git a/packages/compiler/src/backend/llvm/union-repr.ts b/packages/compiler/src/backend/llvm/union-repr.ts index 415aeb851..959f9e51e 100644 --- a/packages/compiler/src/backend/llvm/union-repr.ts +++ b/packages/compiler/src/backend/llvm/union-repr.ts @@ -2,14 +2,19 @@ * helper generators (walkers, dyn). A tagged union is a `%ScrUnion` box; a * nullable union (nullable-unions.ts) is the arm pointer itself: NULL for * its NULL-encoded unit arm and, in `C | null | undefined`, the immortal - * NULLABLE_NULL sentinel for `null`. Every structural read of a union value + * NULLABLE_NULL sentinel for `null`. A shape union's value is likewise the + * record pointer; its tag comes from the record's shape word. Every structural read of a union value * goes through here. */ import type { BlockBuilder } from "./blocks.js"; -import { NULLABLE_NULL, type NullableUnion } from "./nullable-unions.js"; +import { NULLABLE_NULL, shapeTagFn, type PointerUnion } from "./nullable-unions.js"; /** The i32 tag of a union value. */ -export function emitUnionTag(B: BlockBuilder, nullable: NullableUnion | null, u: string): string { +export function emitUnionTag(B: BlockBuilder, nullable: PointerUnion | null, u: string): string { const t = B.tmp(); + if (nullable?.repr === "shapes") { + B.line(`${t} = call i32 ${shapeTagFn(nullable.unionId)}(ptr ${u})`); + return t; + } if (nullable) { const isNull = B.tmp(); B.line(`${isNull} = icmp eq ptr ${u}, null`); @@ -32,8 +37,13 @@ export function emitUnionTag(B: BlockBuilder, nullable: NullableUnion | null, u: /** i1: a nullable union value holds its reference arm (neither NULL nor * the `null` sentinel). */ -export function emitNullablePresent(B: BlockBuilder, nullable: NullableUnion, u: string): string { +export function emitNullablePresent(B: BlockBuilder, nullable: PointerUnion, u: string): string { const present = B.tmp(); + if (nullable.repr === "shapes" && nullable.unitTag < 0) { + // No unit arm: every value is a record (NULL only as a pending dummy). + B.line(`${present} = icmp ne ptr ${u}, null`); + return present; + } B.line(`${present} = icmp ne ptr ${u}, null`); if (nullable.nullTag < 0) return present; const notNullArm = B.tmp(), @@ -46,12 +56,18 @@ export function emitNullablePresent(B: BlockBuilder, nullable: NullableUnion, u: /** i1: a nullable union value holds the arm `tag` (`negated`: does not). */ export function emitNullableIsTag( B: BlockBuilder, - nullable: NullableUnion, + nullable: PointerUnion, u: string, tag: number, negated: boolean, ): string { - if (tag === nullable.refTag) { + if (nullable.repr === "shapes" && tag !== nullable.unitTag && tag !== nullable.nullTag) { + const actual = emitUnionTag(B, nullable, u), + t = B.tmp(); + B.line(`${t} = icmp ${negated ? "ne" : "eq"} i32 ${actual}, ${tag}`); + return t; + } + if (nullable.repr === "single" && tag === nullable.refTag) { const present = emitNullablePresent(B, nullable, u); if (!negated) return present; const t = B.tmp(); @@ -65,7 +81,7 @@ export function emitNullableIsTag( } /** The BORROWED payload pointer of a reference arm. */ -export function emitUnionPeek(B: BlockBuilder, nullable: NullableUnion | null, u: string): string { +export function emitUnionPeek(B: BlockBuilder, nullable: PointerUnion | null, u: string): string { if (nullable) return u; const p = B.tmp(); const t = B.tmp(); @@ -76,7 +92,9 @@ export function emitUnionPeek(B: BlockBuilder, nullable: NullableUnion | null, u /** Text-template forms for helpers written as raw LLVM lines: `%out` * receives the tag / payload of `u`. */ -export function unionTagLines(nullable: NullableUnion | null, u: string, out: string): string[] { +export function unionTagLines(nullable: PointerUnion | null, u: string, out: string): string[] { + if (nullable?.repr === "shapes") + return [` ${out} = call i32 ${shapeTagFn(nullable.unionId)}(ptr ${u})`]; if (nullable && nullable.nullTag < 0) return [ ` ${out}.isnull = icmp eq ptr ${u}, null`, @@ -95,7 +113,7 @@ export function unionTagLines(nullable: NullableUnion | null, u: string, out: st ]; } -export function unionPeekLines(nullable: NullableUnion | null, u: string, out: string): string[] { +export function unionPeekLines(nullable: PointerUnion | null, u: string, out: string): string[] { // A plain copy of the pointer (NULL included) under the template's name. if (nullable) return [` ${out} = getelementptr i8, ptr ${u}, i64 0`]; return [ diff --git a/packages/compiler/src/backend/llvm/union-retype.test.ts b/packages/compiler/src/backend/llvm/union-retype.test.ts index eca6adcd6..0f0abdb68 100644 --- a/packages/compiler/src/backend/llvm/union-retype.test.ts +++ b/packages/compiler/src/backend/llvm/union-retype.test.ts @@ -183,7 +183,9 @@ test("a nullable-pointer source rebuilds the target box", () => { expect(retag).not.toContain("retype.share"); }); -async function llvmFor(source: string): Promise<{ module: IrModule; ir: string }> { +/** `boxed`: keep every union a tagged box (record unions otherwise become + * shape unions, which retype without any box). */ +async function llvmFor(source: string, boxed = true): Promise<{ module: IrModule; ir: string }> { const dir = await mkdtemp(join(tmpdir(), "scriptc-union-retype-")); try { const entry = join(dir, "main.ts"), @@ -194,7 +196,7 @@ async function llvmFor(source: string): Promise<{ module: IrModule; ir: string } throw new Error(result.diagnostics.map((item) => `${item.code}: ${item.message}`).join("\n")); const module = deserializeModule(await readFile(output, "utf8")); expect(validateModule(module)).toEqual([]); - return { module, ir: emitLlvmModule(module) }; + return { module, ir: emitLlvmModule(boxed ? keepUnionsBoxed(module) : module) }; } finally { await rm(dir, { recursive: true, force: true }); } @@ -253,3 +255,16 @@ console.log(leafValue(parsed)); expect(stored).not.toBeNull(); expect(ir).toContain(`load i8, ptr ${stored![1]}`); }); + +test("record unions retype as the record pointer itself", async () => { + const { module, ir } = await llvmFor(VISITOR, false); + const helpers = module.functions.filter((fn) => fn.name.startsWith("%union.retag.")); + expect(helpers.length).toBeGreaterThan(0); + for (const helper of helpers) { + const symbol = `sc_bf__x25_union_retag_${helper.name.slice("%union.retag.".length)}`; + const retag = body(ir, symbol); + expect(retag).not.toMatch(/@scr_union_new|retype\.share|sc_retype_guard/); + } + // No record of the visitor is ever boxed. + expect(ir).not.toMatch(/call ptr @scr_union_new_ref/); +}); diff --git a/packages/compiler/src/backend/llvm/walkers.ts b/packages/compiler/src/backend/llvm/walkers.ts index 9b8d80b87..2f69dd5ec 100644 --- a/packages/compiler/src/backend/llvm/walkers.ts +++ b/packages/compiler/src/backend/llvm/walkers.ts @@ -284,7 +284,8 @@ export class LlWalkers { throw new InternalCompilerError( `llvm emitter bug: jsonStringify of unknown shape ${shapeId}`, ); - const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + 1])); + const fieldBase = this.host.nullableUnions.fieldBase(shapeId); + const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + fieldBase])); if (recordMayKeepSource(shape)) this.emitRecordSourceWriter(B, shapeId); // CYCLE-CAPABLE shapes bracket the walk with the circular-detection // stack; edge labels stamp before members whose walk can re-enter — @@ -437,7 +438,8 @@ export class LlWalkers { throw new InternalCompilerError(`llvm emitter bug: field writer of unknown shape ${shapeId}`); const name = `sc_jwf_${this.recordFieldWriters.size}`; this.recordFieldWriters.set(shapeId, name); - const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + 1])); + const fieldBase = this.host.nullableUnions.fieldBase(shapeId); + const fieldIndex = new Map(shape.fields.map((f, i) => [f.name, i + fieldBase])); const byName = new Map(shape.fields.map((f) => [f.name, f])); const cyclic = traceAdapter(this.host, { kind: "record", shapeId }) !== null; const order = recordSourceFieldOrder(shape); @@ -534,7 +536,7 @@ export class LlWalkers { const ovfp = B.tmp(); const ovf = B.tmp(); B.line( - `${ovfp} = getelementptr inbounds %${mangleRecordStruct(shape.id)}, ptr %v, i64 0, i32 ${shape.fields.length + 1}`, + `${ovfp} = getelementptr inbounds %${mangleRecordStruct(shape.id)}, ptr %v, i64 0, i32 ${shape.fields.length + this.host.nullableUnions.fieldBase(shape.id)}`, ); B.line(`${ovf} = load ptr, ptr ${ovfp} ; overflow map`); if (iv.kind === "dyn") { diff --git a/packages/compiler/src/cli/command.ts b/packages/compiler/src/cli/command.ts index d729de866..6810cc8fb 100644 --- a/packages/compiler/src/cli/command.ts +++ b/packages/compiler/src/cli/command.ts @@ -7,8 +7,10 @@ import { coverageEnvelope, coveragePasses, renderCoverage, + renderExplainDynamic, type CoverageFailOn, } from "../coverage/report.js"; +import { TIER_MODES, type TierMode } from "../frontend/tier-analysis.js"; import { buildEnvelope } from "../diagnostics/envelope.js"; import { renderWarnings } from "../diagnostics/render.js"; import { isNativeOptimization } from "../backend/optimization.js"; @@ -45,8 +47,15 @@ function parseCli( ): ReturnType< typeof parseArgs<{ options: typeof CLI_OPTIONS; allowPositionals: true; allowNegative: true }> > { + // `--explain-dynamic` takes an optional value: alone it lists every unit. + const normalized = args.map((arg) => (arg === "--explain-dynamic" ? "--explain-dynamic=" : arg)); try { - return parseArgs({ args, options: CLI_OPTIONS, allowPositionals: true, allowNegative: true }); + return parseArgs({ + args: normalized, + options: CLI_OPTIONS, + allowPositionals: true, + allowNegative: true, + }); } catch (err) { // parseArgs appends a paragraph about `--` and positionals to the // unknown-option message; the first sentence is the part that names @@ -91,6 +100,8 @@ async function main(args: string[], host: CliHost): Promise { externalTypeArgs.length > 0 || values.out !== undefined || values["emit-ir"] || + values.tier !== undefined || + values["explain-dynamic"] !== undefined || !values["keep-llvm"] ) { fail( @@ -152,10 +163,12 @@ async function main(args: string[], host: CliHost): Promise { values["windows-subsystem"] !== undefined || values["detached-exit"] || (values["npm-static"] ?? []).length > 0 || - externalTypeArgs.length > 0 + externalTypeArgs.length > 0 || + values.tier !== undefined || + values["explain-dynamic"] !== undefined ) { fail( - "scriptc build --lib takes no --backend/--emit/--print/--optimization/--strip/--windows-subsystem/--detached-exit/--npm-static/--ffi/--external-types: the profile pins the emission and optimization, npm imports are judged automatically, outbound FFI belongs to executable builds, and external type mappings belong to coverage", + "scriptc build --lib takes no --backend/--emit/--print/--optimization/--strip/--windows-subsystem/--detached-exit/--npm-static/--ffi/--external-types/--tier/--explain-dynamic: the profile pins the emission and optimization, npm imports are judged automatically, outbound FFI belongs to executable builds, external type mappings belong to coverage, and library artifacts are static-tier only", ); } const profilePath = resolve(profileArg); @@ -223,6 +236,15 @@ async function main(args: string[], host: CliHost): Promise { fail(`--fail-on is a coverage option (a build already fails on blockers)\n\n${USAGE}`); } const failOn = failOnRaw as CoverageFailOn | undefined; + const tierRaw = values.tier; + if (tierRaw !== undefined && !(TIER_MODES as readonly string[]).includes(tierRaw)) { + fail(`unknown --tier value "${tierRaw}" (supported: ${TIER_MODES.join(", ")})\n\n${USAGE}`); + } + const tier = tierRaw as TierMode | undefined; + const explainDynamic = values["explain-dynamic"]; + if (explainDynamic !== undefined && command !== "coverage") { + fail(`--explain-dynamic is a coverage option\n\n${USAGE}`); + } if (printNativeLinkInfo && command !== "build") { fail(`--print=native-link-info is a build option\n\n${USAGE}`); } @@ -340,6 +362,8 @@ async function main(args: string[], host: CliHost): Promise { ...(npmStatic !== undefined ? { npmStatic } : {}), ...(ffiProfilePath !== undefined ? { ffiProfilePath } : {}), ...(Object.keys(externalTypes).length > 0 ? { externalTypes } : {}), + ...(tier !== undefined ? { tier } : {}), + ...(explainDynamic !== undefined ? { explainDynamic: true } : {}), }); if (printDiagnostics) { // Keep stdout pure JSON for tooling. @@ -354,6 +378,18 @@ async function main(args: string[], host: CliHost): Promise { process.stdout.write( renderCoverage(coverage, { color, sourceTexts, root: process.cwd() }) + "\n", ); + if (explainDynamic !== undefined && coverage.tiers !== undefined) { + process.stdout.write( + "\n" + + renderExplainDynamic(coverage.tiers, { + color, + sourceTexts, + root: process.cwd(), + query: explainDynamic, + }) + + "\n", + ); + } } return coveragePasses(coverage, failOn) ? 0 : 1; } @@ -386,6 +422,7 @@ async function main(args: string[], host: CliHost): Promise { ...(npmStatic !== undefined ? { npmStatic } : {}), ...(ffiProfilePath !== undefined ? { ffiProfilePath } : {}), ...(printNativeLinkInfo ? { nativeLinkInfo: true } : {}), + ...(tier !== undefined ? { tier } : {}), }); if (!result.ok) { if (printDiagnostics) { diff --git a/packages/compiler/src/cli/usage.ts b/packages/compiler/src/cli/usage.ts index a0388374b..b5dcec332 100644 --- a/packages/compiler/src/cli/usage.ts +++ b/packages/compiler/src/cli/usage.ts @@ -57,7 +57,8 @@ Options: compile the named npm packages' shipped JS statically as program modules (repeatable; "auto" opts in every eligible direct import: own .d.ts, unminified JS, no build-transform - markers). Packages that fail preflight are refused with a + markers; "pkg..." also names every package pkg depends + on). Packages that fail preflight are refused with a diagnostic. Eligible dependencies compile automatically. --provenance-sources EXPERIMENTAL: compile npm dependencies from their @@ -69,6 +70,16 @@ Options: local declaration file. The declaration supplies types for analysis; the host module remains an explicit external-boundary blocker (repeatable) + --tier + tier policy (default: auto). static-only keeps every + construct without a static lowering a compile error, so + the program has no dynamic-tier demotions. Both modes + currently compile identically + --explain-dynamic[=] + coverage only: list the functions that operate on dynamic + values and the bindings, fields, functions, classes and + modules the dynamic tier would take over, with the + refusal behind each; a name shows only that unit's chain -h, --help show this help -v, --version print the version `; @@ -90,6 +101,8 @@ export const CLI_OPTIONS = { "npm-static": { type: "string", multiple: true }, "provenance-sources": { type: "boolean", default: false }, "external-types": { type: "string", multiple: true }, + tier: { type: "string" }, + "explain-dynamic": { type: "string" }, lib: { type: "boolean", default: false }, profile: { type: "string" }, help: { type: "boolean", short: "h", default: false }, diff --git a/packages/compiler/src/compile-types.ts b/packages/compiler/src/compile-types.ts index de2c1a335..f069df247 100644 --- a/packages/compiler/src/compile-types.ts +++ b/packages/compiler/src/compile-types.ts @@ -3,6 +3,7 @@ import type { NativeLinkInfo } from "./backend/native-link-info.js"; import type { NativeOptimization } from "./backend/optimization.js"; import type { ScrDiagnostic } from "./diagnostics/diagnostic.js"; import type { CoverageInput } from "./coverage/report.js"; +import type { TierMode } from "./frontend/tier-analysis.js"; export type CompileOutputKind = "ir" | "llvm" | "asm" | "obj" | "exe"; @@ -45,6 +46,10 @@ export interface CompileBaseOptions { /** Attach the machine-readable external link recipe to an object result. * Valid only with outputKind "obj"; it never invokes a linker. */ nativeLinkInfo?: boolean; + /** `--tier` (frontend/tier-analysis.ts). "static-only" keeps every demotion + * request a refusal (design invariant I2). Until demotion lands both modes + * compile exactly as before. */ + tier?: TierMode; } /** Executable compile options. This remains the compatibility type for the @@ -161,6 +166,11 @@ export interface AnalyzeOptions { * to analyze project code, but imported runtime values remain explicit * SC1010 blockers rather than being counted as executable. */ externalTypes?: Readonly>; + /** `--tier` (see CompileBaseOptions.tier). */ + tier?: TierMode; + /** `--explain-dynamic`: attach the tier-assignment report + * (CoverageInput.tiers). Such analyses bypass the verdict cache. */ + explainDynamic?: boolean; } export interface AnalyzeResult { diff --git a/packages/compiler/src/coverage/report.ts b/packages/compiler/src/coverage/report.ts index 9d0776d9a..8b03f221d 100644 --- a/packages/compiler/src/coverage/report.ts +++ b/packages/compiler/src/coverage/report.ts @@ -17,10 +17,13 @@ import { type DiagnosticsEnvelope, type LocatedDiagnostic, type SourcePosition, + type TierUnitPosition, + type TiersEnvelope, } from "../diagnostics/envelope.js"; import { renderDiagnostics } from "../diagnostics/render.js"; import type { LowerStats } from "../frontend/lowering/lowerer.js"; import type { ProvenanceSources } from "../frontend/provenance-registry.js"; +import type { DemotionUnit, TierReport } from "../frontend/tier-analysis.js"; export interface CoverageInput { file: string; @@ -51,6 +54,9 @@ export interface CoverageInput { /** --provenance-sources: diagnostics of elided pure-annotated dead * consts (off the build; the provenance section names them). */ provenanceElided?: ScrDiagnostic[]; + /** --explain-dynamic: the report-only tier assignment (which bindings, + * fields, functions, classes and modules the dynamic tier would take). */ + tiers?: TierReport; /** True when tsc preflight failed — no coverage can be computed. */ preflightFailed: boolean; } @@ -61,6 +67,8 @@ export interface NpmStaticStatus { status: "static" | "refused"; /** The first refusal reason for a refused package. */ detail?: string; + /** How to act on the refusal, when there is a specific remedy. */ + hint?: string; } const GREEN = "\x1b[32m"; @@ -180,6 +188,12 @@ export function renderCoverage(input: CoverageInput, opts: CoverageRenderOptions out.push(` statements analyzed ${total}`); out.push(` compile statically ${ok} ${c(pctColor, c(BOLD, `(${pct}%)`))}`); + const dynamic = dynamicStatements(input); + if (dynamic > 0) { + out.push( + ` in dynamic tier ${dynamic} ${c(DIM, "(operate on values typed only at runtime; --explain-dynamic lists them)")}`, + ); + } if (skipped > 0) { out.push( @@ -432,13 +446,147 @@ export function coverageEnvelope( stats: { statementsTotal: total, statementsStatic: ok, - statementsDynamic: 0, + statementsDynamic: dynamicStatements(input), statementsFailed: failed, functionsSkipped: skipped, percentStatic: total === 0 ? 100 : Math.floor((ok / total) * 100), }, }), + ...(input.tiers !== undefined ? { tiers: tiersEnvelope(input.tiers, locator) } : {}), groups, diagnostics: located, }; } + +/** Reached statements in dynamic-tier functions, never more than the + * compiled statements they are part of. */ +function dynamicStatements(input: CoverageInput): number { + const compiled = input.stats.statementsTotal - input.stats.statementsFailed; + return Math.max(0, Math.min(input.stats.statementsDynamic ?? 0, compiled)); +} + +function tiersEnvelope(tiers: TierReport, locator: SourceLocator): TiersEnvelope { + const at = (loc: { file: string; start: number }): SourcePosition => ({ + file: loc.file, + ...locator.position(loc.file, loc.start), + }); + const unit = (u: DemotionUnit): TierUnitPosition => ({ + kind: u.kind, + name: u.name, + ...(u.owner !== undefined ? { owner: u.owner } : {}), + ...at(u.loc), + }); + return { + mode: tiers.mode, + statementsDynamic: tiers.statementsDynamic, + dynamicFunctions: tiers.dynamicFunctions.map((fn) => ({ + name: fn.name, + ...at(fn.loc), + operation: fn.operation, + operationAt: at(fn.operationLoc), + statements: fn.statements, + })), + requests: tiers.requests.map((r) => ({ + unit: unit(r.unit), + enclosing: unit(r.enclosing), + seed: r.seed, + scope: r.scope, + code: r.code, + message: r.message, + ...at(r.loc), + })), + units: tiers.units.map((u) => ({ ...unit(u), requests: u.requests, sites: u.sites })), + }; +} + +/** IR function names as a user reads them: `%init.3` is a module's top + * level, `%Point.norm` the method `Point.norm`; generated helpers keep + * their `%`-prefixed names. */ +function functionDisplayName(name: string): string { + if (name.startsWith("%init.")) return ""; + if (/^%[A-Za-z_$][\w$]*\.[A-Za-z_$#][\w$]*$/.test(name)) return name.slice(1); + return name; +} + +function unitLabel(u: DemotionUnit, root: string | undefined): string { + if (u.kind === "module") return `module ${displayPath(u.name, root)}`; + if (u.kind === "binding" && u.owner !== undefined) return `binding ${u.name} in ${u.owner}`; + return `${u.kind} ${u.name}`; +} + +function unitMatches(u: DemotionUnit, query: string): boolean { + return ( + u.name === query || + u.name.endsWith(`.${query}`) || + u.name.split(" ")[0] === query || + `${u.owner ?? ""}:${u.name}` === query + ); +} + +/** `scriptc coverage --explain-dynamic[=]`: the dynamic tier today + * and the demotion chains the report-only analysis computed. With a query, + * only units whose name matches it (`x`, `Point.x`, `parse:x`). */ +export function renderExplainDynamic( + tiers: TierReport, + opts: CoverageRenderOptions & { query?: string } = {}, +): string { + const c = (code: string, s: string) => (opts.color ? code + s + RESET : s); + const locator = new SourceLocator(opts.sourceTexts); + const where = (loc: { file: string; start: number }) => { + const p = locator.position(loc.file, loc.start); + return `${displayPath(loc.file, opts.root)}:${p.line}:${p.column}`; + }; + const query = opts.query !== undefined && opts.query !== "" ? opts.query : undefined; + const out: string[] = []; + out.push( + ` ${c(BOLD, "dynamic tier")} ${c(DIM, `(tier assignment, report-only; --tier=${tiers.mode})`)}`, + ); + if (query === undefined) { + const fns = tiers.dynamicFunctions; + out.push( + ` ${fns.length} function${fns.length === 1 ? "" : "s"} already operate${fns.length === 1 ? "s" : ""} on dynamic values ${c(DIM, `(${tiers.statementsDynamic} statement${tiers.statementsDynamic === 1 ? "" : "s"})`)}`, + ); + for (const fn of fns) { + out.push( + ` ${functionDisplayName(fn.name)} ${c(DIM, `${where(fn.loc)}: ${fn.operation} at ${where(fn.operationLoc)}`)}`, + ); + } + } + const live = tiers.requests.filter((r) => r.scope !== "unreached" && r.seed !== "excluded"); + const excluded = tiers.requests.filter((r) => r.seed === "excluded").length; + const unreached = tiers.requests.filter((r) => r.scope === "unreached").length; + if (query === undefined) { + out.push(""); + out.push( + ` ${live.length} demotion request${live.length === 1 ? "" : "s"} ${c(DIM, "(refusals and fences the dynamic tier would replace)")}` + + (unreached > 0 ? c(DIM, `, ${unreached} more in unreached code`) : "") + + (excluded > 0 ? c(DIM, `, ${excluded} excluded by policy`) : ""), + ); + } + const units = tiers.units.filter((u) => query === undefined || unitMatches(u, query)); + if (query !== undefined && units.length === 0) { + out.push(` no demoted unit matches '${query}'`); + return out.join("\n"); + } + for (const u of units) { + const counts = [ + u.requests > 0 ? `${u.requests} request${u.requests === 1 ? "" : "s"}` : "", + u.sites > 0 ? `${u.sites} site${u.sites === 1 ? "" : "s"} inside` : "", + ] + .filter((s) => s !== "") + .join(", "); + out.push(` ${unitLabel(u, opts.root)} ${c(DIM, `${where(u.loc)} (${counts})`)}`); + if (query === undefined && u.requests === 0) continue; + for (const r of live) { + if (r.unit.kind !== u.kind || r.unit.name !== u.name || r.unit.loc.start !== u.loc.start) + continue; + if (r.unit.loc.file !== u.loc.file) continue; + out.push( + ` ${c(DIM, "<-")} ${r.code} ${r.seed} at ${where(r.loc)}: ${capMessage(r.message)}`, + ); + if (r.enclosing.name !== r.unit.name || r.enclosing.kind !== r.unit.kind) + out.push(` ${c(DIM, "->")} ${unitLabel(r.enclosing, opts.root)} turns dynamic`); + } + } + return out.join("\n"); +} diff --git a/packages/compiler/src/coverage/verdict-cache.ts b/packages/compiler/src/coverage/verdict-cache.ts index 841e2b57b..250259685 100644 --- a/packages/compiler/src/coverage/verdict-cache.ts +++ b/packages/compiler/src/coverage/verdict-cache.ts @@ -89,6 +89,9 @@ export function verdictKey( opts: AnalyzeOptions, cache: VerdictCacheOptions, ): string | null { + // The tier report carries source locations the semantic replay does not + // move; explaining analyses always lower. + if (opts.explainDynamic === true) return null; let ffi = ""; if (opts.ffiProfilePath !== undefined) { try { @@ -120,6 +123,8 @@ export function verdictKey( opts.ffiProfilePath ?? "", ffi, external.join("\n"), + // Demotion (stage S6) makes the tier mode part of the verdict. + ...(opts.tier === "static-only" ? ["tier=static-only"] : []), ].join("\0"), ); } diff --git a/packages/compiler/src/diagnostics/diagnostic.ts b/packages/compiler/src/diagnostics/diagnostic.ts index eaae1b531..e6ed50ae1 100644 --- a/packages/compiler/src/diagnostics/diagnostic.ts +++ b/packages/compiler/src/diagnostics/diagnostic.ts @@ -877,13 +877,20 @@ export function unsupportedPackageValueDiag(pkg: string, loc: SrcLoc): ScrDiagno ); } -export function npmStaticRefusalDiag(pkg: string, detail: string, loc: SrcLoc): ScrDiagnostic { +export function npmStaticRefusalDiag( + pkg: string, + detail: string, + loc: SrcLoc, + hint?: string, +): ScrDiagnostic { return { code: "SC2013", message: `cannot compile npm package '${pkg}' statically: ${detail}`, loc, milestone: "later", - hint: "use a dependency with supported source, or replace the unsupported operation; scriptc has no interpreter fallback", + hint: + hint ?? + "use a dependency with supported source, or replace the unsupported operation; scriptc has no interpreter fallback", }; } diff --git a/packages/compiler/src/diagnostics/envelope.ts b/packages/compiler/src/diagnostics/envelope.ts index 39b193dd0..4c7c158d9 100644 --- a/packages/compiler/src/diagnostics/envelope.ts +++ b/packages/compiler/src/diagnostics/envelope.ts @@ -81,6 +81,34 @@ export interface DiagnosticGroup { sites: SourcePosition[]; } +export interface TierUnitPosition extends SourcePosition { + kind: "binding" | "field" | "function" | "class" | "module"; + name: string; + owner?: string; +} + +/** `scriptc coverage --explain-dynamic --print=diagnostics`: the tier + * assignment of frontend/tier-analysis.ts with source positions. */ +export interface TiersEnvelope { + mode: "auto" | "static-only"; + statementsDynamic: number; + dynamicFunctions: (SourcePosition & { + name: string; + operation: string; + operationAt: SourcePosition; + statements: number; + })[]; + requests: (SourcePosition & { + unit: TierUnitPosition; + enclosing: TierUnitPosition; + seed: string; + scope: string; + code: string; + message: string; + })[]; + units: (TierUnitPosition & { requests: number; sites: number })[]; +} + export interface DiagnosticsEnvelope { schema: typeof DIAGNOSTICS_SCHEMA; schemaVersion: typeof DIAGNOSTICS_SCHEMA_VERSION; @@ -97,8 +125,10 @@ export interface DiagnosticsEnvelope { stats?: { statementsTotal: number; statementsStatic: number; - /** Always 0: every compiled statement is native. Kept so schema - * version 1 consumers keep reading the same fields. */ + /** The compiled statements (counted in statementsStatic, which covers + * every statement that compiles to native code) that belong to + * dynamic-tier functions: code operating on values whose types are only + * known at runtime. */ statementsDynamic: number; statementsFailed: number; functionsSkipped: number; @@ -106,6 +136,8 @@ export interface DiagnosticsEnvelope { }; /** build only: the produced artifact path. */ artifact?: string; + /** coverage --explain-dynamic only: the report-only tier assignment. */ + tiers?: TiersEnvelope; groups: DiagnosticGroup[]; diagnostics: LocatedDiagnostic[]; } diff --git a/packages/compiler/src/executable/prepare.ts b/packages/compiler/src/executable/prepare.ts index f6012dd2a..d97497c70 100644 --- a/packages/compiler/src/executable/prepare.ts +++ b/packages/compiler/src/executable/prepare.ts @@ -59,6 +59,7 @@ export function prepareExecutableModule( entry.package, entry.detail ?? "its static compilation was refused", fe.npmImportSites.get(entry.package) ?? { file: entryPath, start: 0, end: 0 }, + entry.hint, ), ), ); @@ -69,6 +70,7 @@ export function prepareExecutableModule( lowered = fe.lower({ targetPlatform: buildPlatform, ...(ffi !== null ? { ffiImports: ffi.functions } : {}), + ...(opts.tier !== undefined ? { tierMode: opts.tier } : {}), }); timing("lower"); } catch (e) { diff --git a/packages/compiler/src/frontend/analysis.ts b/packages/compiler/src/frontend/analysis.ts index b38a267fa..f17f6af6c 100644 --- a/packages/compiler/src/frontend/analysis.ts +++ b/packages/compiler/src/frontend/analysis.ts @@ -58,6 +58,7 @@ export function analyzeWithFrontend( entry.package, entry.detail ?? "its static compilation was refused", fe.npmImportSites.get(entry.package) ?? { file: entryPath, start: 0, end: 0 }, + entry.hint, ), ), ]; @@ -90,6 +91,8 @@ export function analyzeWithFrontend( coverage: true, targetPlatform: buildPlatform, ...(ffi !== null ? { ffiImports: ffi.functions } : {}), + ...(opts.tier !== undefined ? { tierMode: opts.tier } : {}), + ...(opts.explainDynamic === true ? { tierReport: { preflight } } : {}), }); const provenance = provenanceSources(); return { @@ -109,6 +112,7 @@ export function analyzeWithFrontend( ...(provenance !== null ? { provenance } : {}), ...(lowered.statsByFile ? { statsByFile: lowered.statsByFile } : {}), ...(lowered.provenanceElided ? { provenanceElided: lowered.provenanceElided } : {}), + ...(lowered.tiers !== undefined ? { tiers: lowered.tiers } : {}), preflightFailed: false, }, sourceTexts: fe.sourceTexts(), diff --git a/packages/compiler/src/frontend/builtin-modules.ts b/packages/compiler/src/frontend/builtin-modules.ts index b45dce6d7..3a1acb934 100644 --- a/packages/compiler/src/frontend/builtin-modules.ts +++ b/packages/compiler/src/frontend/builtin-modules.ts @@ -42,6 +42,8 @@ export const SUPPORTED_BUILTIN_MODULES = [ "module", "console", "process", + "punycode", + "constants", ] as const; /** Builtins Node itself serves ONLY under the node: prefix — diff --git a/packages/compiler/src/frontend/lowering/builtins/class-values.ts b/packages/compiler/src/frontend/lowering/builtins/class-values.ts new file mode 100644 index 000000000..f24db09cf --- /dev/null +++ b/packages/compiler/src/frontend/lowering/builtins/class-values.ts @@ -0,0 +1,159 @@ +import * as ts from "../../ts7/adapter.js"; +import { type Lowerer } from "../lowerer.js"; +import { type ClassInfo } from "../lower-classes.js"; +import { isJsSourceFile } from "../../program.js"; +import { builtinMemberRequireDecl } from "./module-bindings.js"; +import { stripTypeCasts } from "./arguments.js"; +import { streamClassAliasDecl } from "../lower-stream.js"; +import { zlibClassName, zlibStreamModeOf } from "./compression.js"; +import { stdlibGlobalNameOf } from "../surfaces.js"; + +/** A builtin class an expression names at compile time: the error + * classes, node:events EventEmitter, the node:stream classes (each a + * runtime ClassInfo), the node:zlib stream classes (by node_zlib_mode), + * and the Map and Set constructors. `name` is the class's `name` + * property. */ +export type BuiltinClassRef = + | { kind: "error" | "emitter" | "stream"; info: ClassInfo; name: string } + | { kind: "zlib"; mode: number; name: string } + | { kind: "collection"; which: "Map" | "Set"; name: string }; + +/** The builtin class `expr` names: a direct spelling (`EventEmitter`, + * `stream.Readable`, `zlib.Gzip`, `Map`, `globalThis.TypeError`) or a + * `const` binding of one (`const E = EventEmitter`, an alias of an alias, + * an imported alias), through parentheses and type assertions. Null for + * anything else. */ +export function builtinClassRefOf(lowerer: Lowerer, expr: ts.Expression): BuiltinClassRef | null { + const e = stripTypeCasts(expr); + // Checker lookups only: answering must not run the fences and deferred + // diagnostics of a lowered reference (resolveValueSymbol). + let symbol: ts.Symbol | undefined; + if (ts.isIdentifier(e)) { + symbol = lowerer.checker.getSymbolAtLocation(e); + } else if (ts.isPropertyAccessExpression(e) && !e.questionDotToken) { + symbol = lowerer.checker.getSymbolAtLocation(e.name); + } else { + return null; + } + if (!symbol) return null; + const alias = builtinClassAliasOf(lowerer, symbol); + if (alias !== null) return alias; + if (symbol.flags & ts.SymbolFlags.Alias) symbol = lowerer.checker.getAliasedSymbol(symbol); + const which = stdlibGlobalNameOf(lowerer, e); + if (which === "Map" || which === "Set") return { kind: "collection", which, name: which }; + const error = lowerer.builtinErrorInfoOf(symbol); + if (error) return { kind: "error", info: error, name: runtimeClassName(error) }; + const emitter = lowerer.builtinEmitterInfoOf(symbol); + if (emitter) return { kind: "emitter", info: emitter, name: runtimeClassName(emitter) }; + const stream = lowerer.builtinStreamInfoOf(symbol); + if (stream) return { kind: "stream", info: stream, name: runtimeClassName(stream) }; + const mode = zlibStreamModeOf(lowerer, symbol); + if (mode !== null) return { kind: "zlib", mode, name: zlibClassName(mode) }; + return null; +} + +/** The builtin class a `const` binding names (`const E = EventEmitter`, + * `const M = Map`, `const C = zlib.Gzip`, or an alias of such a binding), + * or null. `let` and `var` bindings could be reassigned, and annotated + * bindings give the value another type, so they are not aliases. Resolved from the declaration, so uses lowered before the + * declaration statement (hoisted functions) see the same answer. */ +export function builtinClassAliasOf(lowerer: Lowerer, symbol: ts.Symbol): BuiltinClassRef | null { + const cached = lowerer.builtinClassAliases.get(symbol); + if (cached !== undefined) return cached; + // An import of an exported alias names the exporting binding. + if (symbol.flags & ts.SymbolFlags.Alias) { + lowerer.builtinClassAliases.set(symbol, null); + const target = lowerer.checker.getAliasedSymbol(symbol); + const ref = target === symbol ? null : builtinClassAliasOf(lowerer, target); + lowerer.builtinClassAliases.set(symbol, ref); + return ref; + } + const init = constInitializerOf(lowerer, symbol); + if (init === null) { + lowerer.builtinClassAliases.set(symbol, null); + return null; + } + // Marked first: a cycle through other bindings answers null. + lowerer.builtinClassAliases.set(symbol, null); + const ref = builtinClassRefOf(lowerer, init); + lowerer.builtinClassAliases.set(symbol, ref); + return ref; +} + +/** True when `symbol` is a builtin class alias whose declaration owns no + * runtime slot: new, instanceof, typeof, .name and === lower at each use, + * and other uses report SC1090. In JavaScript files, a binding whose + * initializer has a value (`const E = EventEmitter`, `const M = + * globalThis.Map`, both identity tokens) keeps its slot; only builtin + * module members (`zlib.Gzip`) and copies of slotless aliases own none. + * The `require("events").EventEmitter` and `stream.Writable` spellings + * keep their own declaration handling. */ +export function builtinClassAliasOwnsNoSlot(lowerer: Lowerer, symbol: ts.Symbol): boolean { + if (builtinClassAliasOf(lowerer, symbol) === null) return false; + const binding = + symbol.flags & ts.SymbolFlags.Alias ? lowerer.checker.getAliasedSymbol(symbol) : symbol; + const decl = lowerer.checker.valueDeclarationOf(binding); + if (decl === undefined || !ts.isVariableDeclaration(decl)) return false; + if (builtinMemberRequireDecl(decl.name, decl.initializer)) return false; + if (streamClassAliasDecl(lowerer, decl.name, decl.initializer)) return false; + if (!isJsSourceFile(decl.getSourceFile())) return true; + let init = decl.initializer; + while (init !== undefined && ts.isParenthesizedExpression(init)) init = init.expression; + if (init === undefined) return true; + let source: ts.Symbol | undefined; + if (ts.isPropertyAccessExpression(init)) { + if (lowerer.builtinNamespaceModuleOf(init.expression) !== null) return true; + source = lowerer.checker.getSymbolAtLocation(init.name); + } else { + source = lowerer.checker.getSymbolAtLocation(init); + } + return source !== undefined && builtinClassAliasOwnsNoSlot(lowerer, source); +} + +/** True for a `const` declaration that is a builtin class alias owning no + * runtime slot (see builtinClassAliasOwnsNoSlot): both declaration walks + * skip it. */ +export function registerBuiltinClassAlias( + lowerer: Lowerer, + nameNode: ts.Node, + init: ts.Expression | undefined, +): boolean { + if (!ts.isIdentifier(nameNode) || init === undefined) return false; + const symbol = lowerer.checker.getSymbolAtLocation(nameNode); + return symbol !== undefined && builtinClassAliasOwnsNoSlot(lowerer, symbol); +} + +/** True when two references name the same builtin class. */ +export function sameBuiltinClass(a: BuiltinClassRef, b: BuiltinClassRef): boolean { + if (a.kind === "zlib") return b.kind === "zlib" && a.mode === b.mode; + if (a.kind === "collection") return b.kind === "collection" && a.which === b.which; + return b.kind === a.kind && b.info === a.info; +} + +/** The initializer of an unannotated `const` binding whose initializer + * can name a class (an identifier or a property access), or null. */ +function constInitializerOf(lowerer: Lowerer, symbol: ts.Symbol): ts.Expression | null { + if ((symbol.flags & ts.SymbolFlags.BlockScopedVariable) === 0) return null; + const decl = lowerer.checker.valueDeclarationOf(symbol); + if ( + decl === undefined || + !ts.isVariableDeclaration(decl) || + !ts.isIdentifier(decl.name) || + decl.type !== undefined + ) { + return null; + } + const list = decl.parent; + if (!ts.isVariableDeclarationList(list) || (list.flags & ts.NodeFlags.Const) === 0) return null; + let init = decl.initializer; + while (init !== undefined && ts.isParenthesizedExpression(init)) init = init.expression; + if (init === undefined) return null; + if (ts.isIdentifier(init)) return init; + if (ts.isPropertyAccessExpression(init) && !init.questionDotToken) return init; + return null; +} + +/** Node's constructor name of a runtime class (`%TypeError` → TypeError). */ +function runtimeClassName(info: ClassInfo): string { + return info.def.name.startsWith("%") ? info.def.name.slice(1) : info.def.name; +} diff --git a/packages/compiler/src/frontend/lowering/builtins/compression.ts b/packages/compiler/src/frontend/lowering/builtins/compression.ts index c9553c2f9..6de345dfe 100644 --- a/packages/compiler/src/frontend/lowering/builtins/compression.ts +++ b/packages/compiler/src/frontend/lowering/builtins/compression.ts @@ -132,8 +132,8 @@ export function zlibStreamModeOf( symbol: ts.Symbol | null | undefined, ): number | null { if (!symbol) return null; - const aliasMode = zlibClassAliasModeOf(lowerer, symbol, 0); - if (aliasMode !== undefined) return aliasMode; + const alias = lowerer.builtinClassAliasOf(symbol); + if (alias !== null) return alias.kind === "zlib" ? alias.mode : null; const target = symbol.flags & ts.SymbolFlags.Alias ? lowerer.checker.getAliasedSymbol(symbol) : symbol; const mode = ZLIB_STREAM_MODES.get(target.name); @@ -499,80 +499,12 @@ export function lowerZlibModuleCall( throw new InternalCompilerError(`unhandled lowered zlib member ${bi.member}`); } -/** The zlib stream class an expression names (`zlib.Gzip`, an imported - * `Gzip`, a `const C = zlib.Gzip` alias), as its node_zlib_mode, or null. */ -export function zlibClassModeOfExpr(lowerer: Lowerer, expr: ts.Expression): number | null { - let e = expr; - while (ts.isParenthesizedExpression(e)) e = e.expression; - if (ts.isIdentifier(e)) return zlibStreamModeOf(lowerer, lowerer.checker.getSymbolAtLocation(e)); - if (ts.isPropertyAccessExpression(e) && !e.questionDotToken) { - return zlibStreamModeOf(lowerer, lowerer.checker.getSymbolAtLocation(e.name)); - } - return null; -} - /** The class name of a zlib stream mode (Node's constructor name). */ export function zlibClassName(mode: number): string { for (const [name, m] of ZLIB_STREAM_MODES) if (m === mode) return name; return "Zlib"; } -/** The mode of a `const C = zlib.Gzip` binding (or an alias of such a - * binding), null when the symbol is a const binding of something else, - * undefined when it is not a const binding. Resolved from the declaration, - * so uses lowered before the declaration statement see the same answer. */ -function zlibClassAliasModeOf( - lowerer: Lowerer, - symbol: ts.Symbol, - depth: number, -): number | null | undefined { - const cached = lowerer.zlibClassAliases.get(symbol); - if (cached !== undefined) return cached; - if ((symbol.flags & ts.SymbolFlags.BlockScopedVariable) === 0) return undefined; - const decl = lowerer.checker.valueDeclarationOf(symbol); - if (decl === undefined || !ts.isVariableDeclaration(decl) || !ts.isIdentifier(decl.name)) { - return undefined; - } - // const only: a var or let binding could be reassigned - const list = decl.parent; - if (!ts.isVariableDeclarationList(list) || (list.flags & ts.NodeFlags.Const) === 0) { - return undefined; - } - let init = decl.initializer; - while (init !== undefined && ts.isParenthesizedExpression(init)) init = init.expression; - if (init === undefined || depth > 16) return null; - let mode: number | null = null; - if (ts.isIdentifier(init)) { - const next = lowerer.checker.getSymbolAtLocation(init); - if (next !== undefined) { - const nested = zlibClassAliasModeOf(lowerer, next, depth + 1); - mode = nested === undefined ? zlibStreamModeOf(lowerer, next) : nested; - } - } else if (ts.isPropertyAccessExpression(init) && !init.questionDotToken) { - mode = zlibStreamModeOf(lowerer, lowerer.checker.getSymbolAtLocation(init.name)); - } - if (mode !== null) lowerer.zlibClassAliases.set(symbol, mode); - return mode; -} - -/** True for `const C = zlib.Gzip` (or an alias of an alias): a - * compile-time zlib class alias that owns no runtime slot, so the - * statement lowering skips it. */ -export function registerZlibClassAlias( - lowerer: Lowerer, - nameNode: ts.Node, - init: ts.Expression | undefined, -): boolean { - if (!ts.isIdentifier(nameNode) || init === undefined) return false; - const symbol = lowerer.checker.getSymbolAtLocation(nameNode); - return symbol !== undefined && zlibClassAliasOf(lowerer, symbol) !== null; -} - -/** The mode of a `const C = zlib.Gzip` alias symbol, or null. */ -export function zlibClassAliasOf(lowerer: Lowerer, symbol: ts.Symbol): number | null { - return zlibClassAliasModeOf(lowerer, symbol, 0) ?? null; -} - /** `value instanceof zlib.Gzip` (any spelling of the class, aliases * included). zlib streams are runtime Transforms: a Transform operand asks * the runtime for the stream's mode, and an operand typed as one of diff --git a/packages/compiler/src/frontend/lowering/builtins/numbers.ts b/packages/compiler/src/frontend/lowering/builtins/numbers.ts index 08d6cb349..4f86e6f74 100644 --- a/packages/compiler/src/frontend/lowering/builtins/numbers.ts +++ b/packages/compiler/src/frontend/lowering/builtins/numbers.ts @@ -2,7 +2,7 @@ import * as ts from "../../ts7/adapter.js"; import { type Lowerer, own } from "../lowerer.js"; import { lowerCheckedPredicateValue } from "../lower-builtin-values.js"; import { locOf } from "../../program.js"; -import { BOOL, F64, type IrExpr, type IrLibFn } from "../../../ir/ir.js"; +import { BOOL, F64, STRING, type IrExpr, type IrLibFn } from "../../../ir/ir.js"; import { lowerBuiltinValuePreservingUndefined, lowerOptionalNumberPredicate } from "./arguments.js"; import { tryLowerExpression } from "../expressions/try-lower-expression.js"; import { lowerStaticNumberGlobal } from "../lower-calls.js"; @@ -88,6 +88,22 @@ export function lowerNumberStaticCall( const parsed = lowerStaticNumberGlobal(lowerer, member, call, call.arguments, loc); if (parsed !== null) return parsed; } + // A checked-dynamic argument (`Number.parseInt(env.FORCE_COLOR, 10)` + // over a JavaScript dictionary read): the parsers' own ToString, + // exactly the global parseFloat's dyn form. + if (probed?.type.kind === "dyn") { + const text: IrExpr = { + kind: "libCall", + fn: "dyn.toStringCoerce", + args: [probed], + type: STRING, + loc, + }; + if (member === "parseFloat") + return { kind: "libCall", fn: "num.parseFloat", args: [text], type: F64, loc }; + const radix = lowerer.lowerExprExpecting(call.arguments[1]!, F64); + return { kind: "libCall", fn: "num.parseInt", args: [text, radix], type: F64, loc }; + } const argText = probed !== null ? lowerer.fmt(probed.type) diff --git a/packages/compiler/src/frontend/lowering/builtins/process.ts b/packages/compiler/src/frontend/lowering/builtins/process.ts index 97bf1e963..94c2cdba7 100644 --- a/packages/compiler/src/frontend/lowering/builtins/process.ts +++ b/packages/compiler/src/frontend/lowering/builtins/process.ts @@ -233,6 +233,41 @@ export function lowerProcessStreamProperty( * read in source order). Method members referenced without a call are * rejected specifically. Null for non-process receivers (the chain keeps * trying other property lowerings). */ +/** The value of a DATA member of the process global taken off by + * destructuring (`const { env, argv } = process` in JavaScript, the + * supports-color opening): the same values the spelled `process.env` / + * `process.argv` reads produce, typed by the binding. Null for members + * this does not cover (methods keep the destructuring fence). */ +export function processDataMemberValue( + lowerer: Lowerer, + member: string, + binding: ts.Node, + loc: SrcLoc, +): IrExpr | null { + switch (member) { + case "env": { + const mapped = lowerer.mapTypeOf(lowerer.typeOf(binding)); + if (mapped?.kind !== "record") return null; + const helper = envSnapshotHelper(lowerer, mapped.shapeId, loc); + return helper === null ? null : { kind: "call", callee: helper, args: [], type: mapped, loc }; + } + case "argv": + return { kind: "libCall", fn: "process.argv", args: [], type: arrayOf(STRING), loc }; + case "platform": + return { kind: "libCall", fn: "process.platform", args: [], type: STRING, loc }; + case "arch": + return { kind: "libCall", fn: "process.arch", args: [], type: STRING, loc }; + case "pid": + return { kind: "libCall", fn: "process.pid", args: [], type: F64, loc }; + case "execPath": + return { kind: "libCall", fn: "process.execPath", args: [], type: STRING, loc }; + case "versions": + return { kind: "libCall", fn: "process.versions", args: [], type: DYN, loc }; + default: + return null; + } +} + export function lowerProcessProperty( lowerer: Lowerer, expr: ts.PropertyAccessExpression, diff --git a/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts b/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts index 444a9cf23..120032bf5 100644 --- a/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts +++ b/packages/compiler/src/frontend/lowering/class-dynamic-dispatch.ts @@ -1159,7 +1159,9 @@ export class ClassDynamicDispatch { // Runtime-owned roots keep their dedicated property contracts. A // family whose fields all exist from construction and are never // deleted keeps a fixed layout, as TypeScript classes do. - this.deletedNames ??= deletedPropertyNames(lowerer.moduleOrder); + // null is a result too; undefined means not computed yet. + if (this.deletedNames === undefined) + this.deletedNames = deletedPropertyNames(lowerer.moduleOrder); if (!root.base?.def.runtime && !fixedFieldFamily(root, this.deletedNames)) track(root); } const loc = this.propertyBag.loc; diff --git a/packages/compiler/src/frontend/lowering/construction-escapes.ts b/packages/compiler/src/frontend/lowering/construction-escapes.ts index a1626f12d..484920a1c 100644 --- a/packages/compiler/src/frontend/lowering/construction-escapes.ts +++ b/packages/compiler/src/frontend/lowering/construction-escapes.ts @@ -124,15 +124,6 @@ export function constructionEscape(inputs: ConstructionInputs): ConstructionEsca continue; } if (ts.isExpressionStatement(stmt) && !exposes(stmt.expression, depth)) continue; - // A check that only reads assigned fields and throws or does nothing - // with the receiver (`if (this.size < 0) throw new RangeError()`) - // does not expose the instance; assignments in its branches are not - // definite and still count as exposures. - if ( - (ts.isIfStatement(stmt) || ts.isThrowStatement(stmt) || ts.isBlock(stmt)) && - !exposes(stmt, depth) - ) - continue; return false; } return true; diff --git a/packages/compiler/src/frontend/lowering/function-returns.ts b/packages/compiler/src/frontend/lowering/function-returns.ts index d3c5eb1ba..5ba63be26 100644 --- a/packages/compiler/src/frontend/lowering/function-returns.ts +++ b/packages/compiler/src/frontend/lowering/function-returns.ts @@ -429,11 +429,14 @@ export function resolveInferredReturn( if (ice.length > 0) lowerer.diags.push(...ice); lowerer.runtimeFences.push(...captured.filter((d) => d.code !== "SC9001")); const first = captured.find((d) => d.code !== "SC9001"); + // Property writes, not literal element writes: the class method + // census reads `x["kind"] = …` as a possible write to every method + // named `kind`, which would make those methods dynamic in native builds. const mutable = e.stmt as unknown as Record; - delete mutable["value"]; - mutable["kind"] = "runtimeFence"; - mutable["code"] = first?.code ?? "SC1090"; - mutable["message"] = first + delete mutable.value; + mutable.kind = "runtimeFence"; + mutable.code = first?.code ?? "SC1090"; + mutable.message = first ? `${first.message} [${first.code}]` : "this return value has no lowering onto the instance's settled return type [SC1090]"; } diff --git a/packages/compiler/src/frontend/lowering/js-dynamic-objects.test.ts b/packages/compiler/src/frontend/lowering/js-dynamic-objects.test.ts new file mode 100644 index 000000000..23ebdfaa9 --- /dev/null +++ b/packages/compiler/src/frontend/lowering/js-dynamic-objects.test.ts @@ -0,0 +1,38 @@ +import { afterAll, expect, test } from "vitest"; +import * as ts from "../ts7/adapter.js"; +import { closeSourceParser, parseSourceFile } from "../ts7/source-parser-node.js"; +import { isValueReturningConstructorClass } from "./js-dynamic-objects.js"; + +afterAll(closeSourceParser); + +function classify(source: string): boolean { + const file = parseSourceFile("classes.js", source, "js"); + let decl: ts.ClassDeclaration | undefined; + ts.walkPreorder(file, (node) => { + if (!decl && ts.isClassDeclaration(node)) decl = node; + }); + if (!decl) throw new Error("fixture has no class"); + return isValueReturningConstructorClass(decl); +} + +test("a constructor-only class returning a value is claimed", () => { + expect(classify(`class Chalk { constructor(options) { return chalkFactory(options); } }`)).toBe( + true, + ); + expect(classify(`class C { constructor(a) { if (a) return { a }; return [a]; } ; }`)).toBe(true); + expect(classify(`class C { constructor() { const f = () => 1; return { f }; } }`)).toBe(true); +}); + +test("classes that keep their instance layout are left alone", () => { + // No value return, or only nested functions return. + expect(classify(`class C { constructor() { return; } }`)).toBe(false); + expect(classify(`class C { constructor() { function f() { return 1; } f(); } }`)).toBe(false); + expect(classify(`class C { constructor() { const f = () => { return 1; }; f(); } }`)).toBe(false); + // Other members, heritage, or a constructor that observes `this`. + expect(classify(`class C { constructor() { return {}; } m() {} }`)).toBe(false); + expect(classify(`class C { x = 1; constructor() { return {}; } }`)).toBe(false); + expect(classify(`class C extends B { constructor() { super(); return {}; } }`)).toBe(false); + expect(classify(`class C { constructor() { this.a = 1; return {}; } }`)).toBe(false); + expect(classify(`class C { constructor() { const g = () => this; return { g }; } }`)).toBe(false); + expect(classify(`class C {}`)).toBe(false); +}); diff --git a/packages/compiler/src/frontend/lowering/js-dynamic-objects.ts b/packages/compiler/src/frontend/lowering/js-dynamic-objects.ts new file mode 100644 index 000000000..1d7ab6eee --- /dev/null +++ b/packages/compiler/src/frontend/lowering/js-dynamic-objects.ts @@ -0,0 +1,312 @@ +/* JavaScript declarations whose values only the checked-dynamic + * representation can hold faithfully. TypeScript infers a fixed object + * type for each of them, but the program changes the value's shape or + * identity at run time, so a static layout would answer stale members + * (or throw on a value that no longer fits): + * + * - a non-empty object literal bound to a variable that the same file + * grows or reshapes dynamically: a keyed write with a runtime key + * (`o[k] = v`), a `delete`, or `Object.defineProperty(o, ...)` / + * `Object.defineProperties(o, ...)` (ansi-styles' `assembleStyles`). + * The object literals nested in its property values belong to the same + * value graph and are marked with it; + * - a class whose only member is a constructor that returns a value and + * never reads `this` (chalk's `class Chalk { constructor(options) { + * return chalkFactory(options); } }`): `new C()` answers whatever the + * constructor returns, so its instance type has no fixed layout. + * + * The type mapper answers the checked-dynamic type for the instance and + * object-literal types these declarations own (TypeMapperCtx + * .jsDynamicDeclaration). The scan is syntactic, runs over JavaScript + * files only, and an empty answer leaves every mapping unchanged, so + * TypeScript programs and JavaScript that never uses these patterns + * compile exactly as before. */ +import * as ts from "../ts7/adapter.js"; +import { isJsSourceFile, locOf } from "../program.js"; +import { BOOL, DYN, type IrExpr, type IrStmt } from "../../ir/ir.js"; +import { dynUndefinedExpr, nodeThrowExpr } from "../../ir/build.js"; +import type { Lowerer } from "./lowerer.js"; +import type { ParamShape } from "./call-signatures.js"; +import { newFnCtx } from "./function-context.js"; + +const SCANNED_KINDS: ReadonlySet = new Set([ + ts.SyntaxKind.VariableDeclaration, + ts.SyntaxKind.ClassDeclaration, + ts.SyntaxKind.BinaryExpression, + ts.SyntaxKind.DeleteExpression, + ts.SyntaxKind.CallExpression, +]); + +function unparenthesized(node: ts.Expression): ts.Expression { + while (ts.isParenthesizedExpression(node)) node = node.expression; + return node; +} + +function isLiteralKey(node: ts.Expression): boolean { + const key = unparenthesized(node); + return ( + ts.isStringLiteral(key) || ts.isNumericLiteral(key) || ts.isNoSubstitutionTemplateLiteral(key) + ); +} + +function isAssignmentOperator(kind: ts.SyntaxKind): boolean { + return kind >= ts.SyntaxKind.FirstAssignment && kind <= ts.SyntaxKind.LastAssignment; +} + +/** `class C { constructor(...) { ...; return value; } }` with nothing else: + * no heritage, decorators or other members, a constructor body that + * returns a value at its own level and never mentions `this`/`super`. */ +export function isValueReturningConstructorClass(decl: ts.ClassDeclaration): boolean { + if (decl.heritageClauses !== undefined && decl.heritageClauses.length > 0) return false; + if (decl.modifiers?.some((modifier) => modifier.kind === ts.SyntaxKind.Decorator)) return false; + let ctor: ts.ConstructorDeclaration | undefined; + for (const member of decl.members) { + if (ts.isSemicolonClassElement(member)) continue; + if (!ts.isConstructorDeclaration(member) || ctor !== undefined) return false; + ctor = member; + } + if (ctor?.body === undefined) return false; + let returnsValue = false; + let observesThis = false; + const visit = (node: ts.Node): void => { + if (observesThis) return; + if (node.kind === ts.SyntaxKind.ThisKeyword || node.kind === ts.SyntaxKind.SuperKeyword) { + observesThis = true; + return; + } + // Arrow functions share the constructor's `this`; other functions + // have their own `this` and their own returns. + if (ts.isFunctionLike(node) && !ts.isArrowFunction(node)) return; + if (ts.isClassDeclaration(node) || ts.isClassExpression(node)) return; + if (ts.isReturnStatement(node) && node.expression !== undefined && !insideArrow(node, ctor!)) + returnsValue = true; + node.forEachChild(visit); + }; + ctor.body.forEachChild(visit); + for (const parameter of ctor.parameters) parameter.forEachChild(visit); + return returnsValue && !observesThis; +} + +function insideArrow(node: ts.Node, ctor: ts.ConstructorDeclaration): boolean { + for (let p = node.parent; p !== undefined && p !== ctor; p = p.parent) { + if (ts.isArrowFunction(p)) return true; + } + return false; +} + +function markNested(literal: ts.ObjectLiteralExpression, out: Set): void { + if (out.has(literal)) return; + out.add(literal); + for (const property of literal.properties) { + if (!ts.isPropertyAssignment(property)) continue; + const value = unparenthesized(property.initializer); + if (ts.isObjectLiteralExpression(value)) markNested(value, out); + } +} + +export function jsDynamicDeclarations( + files: readonly ts.SourceFile[], + checker: ts.TypeChecker, +): Set { + const out = new Set(); + for (const file of files) { + if (!isJsSourceFile(file) || file.isDeclarationFile) continue; + const literals = new Map(); + const reshaped = new Set(); + const receiver = (expression: ts.Expression | undefined): void => { + if (expression === undefined) return; + const target = unparenthesized(expression); + if (!ts.isIdentifier(target)) return; + const symbol = checker.getSymbolAtLocation(target); + if (symbol !== undefined) reshaped.add(symbol); + }; + ts.walkPreorder( + file, + (node) => { + if (ts.isVariableDeclaration(node)) { + if (!ts.isIdentifier(node.name) || node.initializer === undefined) return; + const init = unparenthesized(node.initializer); + // An empty literal's binding is already checked-dynamic + // (jsOpenObjectType), and its type describes no layout. + if (!ts.isObjectLiteralExpression(init) || init.properties.length === 0) return; + const symbol = checker.getSymbolAtLocation(node.name); + if (symbol !== undefined) literals.set(symbol, init); + return; + } + if (ts.isClassDeclaration(node)) { + if (isValueReturningConstructorClass(node)) out.add(node); + return; + } + if (ts.isBinaryExpression(node)) { + if (!isAssignmentOperator(node.operatorToken.kind)) return; + const left = unparenthesized(node.left); + if (ts.isElementAccessExpression(left) && !isLiteralKey(left.argumentExpression)) + receiver(left.expression); + return; + } + if (ts.isDeleteExpression(node)) { + const operand = unparenthesized(node.expression); + if (ts.isElementAccessExpression(operand) || ts.isPropertyAccessExpression(operand)) + receiver(operand.expression); + return; + } + if (ts.isCallExpression(node)) { + const callee = unparenthesized(node.expression); + if ( + ts.isPropertyAccessExpression(callee) && + ts.isIdentifier(callee.expression) && + callee.expression.text === "Object" && + (callee.name.text === "defineProperty" || callee.name.text === "defineProperties") + ) + receiver(node.arguments[0]); + } + }, + SCANNED_KINDS, + ); + for (const symbol of reshaped) { + const literal = literals.get(symbol); + if (literal !== undefined) markNested(literal, out); + } + } + return out; +} + +/** `new C(args)` for a value-returning constructor class (above): the + * constructor body runs as an ordinary JavaScript function whose result + * is the construction's value. A returned object or function is Node's + * answer; a fall-through or primitive return would answer the bare + * instance, which has no static home here, and throws the deferred + * refusal instead. Null when the callee is not such a class. */ +export function lowerValueReturningConstructorNew( + lowerer: Lowerer, + expr: ts.NewExpression, +): IrExpr | null { + const isDynamic = lowerer.typeCtx.jsDynamicDeclaration; + if (isDynamic === undefined) return null; + const callee = unparenthesized(expr.expression); + if (!ts.isIdentifier(callee)) return null; + let symbol = lowerer.resolveValueSymbol(callee); + if (symbol !== null && (symbol.flags & ts.SymbolFlags.Alias) !== 0) + symbol = lowerer.checker.getAliasedSymbol(symbol); + const decl = symbol + ? lowerer.checker + .declarationsOf(symbol) + .find((node): node is ts.ClassDeclaration => ts.isClassDeclaration(node) && isDynamic(node)) + : undefined; + if (decl === undefined) return null; + const loc = locOf(expr); + const ctor = decl.members.find(ts.isConstructorDeclaration)!; + const simple = ctor.parameters.every( + (p) => ts.isIdentifier(p.name) && p.dotDotDotToken === undefined && p.initializer === undefined, + ); + if (!simple || !ts.isSourceFile(decl.parent)) { + lowerer.unsupported( + "SC1090", + expr, + "constructing a class whose constructor returns a value, with parameter defaults, rest or patterns, or declared inside a function", + ); + } + const args = expr.arguments ?? []; + if (args.some(ts.isSpreadElement)) { + lowerer.unsupported( + "SC1090", + expr, + "spread arguments when constructing a class whose constructor returns a value", + ); + } + const key = `ctorValue:${decl.getSourceFile().fileName}:${decl.getStart()}`; + let name = lowerer.valueHelpers.get(key); + if (name === undefined) { + name = `%ctorValue.${lowerer.valueHelpers.size}`; + lowerer.valueHelpers.set(key, name); + const savedClass = lowerer.currentClass; + lowerer.currentClass = null; + lowerer.fnStack.push(newFnCtx(false, null, null, DYN)); + try { + const shapes: ParamShape[] = ctor.parameters.map(() => ({ type: DYN, mode: "required" })); + const declared = lowerer.declareParams(ctor.parameters, shapes); + const body: IrStmt[] = [ + ...declared.prologue, + ...lowerer.lowerStmts(ctor.body!.statements), + { + kind: "runtimeFence", + code: "SC1090", + message: `constructors that return no object (the bare instance of '${decl.name?.text ?? "class"}' has no representation here) are not supported yet [SC1090 at ${decl.getSourceFile().fileName}:${ts.getLineAndCharacterOfPosition(decl.getSourceFile(), ctor.getStart()).line + 1}]`, + loc: locOf(ctor), + }, + ]; + lowerer.liftedFns.push({ + name, + params: declared.params, + returnType: DYN, + locals: lowerer.ctx.locals, + body, + loc: locOf(ctor), + }); + } finally { + lowerer.fnStack.pop(); + lowerer.currentClass = savedClass; + } + } + const lowered = ctor.parameters.map((_, i) => + args[i] !== undefined ? lowerer.lowerExprExpecting(args[i]!, DYN) : dynUndefinedExpr(loc), + ); + // Surplus arguments still evaluate, left to right, after the others. + const surplus: IrStmt[] = args + .slice(ctor.parameters.length) + .map((arg) => ({ kind: "exprStmt", expr: lowerer.lowerExpr(arg), loc: locOf(arg) })); + const result = lowerer.declareHiddenLocal("%constructed", DYN); + const value: IrExpr = { kind: "varRef", localId: result.id, type: DYN, loc }; + const sf = expr.getSourceFile(); + const line = ts.getLineAndCharacterOfPosition(sf, expr.getStart()).line + 1; + const objectLike: IrExpr = { + kind: "logical", + op: "||", + left: { kind: "dynTest", test: "function", value, type: BOOL, loc }, + right: { + kind: "logical", + op: "&&", + left: { kind: "dynTest", test: "object", value, type: BOOL, loc }, + right: { kind: "dynTest", test: "null", negated: true, value, type: BOOL, loc }, + type: BOOL, + loc, + }, + type: BOOL, + loc, + }; + const args0: IrStmt[] = []; + const argLocals = lowered.map((arg) => { + const local = lowerer.declareHiddenLocal("%constructArg", DYN); + args0.push({ kind: "varDecl", localId: local.id, init: arg, loc }); + return { kind: "varRef", localId: local.id, type: DYN, loc } as IrExpr; + }); + return { + kind: "seqExpr", + stmts: [ + ...args0, + ...surplus, + { + kind: "varDecl", + localId: result.id, + init: { kind: "call", callee: name, args: argLocals, type: DYN, loc }, + loc, + }, + ], + result: { + kind: "ternary", + cond: objectLike, + then: value, + else_: nodeThrowExpr( + 0, + "SC1090", + `constructors that return no object are not supported yet [SC1090 at ${sf.fileName}:${line}]`, + DYN, + loc, + ), + type: DYN, + loc, + }, + type: DYN, + loc, + }; +} diff --git a/packages/compiler/src/frontend/lowering/lower-builtin-values.ts b/packages/compiler/src/frontend/lowering/lower-builtin-values.ts index a9df9e662..c05430ad3 100644 --- a/packages/compiler/src/frontend/lowering/lower-builtin-values.ts +++ b/packages/compiler/src/frontend/lowering/lower-builtin-values.ts @@ -23,6 +23,7 @@ import { builtinModulesArrayLit, stdlibGlobalNameOf, } from "./surfaces.js"; +import { CRYPTO_CONSTANTS } from "./crypto-tables.js"; const PROTOTYPE_METHODS: Record = { Date: ["getTime", "valueOf", "toISOString", "toJSON"], @@ -680,3 +681,34 @@ export function lowerBuiltinLoaderValue(lowerer: Lowerer, loc: SrcLoc): IrExpr { loc, }; } + +/** The deprecated node:constants module object: the runtime merges + * os.constants' groups and fs.constants with crypto.constants, which a + * lifted function answers from the baked table the first time the object + * is built. */ +export function builtinConstantsObject(lowerer: Lowerer, loc: SrcLoc): IrExpr { + const name = "%builtin.constants.crypto"; + if (!lowerer.builtinCallableValueFns.has(name)) { + lowerer.builtinCallableValueFns.set(name, name); + const fields = Object.entries(CRYPTO_CONSTANTS).map(([key, value]) => ({ + key: strLit(key, loc), + value: box(builtinConstLit(value, loc)), + })); + lowerer.liftedFns.push({ + name, + params: [], + returnType: DYN, + locals: [], + body: [{ kind: "return", value: { kind: "dynObjLit", fields, type: DYN, loc }, loc }], + loc, + }); + } + const table: IrExpr = { + kind: "closure", + fnName: name, + captures: [], + type: { kind: "func", params: [], ret: DYN }, + loc, + }; + return { kind: "libCall", fn: "constants.object", args: [box(table)], type: DYN, loc }; +} diff --git a/packages/compiler/src/frontend/lowering/lower-calls.ts b/packages/compiler/src/frontend/lowering/lower-calls.ts index 3260c0484..2037ba17c 100644 --- a/packages/compiler/src/frontend/lowering/lower-calls.ts +++ b/packages/compiler/src/frontend/lowering/lower-calls.ts @@ -3480,13 +3480,41 @@ function lowerDynReceiverMethodCall( // its name overlaps a primitive prototype (`colors.bold(...)`): the // source value was compiled from that object shape, while genuinely // untyped/any receivers retain the runtime-kind ambiguity and fence. - if (DYN_PROTO_METHOD_NAMES.has(access.name.text) && !ownCallable) return null; + // A name only String.prototype declares (`bold`, `italics`, `link`, + // ...) on a checker-untyped receiver: every other dyn kind answers its + // own or inherited member exactly like a non-prototype name (chalk's + // `chalk.red.bold(...)` reads a prototype getter of a function), so + // only a string receiver keeps the refusal, deferred to run time. + const stringOnlyName = + checkerUntyped && !ownCallable && DYN_STRING_PROTO_ONLY_NAMES.has(access.name.text); + if (DYN_PROTO_METHOD_NAMES.has(access.name.text) && !ownCallable && !stringOnlyName) return null; // Optional forms (`obj.cb?.()`, `obj?.cb()`) belong to the chain // machinery's short-circuit semantics — not modeled here yet. if (call.questionDotToken || access.questionDotToken) return null; const loc = locOf(call); const receiverLocal = lowerer.declareHiddenLocal("%dynCallReceiver", DYN); const receiver: IrExpr = { kind: "varRef", localId: receiverLocal.id, type: DYN, loc }; + // The string refusal throws in place of the member read (a seqExpr + // statement cannot leave the expression). + const stringRefusal = (read: IrExpr): IrExpr => { + if (!stringOnlyName) return read; + const sf = call.getSourceFile(); + const line = ts.getLineAndCharacterOfPosition(sf, call.getStart(sf)).line + 1; + return { + kind: "ternary", + cond: { kind: "dynTest", test: "string", value: receiver, type: BOOL, loc }, + then: nodeThrowExpr( + 0, + "SC1090", + `method calls like '${access.getText()}' on a string are not supported yet [SC1090 at ${sf.fileName}:${line}]`, + DYN, + loc, + ), + else_: read, + type: DYN, + loc, + }; + }; const member: IrExpr = { kind: "dynKeyGet", key: { kind: "strLit", value: access.name.text, type: STRING, loc: locOf(access) }, @@ -3498,7 +3526,7 @@ function lowerDynReceiverMethodCall( const callee = varRef(calleeLocal.id, DYN, loc); const stmts: IrStmt[] = [ { kind: "varDecl", localId: receiverLocal.id, init: recv, loc }, - { kind: "varDecl", localId: calleeLocal.id, init: member, loc }, + { kind: "varDecl", localId: calleeLocal.id, init: stringRefusal(member), loc }, ]; if (call.arguments.some(ts.isSpreadElement)) { const spread = lowerSpreadArgsCall(lowerer, call, callee, loc); @@ -3607,6 +3635,31 @@ const DYN_PROTO_METHOD_NAMES = new Set([ "toPrecision", ]); +/** The DYN_PROTO_METHOD_NAMES that only String.prototype declares among + * the dyn-representable kinds' prototypes. */ +const DYN_STRING_PROTO_ONLY_NAMES = new Set([ + "anchor", + "big", + "blink", + "bold", + "codePointAt", + "fixed", + "fontcolor", + "fontsize", + "isWellFormed", + "italics", + "link", + "localeCompare", + "normalize", + "small", + "strike", + "sub", + "sup", + "toLocaleLowerCase", + "toLocaleUpperCase", + "toWellFormed", +]); + /** The SHARED prototype names scr_dyn_invoke dispatches at runtime (the * subset of DYN_PROTO_METHOD_NAMES with a receiver-kind dispatch): the * runtime runs the real method for the receiver's kind, throws Node's diff --git a/packages/compiler/src/frontend/lowering/lower-classes.ts b/packages/compiler/src/frontend/lowering/lower-classes.ts index 7b2d176c3..f3a4830c3 100644 --- a/packages/compiler/src/frontend/lowering/lower-classes.ts +++ b/packages/compiler/src/frontend/lowering/lower-classes.ts @@ -1,3 +1,4 @@ +import { lowerValueReturningConstructorNew } from "./js-dynamic-objects.js"; import { dynUndefinedExpr, nodeThrowExpr, varRef } from "../../ir/build.js"; import { checkedIterableSpread } from "./checked-iterable-spread.js"; import { lowerReadlineUiNew } from "./builtins/readline.js"; @@ -11,6 +12,11 @@ import { InternalCompilerError } from "../../errors.js"; import { fsStreamClassOf, lowerFsStreamNew } from "./builtins/fs-streams.js"; import * as ts from "../ts7/adapter.js"; import { lowerZlibStreamNew, zlibClassName, zlibStreamModeOf } from "./builtins/compression.js"; +import { + type BuiltinClassRef, + builtinClassAliasOf, + builtinClassRefOf, +} from "./builtins/class-values.js"; import type { FnCtx, Lowerer } from "./lowerer.js"; import { BOOL, @@ -143,7 +149,7 @@ import { lowerObjectFactoryNew } from "./object-factory-new.js"; import { tryLowerExpression } from "./expressions/try-lower-expression.js"; import { lowerInstanceConstructorNew } from "./class-instance-constructor.js"; import { classPrototypeData } from "./class-prototypes.js"; -import { constructionEscape, mentionsReceiver } from "./construction-escapes.js"; +import { constructionEscape } from "./construction-escapes.js"; import { checkedClassConstruction, checkedClassConstructionPacked } from "./class-construction.js"; import { initializeRuntimeStatics, hasRuntimeStatics } from "./class-runtime-statics.js"; import { reflectedClassStaticMethodValue } from "./class-method-values.js"; @@ -468,11 +474,6 @@ export interface ClassInfo { * unassigned read throws the catchable TypeError instead of yielding * an undefined the declared type cannot hold (SEMANTICS.md). */ deferredInitFields?: Set; - /** Fields (inherited included) that code construction exposes the - * instance to can read before their assignment: the slot is the - * undefined-armed union, allocation writes undefined, and reads yield the - * union value, so an early read sees undefined as in Node. */ - earlyReadFields?: Set; /** True when construction may hand the instance to other code (a method * call, an argument, a callback, or a base constructor that does) — a * subclass's own fields can then be read before their assignment. */ @@ -928,12 +929,15 @@ export function registerBuiltinStreamClasses(lowerer: Lowerer): void { * "stream" ambient module, EXCLUDING @types/node's (whose stream.Readable * also types child stdio — under @types/node the childStream mapping * keeps priority and the static stream classes stand down; the shipped - * fallback declarations are the supported surface). */ + * fallback declarations are the supported surface). A const binding of + * one (`const R = Readable`) names the same class (builtinClassAliasOf). */ export function builtinStreamInfoOf( lowerer: Lowerer, symbol: ts.Symbol | null | undefined, ): ClassInfo | null { if (!symbol) return null; + const alias = builtinClassAliasOf(lowerer, symbol); + if (alias !== null) return alias.kind === "stream" ? alias.info : null; if (!lowerer.isStdlibSymbol(symbol)) { // A const ALIAS of a namespace member (`const Writable = // stream.Writable` — the two-step spelling; the one-step @@ -1288,12 +1292,16 @@ function undefArmedFieldType(lowerer: Lowerer, p: ts.Symbol): IrType | null { * `extends EventEmitter`, `x instanceof EventEmitter`) — any import * spelling (named/default/namespace member, CJS require) resolves to * the ambient class. Provenance-checked like the error classes: only a - * stdlib-file declaration inside the "events" ambient module counts. */ + * stdlib-file declaration inside the "events" ambient module counts. A + * const binding of it (`const E = EventEmitter`) names the same class + * (builtinClassAliasOf). */ export function builtinEmitterInfoOf( lowerer: Lowerer, symbol: ts.Symbol | null | undefined, ): ClassInfo | null { if (!symbol) return null; + const alias = builtinClassAliasOf(lowerer, symbol); + if (alias !== null) return alias.kind === "emitter" ? alias.info : null; if (!lowerer.isStdlibSymbol(symbol)) { // A const ALIAS of the emitter class member (`const EventEmitter = // require('node:events').EventEmitter` — commander's spelling; the @@ -1334,12 +1342,17 @@ export function builtinEmitterInfoOf( /** The builtin error ClassInfo a VALUE symbol refers to (`new Error`, * `extends TypeError`, `x instanceof RangeError`), or null. Provenance- * checked: only the standard library's declarations count — a user's own - * `class Error` resolves through classBySymbol instead. */ + * `class Error` resolves through classBySymbol instead. A const binding + * of one (`const E = TypeError`) names the same class + * (builtinClassAliasOf). */ export function builtinErrorInfoOf( lowerer: Lowerer, symbol: ts.Symbol | null | undefined, ): ClassInfo | null { - if (!symbol || !lowerer.isStdlibSymbol(symbol)) return null; + if (!symbol) return null; + const alias = builtinClassAliasOf(lowerer, symbol); + if (alias !== null) return alias.kind === "error" ? alias.info : null; + if (!lowerer.isStdlibSymbol(symbol)) return null; for (const [irName, rec] of RUNTIME_ERROR_CLASSES) { if (rec.lib === symbol.name) return lowerer.classes.get(irName) ?? null; } @@ -3507,54 +3520,55 @@ export function collectClassShapeInner( const derived = decl.heritageClauses?.some((clause) => clause.token === ts.SyntaxKind.ExtendsKeyword) ?? false; - // A generic family is not emitted; its instances and each mixin - // application analyze their own construction with their actual base. - const escape: { escapes: boolean; assigned: Set | null } = familyMode - ? { escapes: derived, assigned: null } - : constructionEscape({ - ctor, - fields: fieldOrder.flatMap((f) => - paramProps.some((pp) => pp.name === f.name) - ? [] - : [{ name: f.name, initializer: f.initializer }], - ), - paramProps: paramProps.map((pp) => pp.name), - derived, - baseEscapes: - base === null || - callableBase !== undefined || - factoryBaseExpression !== undefined || - base.constructionEscapes === true, - isField: (name) => - fields.has(name) && - !methods.has(`set:${name}`) && - !lowerer.findMethodOn(base, `set:${name}`), - privateMethodBody: (call) => { - const callee = call.expression; - if (!ts.isPropertyAccessExpression(callee)) return null; - const name = callee.name.text; - const declaration = decl.members.find( - (member): member is ts.MethodDeclaration => - ts.isMethodDeclaration(member) && - member.body !== undefined && - (ts.isIdentifier(member.name) || ts.isPrivateIdentifier(member.name)) && - member.name.text === name, - ); - if (!declaration?.body || declaration.asteriskToken) return null; - const modifiers = ts.getModifiers(declaration) ?? []; - if ( - modifiers.some( - (m) => - m.kind === ts.SyntaxKind.StaticKeyword || m.kind === ts.SyntaxKind.AsyncKeyword, + // Generic instances and mixins share layouts across declarations, so + // they only report whether construction exposes the instance. + const escape: { escapes: boolean; assigned: Set | null } = + familyMode || mixin + ? { escapes: derived, assigned: null } + : constructionEscape({ + ctor, + fields: fieldOrder.flatMap((f) => + paramProps.some((pp) => pp.name === f.name) + ? [] + : [{ name: f.name, initializer: f.initializer }], + ), + paramProps: paramProps.map((pp) => pp.name), + derived, + baseEscapes: + base === null || + callableBase !== undefined || + factoryBaseExpression !== undefined || + base.constructionEscapes === true, + isField: (name) => + fields.has(name) && + !methods.has(`set:${name}`) && + !lowerer.findMethodOn(base, `set:${name}`), + privateMethodBody: (call) => { + const callee = call.expression; + if (!ts.isPropertyAccessExpression(callee)) return null; + const name = callee.name.text; + const declaration = decl.members.find( + (member): member is ts.MethodDeclaration => + ts.isMethodDeclaration(member) && + member.body !== undefined && + (ts.isIdentifier(member.name) || ts.isPrivateIdentifier(member.name)) && + member.name.text === name, + ); + if (!declaration?.body || declaration.asteriskToken) return null; + const modifiers = ts.getModifiers(declaration) ?? []; + if ( + modifiers.some( + (m) => + m.kind === ts.SyntaxKind.StaticKeyword || m.kind === ts.SyntaxKind.AsyncKeyword, + ) ) - ) - return null; - const isPrivate = - ts.isPrivateIdentifier(declaration.name) || - modifiers.some((m) => m.kind === ts.SyntaxKind.PrivateKeyword); - return isPrivate ? declaration.body : null; - }, - }); + return null; + const isPrivate = + ts.isPrivateIdentifier(declaration.name) || + modifiers.some((m) => m.kind === ts.SyntaxKind.PrivateKeyword); + return isPrivate ? declaration.body : null; + }, + }); // The deferred definite-assignment check: a field on the unguarded // list passes only with an unconditional `this.x = ...` at the // constructor's TOP LEVEL — the same standard the JS-class path @@ -3610,30 +3624,18 @@ export function collectClassShapeInner( } } - // Fields assigned after construction can expose the instance (a method - // call, a base constructor that calls an override) can be read while - // Node still holds undefined. Their slots admit undefined, allocation - // writes it, and reads yield it, as in Node. - const earlyReadFields = new Set(base?.earlyReadFields ?? []); - if (escape.assigned && escape.escapes) { + if (escape.assigned && !inst) { for (const f of fieldOrder) { if (f.redeclared || escape.assigned.has(f.name) || deferredInitFields.has(f.name)) continue; + if (paramProps.some((pp) => pp.name === f.name)) continue; const declared = fields.get(f.name); - if (declared === undefined || !earlyReadable(lowerer, declared)) continue; - // A declared `T | undefined` field already starts undefined. - if (declared.kind === "union" && lowerer.armTag(declared.unionId, UNDEFINED_T) >= 0) + if (declared?.kind !== "object" || lowerer.classes.get(declared.className)?.def.runtime) continue; - const armed = lowerer.runtimeOptionalType(declared); - if (armed.kind !== "union") continue; + const armed = lowerer.withUndefinedArm(declared); + if (armed === null || armed.kind !== "union") continue; fields.set(f.name, armed); f.type = armed; - // Primitive and union slots yield the undefined value itself. - // Reference slots keep the deferred-init reads: consumers that - // observe undefined (printing, equality, typeof, nullish defaults) - // see it, and member access on it throws a TypeError, as in Node. - if (declared.kind === "union" || PRIMITIVE_EARLY_READS.has(declared.kind)) - earlyReadFields.add(f.name); - else deferredInitFields.add(f.name); + deferredInitFields.add(f.name); } } @@ -3738,43 +3740,6 @@ export function collectClassShapeInner( (name) => !hasAccessor(name) && !methods.has(name) && !lowerer.findMethodOn(base, name), ) : null; - // Constructor statements that run after the instance may have been - // exposed (a method call, `this` passed along, a base constructor - // that exposes it): a field they create can be read before its - // assignment, while Node holds undefined. - const exposedBefore = new Set(); - { - let exposed = false; - for (const stmt of ctor?.body?.statements ?? []) { - const expression = ts.isExpressionStatement(stmt) ? stmt.expression : null; - if ( - expression && - ts.isCallExpression(expression) && - expression.expression.kind === ts.SyntaxKind.SuperKeyword - ) { - if (base?.constructionEscapes === true || expression.arguments.some(mentionsReceiver)) - exposed = true; - continue; - } - const assigned = - expression && - ts.isBinaryExpression(expression) && - expression.operatorToken.kind === ts.SyntaxKind.EqualsToken && - ts.isPropertyAccessExpression(expression.left) && - expression.left.expression.kind === ts.SyntaxKind.ThisKeyword - ? expression - : null; - if (exposed || (assigned && mentionsReceiver(assigned.right))) exposedBefore.add(stmt); - const plainAssignment = - assigned !== null && - ts.isPropertyAccessExpression(assigned.left) && - !mentionsReceiver(assigned.right) && - !hasAccessor(assigned.left.name.text) && - !methods.has(assigned.left.name.text) && - !lowerer.findMethodOn(base, assigned.left.name.text); - if (!plainAssignment && mentionsReceiver(stmt)) exposed = true; - } - } if (ctor?.body) { for (const stmt of ctor.body.statements) { const lhs = @@ -3845,15 +3810,8 @@ export function collectClassShapeInner( type = inheritedJsFieldType(lowerer, decl, name, type); // super() can invoke a subclass setter or method before its // constructor assignments run. These slots start undefined. - const sealed = sealedPrefix?.created.has(name) === true; - if (base && lowerer.dynConvertible(type) && !sealed) type = DYN; - if (exposedBefore.has(stmt) && !sealed && earlyReadable(lowerer, type)) { - const armed = lowerer.runtimeOptionalType(type); - if (armed.kind === "union") { - type = armed; - earlyReadFields.add(name); - } - } + if (base && lowerer.dynConvertible(type) && !sealedPrefix?.created.has(name)) + type = DYN; fields.set(name, type); fieldOrder.push({ name, type, initializer: undefined }); continue; @@ -4114,7 +4072,6 @@ export function collectClassShapeInner( ? { classDecorators: { nodes: classDecoratorNodes } } : {}), ...(deferredInitFields.size > 0 ? { deferredInitFields } : {}), - ...(earlyReadFields.size > 0 ? { earlyReadFields } : {}), ...(escape.escapes ? { constructionEscapes: true as const } : {}), }; // GENERIC members get their declaring-class backlink now that the @@ -4419,6 +4376,7 @@ export function lowerStaticFieldInits(lowerer: Lowerer, info: ClassInfo): IrStmt // instantiation's bindings/context (a no-op for everything else — // generic FAMILIES own their statics and carry no genericInstance). const diagsBefore = lowerer.diags.length; + const sinkMark = lowerer.diagSinkMark(); try { return withInstanceBindings(lowerer, info, () => lowerStaticFieldInitsInner(lowerer, info)); } catch (error) { @@ -4426,8 +4384,11 @@ export function lowerStaticFieldInits(lowerer: Lowerer, info: ClassInfo): IrStmt // Keep their refusal attached to the class declaration instead of // letting a poisoned initializer escape the module lowering pass. if (!(error instanceof PoisonError)) throw error; - if (info.decl && isJsSourceFile(info.decl.getSourceFile()) && lowerer.diagSink === null) { - const fence = lowerer.deferToRuntimeFence(diagsBefore, info.decl, { kind: "statement" }); + if (info.decl && isJsSourceFile(info.decl.getSourceFile())) { + const fence = + lowerer.diagSink === null + ? lowerer.deferToRuntimeFence(diagsBefore, info.decl, { kind: "statement" }) + : lowerer.capturedRuntimeFence(sinkMark, info.decl); if (fence) return [fence]; } return []; @@ -4564,6 +4525,7 @@ function lowerStaticFieldInitsInner(lowerer: Lowerer, info: ClassInfo): IrStmt[] lowerer.stats.statementsTotal++; lowerer.bumpFileStat(locOf(f.initializer).file, "total"); const diagsBefore = lowerer.diags.length; + const sinkMark = lowerer.diagSinkMark(); try { const previousThis = lowerer.ctx.thisLocal; const receiver = @@ -4599,10 +4561,11 @@ function lowerStaticFieldInitsInner(lowerer: Lowerer, info: ClassInfo): IrStmt[] if (!(e instanceof PoisonError)) throw e; lowerer.stats.statementsFailed++; lowerer.bumpFileStat(locOf(f.initializer).file, "failed"); - if (isJsSourceFile(f.initializer.getSourceFile()) && lowerer.diagSink === null) { - const fence = lowerer.deferToRuntimeFence(diagsBefore, f.initializer, { - kind: "statement", - }); + if (isJsSourceFile(f.initializer.getSourceFile())) { + const fence = + lowerer.diagSink === null + ? lowerer.deferToRuntimeFence(diagsBefore, f.initializer, { kind: "statement" }) + : lowerer.capturedRuntimeFence(sinkMark, f.initializer); if (fence) out.push(fence); } } @@ -6289,33 +6252,6 @@ function overrideReturnAbi( return declared; } -/** Field kinds whose early reads yield the undefined value (earlyReadFields). */ -const PRIMITIVE_EARLY_READS: ReadonlySet = new Set([ - "f64", - "bool", - "string", - "bigint", -]); - -/** Field types whose slot can admit undefined for reads before the field's - * assignment. Date values are scalars, so their unassigned slot is memory - * safe; checked-dynamic slots already hold undefined. */ -function earlyReadable(lowerer: Lowerer, type: IrType): boolean { - if (isUnitType(type)) return false; - switch (type.kind) { - case "dyn": - case "void": - case "date": - case "generator": - case "caught": - return false; - case "object": - return lowerer.classes.get(type.className)?.def.runtime !== true; - default: - return true; - } -} - /** The nearest declaration of `name` at or above `info` — the method a * receiver of that static class runs when nothing below overrides it. */ export function findMethodOn( @@ -6502,7 +6438,7 @@ export function collectInheritedJsFieldWrites( * hierarchy identifies the original declarations through aliases as well. */ export function collectVirtualJsMethods(lowerer: Lowerer, files: readonly ts.SourceFile[]): void { const families: [ts.MethodDeclaration, ts.MethodDeclaration][] = []; - collectClassMethodMutations(lowerer, files); + lowerer.speculativeTypeQueries(() => collectClassMethodMutations(lowerer, files)); const visit = (node: ts.Node): void => { if (ts.isClassDeclaration(node) || ts.isClassExpression(node)) { const accessors = node.members.filter( @@ -7703,6 +7639,7 @@ export function fieldInitStmts(lowerer: Lowerer, info: ClassInfo, thisLocal: IrL lowerer.stats.statementsTotal++; lowerer.bumpFileStat(locOf(f.initializer).file, "total"); const diagsBefore = lowerer.diags.length; + const sinkMark = lowerer.diagSinkMark(); try { const value = lowerer.lowerExprExpecting(f.initializer, f.type); out.push({ @@ -7717,10 +7654,11 @@ export function fieldInitStmts(lowerer: Lowerer, info: ClassInfo, thisLocal: IrL if (!(e instanceof PoisonError)) throw e; lowerer.stats.statementsFailed++; lowerer.bumpFileStat(locOf(f.initializer).file, "failed"); - if (isJsSourceFile(f.initializer.getSourceFile()) && lowerer.diagSink === null) { - const fence = lowerer.deferToRuntimeFence(diagsBefore, f.initializer, { - kind: "statement", - }); + if (isJsSourceFile(f.initializer.getSourceFile())) { + const fence = + lowerer.diagSink === null + ? lowerer.deferToRuntimeFence(diagsBefore, f.initializer, { kind: "statement" }) + : lowerer.capturedRuntimeFence(sinkMark, f.initializer); if (fence) out.push(fence); } } @@ -7753,11 +7691,7 @@ function paramPropInitStmts(lowerer: Lowerer, info: ClassInfo, thisLocal: IrLoca obj: { kind: "varRef", localId: thisLocal.id, type: thisType, loc }, className: info.def.name, field: pp.name, - // An early-readable field's slot admits undefined (earlyReadFields). - value: lowerer.coerceToExpected( - { kind: "varRef", localId: local!.id, type: local!.type, loc }, - info.fields.get(pp.name) ?? local!.type, - ), + value: { kind: "varRef", localId: local!.id, type: local!.type, loc }, loc, }); } @@ -8526,6 +8460,12 @@ function assignedThisFieldType(lowerer: Lowerer, expr: ts.NewExpression): IrType return lowerer.currentClass?.fields.get(parent.left.name.text) ?? null; } +/** Map, Set and the error classes construct in lowerNew's identifier + * branch, which a module member alias of one (`new mod.M()`) also takes. */ +function isCollectionOrErrorClass(ref: BuiltinClassRef | null): boolean { + return ref?.kind === "collection" || ref?.kind === "error"; +} + export function lowerNew(lowerer: Lowerer, expr: ts.NewExpression): IrExpr { const loc = locOf(expr); const worker = lowerWorkerNew(lowerer, expr); @@ -8551,6 +8491,8 @@ export function lowerNew(lowerer: Lowerer, expr: ts.NewExpression): IrExpr { undefined, lowerer.checker.getSymbolAtLocation(expr.expression), ); + const valueReturning = lowerValueReturningConstructorNew(lowerer, expr); + if (valueReturning) return valueReturning; const instanceConstructor = lowerInstanceConstructorNew(lowerer, expr); if (instanceConstructor) return instanceConstructor; const objectFactory = lowerObjectFactoryNew(lowerer, expr); @@ -8818,7 +8760,10 @@ export function lowerNew(lowerer: Lowerer, expr: ts.NewExpression): IrExpr { ts.isIdentifier(expr.expression) || (ts.isPropertyAccessExpression(expr.expression) && ts.isIdentifier(expr.expression.name) && - lowerer.isStdlibGlobal(expr.expression.expression, "globalThis")) + (lowerer.isStdlibGlobal(expr.expression.expression, "globalThis") || + // `new mod.M()` over an exported `const M = Map` or `const E = + // TypeError` + isCollectionOrErrorClass(builtinClassRefOf(lowerer, expr.expression)))) ) { // `import C = N.C; new C()` — the alias's own source-order guards // (a no-op for every non-import= binding). @@ -9183,7 +9128,8 @@ export function lowerNew(lowerer: Lowerer, expr: ts.NewExpression): IrExpr { if (symbol?.name === "Array" && lowerer.isStdlibSymbol(symbol)) { return lowerArrayConstructor(lowerer, expr, expr.arguments ?? []); } - if (symbol?.name === "Map" && lowerer.isStdlibSymbol(symbol)) { + // A const alias (`const M = Map`) constructs the same Map. + if (ts.isIdentifier(identifier) && lowerer.isStdlibGlobal(identifier, "Map")) { const seedArg = (expr.arguments?.length ?? 0) === 1 ? expr.arguments![0]! : null; const isPairLit = (el: ts.Expression): el is ts.ArrayLiteralExpression => ts.isArrayLiteralExpression(el) && @@ -9315,7 +9261,7 @@ export function lowerNew(lowerer: Lowerer, expr: ts.NewExpression): IrExpr { } // `new Set()`: construct + bulk add from supported values, // including live collection cursors. Duplicates collapse in insertion order. - if (symbol?.name === "Set" && lowerer.isStdlibSymbol(symbol)) { + if (ts.isIdentifier(identifier) && lowerer.isStdlibGlobal(identifier, "Set")) { const tsType = lowerer.typeOf(expr); let mapped = lowerer.mapTypeOf(tsType); const fieldType = assignedThisFieldType(lowerer, expr); diff --git a/packages/compiler/src/frontend/lowering/lower-exprs.ts b/packages/compiler/src/frontend/lowering/lower-exprs.ts index 4ef139293..149c25248 100644 --- a/packages/compiler/src/frontend/lowering/lower-exprs.ts +++ b/packages/compiler/src/frontend/lowering/lower-exprs.ts @@ -35,10 +35,12 @@ import { isZlibStreamMember, lowerZlibInstanceOf, lowerZlibStreamBytesWritten, - zlibClassAliasOf, - zlibClassModeOfExpr, - zlibClassName, } from "./builtins/compression.js"; +import { + builtinClassAliasOwnsNoSlot, + builtinClassRefOf, + sameBuiltinClass, +} from "./builtins/class-values.js"; import { constituentTypes } from "../ts7/checker.js"; import { dirname } from "node:path"; import * as posix from "node:path/posix"; @@ -203,6 +205,7 @@ import { tryLowerExpression } from "./expressions/try-lower-expression.js"; import { fenceNodeModuleMutation, isNodeModuleValue, + lowerCjsTopLevelThis, lowerNodeModuleIdentifier, lowerNodeModuleProperty, lowerRequireCacheElement, @@ -227,6 +230,7 @@ import { } from "./class-prototypes.js"; import { isClassCallback } from "./class-callbacks.js"; import { + builtinConstantsObject, builtinPrototypeMethod, lowerArrayIsArrayValue, lowerCheckedPredicateValue, @@ -615,8 +619,9 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { if (isNodeModuleValue(lowerer, expr.expression)) { return { kind: "strLit", value: "object", type: STRING, loc }; } - // `typeof zlib.Gzip`, `typeof C` over a zlib class alias. - if (zlibClassModeOfExpr(lowerer, expr.expression) !== null) { + // `typeof EventEmitter`, `typeof zlib.Gzip`, `typeof C` over a + // builtin class alias. + if (builtinClassRefOf(lowerer, expr.expression) !== null) { return { kind: "strLit", value: "function", type: STRING, loc }; } if (ts.isPropertyAccessExpression(expr.expression)) { @@ -690,6 +695,30 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { if (!sym) { return { kind: "strLit", value: "undefined", type: STRING, loc }; } + // `const { inspect } = require("util")` / `import { inspect } from + // "node:util"`: a binding of a builtin module's exported function, + // class or object. JavaScript sources represent such a value without + // its own lowering as an identity token (a string), so the operand's + // representation cannot answer. Node's answer follows the export's + // kind, and the binding read has no effects. + const imported = isJsSourceFile(tested.getSourceFile()) + ? lowerer.builtinImportOf(tested) + : null; + if ( + imported && + builtinModuleConstOf(lowerer, imported.module, imported.member) === undefined + ) { + const t = lowerer.typeOf(tested); + if ( + lowerer.checker.getCallSignatures(t).length > 0 || + lowerer.checker.getConstructSignatures(t).length > 0 + ) { + return { kind: "strLit", value: "function", type: STRING, loc }; + } + if ((t.flags & ts.TypeFlags.Object) !== 0) { + return { kind: "strLit", value: "object", type: STRING, loc }; + } + } if (lowerer.isStdlibSymbol(sym)) { const t = lowerer.typeOf(expr.expression); if ( @@ -877,13 +906,13 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { `a builtin function alias as an escaping value (call '${expr.text}' directly)`, ); } - // `const C = zlib.Gzip` owns no runtime slot: construction, + // `const E = EventEmitter` owns no runtime slot: construction, // instanceof, typeof and .name lower at their use sites. - if (sym && zlibClassAliasOf(lowerer, sym) !== null) { + if (sym && builtinClassAliasOwnsNoSlot(lowerer, sym)) { lowerer.unsupported( "SC1090", expr, - `a zlib class alias as an escaping value (use '${expr.text}' with new, instanceof, typeof or .name)`, + `a builtin class alias as an escaping value (use '${expr.text}' with new, instanceof, typeof or .name)`, ); } } @@ -1228,6 +1257,17 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { if (bi.module === "os" && bi.member === "constants") { return { kind: "libCall", fn: "os.constants", args: [], type: DYN, loc }; } + // `import { O_RDONLY } from "constants"`: the member of the frozen + // module object. + if (bi.module === "constants") { + return { + kind: "dynKeyGet", + value: builtinConstantsObject(lowerer, loc), + key: { kind: "strLit", value: bi.member, type: STRING, loc }, + type: DYN, + loc, + }; + } // tls.rootCertificates: a runtime-valued module constant (the // cached bundled-CA array) — the one member read that lowers // to a libCall instead of a baked literal. @@ -1270,6 +1310,7 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { // their access sites only. The CommonJS namespace binding // (`const lib = require("./lib.js")`) gets the same fence. const builtinNamespace = lowerer.builtinNamespaceModuleOf(expr); + if (builtinNamespace === "constants") return builtinConstantsObject(lowerer, loc); if (builtinNamespace !== null) { const type: IrType = { kind: "moduleNs", moduleId: `builtin:${builtinNamespace}` }; return { kind: "moduleNsRef", moduleId: type.moduleId, type, loc }; @@ -1559,6 +1600,7 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { return { kind: "moduleNsRef", moduleId: moduleNs.moduleId, type: moduleNs, loc }; } const builtinNs = lowerer.builtinNamespaceModuleOf(expr); + if (builtinNs === "constants") return builtinConstantsObject(lowerer, loc); if (builtinNs !== null) { const type: IrType = { kind: "moduleNs", moduleId: `builtin:${builtinNs}` }; return { kind: "moduleNsRef", moduleId: type.moduleId, type, loc }; @@ -1661,6 +1703,9 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { // constant. TypeScript keeps the fence (noImplicitThis makes it a // compile-time story there). if (isJsSourceFile(expr.getSourceFile())) { + // Module scope of a CommonJS file: `this` is module.exports. + const exportsThis = lowerCjsTopLevelThis(lowerer, expr); + if (exportsThis) return exportsThis; return { kind: "libCall", fn: "dyn.this", args: [], type: DYN, loc }; } lowerer.unsupported("SC1080", expr); @@ -1821,10 +1866,11 @@ function lowerExprInner(lowerer: Lowerer, expr: ts.Expression): IrExpr { const table = lowerInternTableProperty(lowerer, expr); if (table) return table; } - // `zlib.Gzip.name`, `C.name` over a zlib class alias. + // `EventEmitter.name`, `zlib.Gzip.name`, `C.name` over a builtin + // class alias. if (expr.name.text === "name" && !expr.questionDotToken) { - const mode = zlibClassModeOfExpr(lowerer, expr.expression); - if (mode !== null) return { kind: "strLit", value: zlibClassName(mode), type: STRING, loc }; + const ref = builtinClassRefOf(lowerer, expr.expression); + if (ref !== null) return { kind: "strLit", value: ref.name, type: STRING, loc }; } // Expando statics may still be absent even when JS inference sees // the eventual assignment's scalar type. Preserve the checked value. @@ -3161,51 +3207,6 @@ export function lowerTernary( * the same union (unnarrowed use), to a SUB-union (partial narrowing — * unrepresentable without a re-tag), or to nothing (`never` in an * exhaustive default) leaves the expression union-typed. */ -/** Uses that read an early-read field's value as Node does when it is - * undefined: printing, templates, operators, tests, member access, calls of - * program functions (whose parameters the optional-value analysis widens), - * and stores into bindings, fields and returns. Other uses, such as library - * arguments, extract the declared type with a check. */ -function observesEarlyRead(node: ts.Node): boolean { - let child = node; - let parent = node.parent; - while (parent && ts.isParenthesizedExpression(parent)) { - child = parent; - parent = parent.parent; - } - if (!parent) return false; - if ( - ts.isTemplateSpan(parent) || - ts.isBinaryExpression(parent) || - ts.isPrefixUnaryExpression(parent) || - ts.isPostfixUnaryExpression(parent) || - ts.isTypeOfExpression(parent) || - ts.isConditionalExpression(parent) || - ts.isIfStatement(parent) || - ts.isWhileStatement(parent) || - ts.isReturnStatement(parent) || - ts.isVariableDeclaration(parent) || - ts.isPropertyAssignment(parent) || - ts.isArrayLiteralExpression(parent) || - (ts.isPropertyAccessExpression(parent) && parent.expression === child) || - (ts.isElementAccessExpression(parent) && parent.expression === child) - ) - return true; - if (ts.isCallExpression(parent) && parent.arguments.includes(child as ts.Expression)) { - const callee = parent.expression; - if ( - ts.isPropertyAccessExpression(callee) && - ts.isIdentifier(callee.expression) && - callee.expression.text === "console" - ) - return true; - return ( - !ts.isPropertyAccessExpression(callee) || callee.expression.kind === ts.SyntaxKind.ThisKeyword - ); - } - return false; -} - export function maybeNarrow(lowerer: Lowerer, expr: IrExpr, node: ts.Node): IrExpr { if ( (expr.type.kind === "union" || expr.type.kind === "dyn") && @@ -3416,25 +3417,6 @@ export function maybeNarrow(lowerer: Lowerer, expr: IrExpr, node: ts.Node): IrEx if (expr.type.kind !== "union") return expr; if (ts.isPropertyAccessExpression(node) && isOptionalProcessStreamProperty(lowerer, node)) return expr; - // A field read before its assignment holds undefined although its - // declared type omits it; the declared type is no narrowing proof. A flow - // narrowing past the declared type (`if (this.x !== null)`) extracts the - // narrowed arm with a check, since the slot may still hold undefined. - if ( - expr.kind === "fieldGet" && - lowerer.classes.get(expr.className)?.earlyReadFields?.has(expr.field) === true - ) { - const symbol = ts.isPropertyAccessExpression(node) - ? lowerer.checker.getSymbolAtLocation(node.name) - : undefined; - const declared = symbol ? lowerer.mapTypeOf(lowerer.checker.getTypeOfSymbol(symbol)) : null; - const narrowedArm = lowerer.mapTypeOf(lowerer.typeOf(node)); - if (narrowedArm === null || narrowedArm.kind === "union") return expr; - if (declared !== null && typeEquals(declared, narrowedArm) && observesEarlyRead(node)) - return expr; - const helper = lowerer.narrowedArmHelper(expr.type.unionId, narrowedArm, expr.loc); - return helper ? lowerer.checkedNarrowCall(helper, expr, narrowedArm, expr.loc) : expr; - } // Represented fields and index values can include undefined even when // checker inference or lib.d.ts declares strings (RegExp captures). @@ -5082,6 +5064,34 @@ export function lowerIntrinsicProperty( return { kind: "libCall", fn: "dyn.arrayPrototype", args: [], type: DYN, loc: locOf(expr) }; if (lowerer.stdlibGlobalMember(expr, "Object") === "prototype") return { kind: "libCall", fn: "dyn.objectPrototype", args: [], type: DYN, loc: locOf(expr) }; + // Function.prototype as a VALUE (a prototype-chain link: + // `Object.setPrototypeOf(F.prototype, Function.prototype)`): one + // anonymous function answering undefined, length 0 — a zero-capture + // closure of a compiler helper, so every read is the same function + // (closure identity) and shares its property table. Reads through it + // find the function members the checked runtime models for every + // function (apply, constructor, name, length). Member forms keep their + // own lowerings. + if (lowerer.stdlibGlobalMember(expr, "Function") === "prototype") { + const loc = locOf(expr); + const name = "%dyn.functionPrototype"; + if (!lowerer.liftedFns.some((fn) => fn.name === name)) + lowerer.liftedFns.push({ + name, + params: [], + locals: [], + returnType: DYN, + body: [{ kind: "return", value: dynUndefinedExpr(loc), loc }], + loc, + }); + return { + kind: "dynFrom", + value: { kind: "closure", fnName: name, captures: [], type: funcOf([], DYN), loc }, + fnName: "", + type: DYN, + loc, + }; + } if (lowerer.stdlibGlobalMember(expr, "Array") === "isArray") return lowerArrayIsArrayValue(lowerer, locOf(expr)); if (lowerer.stdlibGlobalMember(expr, "Array") === "from") @@ -10274,17 +10284,18 @@ export function lowerBinary(lowerer: Lowerer, expr: ts.BinaryExpression): IrExpr lowerUnionTypeofTest(lowerer, expr, loc); if (test) return test; } - // `C === zlib.Gzip` between two zlib class references: the classes - // are compile-time values, so the answer folds. + // `C === EventEmitter` between two builtin class references: the + // classes are compile-time values, so the answer folds. if ( op === ts.SyntaxKind.EqualsEqualsEqualsToken || op === ts.SyntaxKind.ExclamationEqualsEqualsToken ) { - const leftClass = zlibClassModeOfExpr(lowerer, expr.left); - const rightClass = leftClass === null ? null : zlibClassModeOfExpr(lowerer, expr.right); + const leftClass = builtinClassRefOf(lowerer, expr.left); + const rightClass = leftClass === null ? null : builtinClassRefOf(lowerer, expr.right); if (leftClass !== null && rightClass !== null) { const negated = op === ts.SyntaxKind.ExclamationEqualsEqualsToken; - return { kind: "boolLit", value: (leftClass === rightClass) !== negated, type: BOOL, loc }; + const same = sameBuiltinClass(leftClass, rightClass); + return { kind: "boolLit", value: same !== negated, type: BOOL, loc }; } } @@ -11538,6 +11549,34 @@ function lowerLogicalPair(lowerer: Lowerer, expr: ts.BinaryExpression, left: IrE // same lift the union-operand path above takes, arriving here // with two plain arm values instead. const target = lowerer.mapTypeOf(lowerer.typeOf(expr)); + // An always-truthy left operand (a record, object, array or function + // value: every JS object is truthy): `o && x` evaluates `o` for its + // effects and answers `x`; `o || x` answers `o` and never evaluates + // `x`. supports-color's `stream && stream.isTTY` is the shape. Ahead + // of the union lift below exactly where that lift would store the + // object in a result union with no arm for it; a union holding the + // operand keeps the lift. + if ( + REF_TRUTHY_KINDS.has(kind) && + (target?.kind !== "union" || + (op === ts.SyntaxKind.AmpersandAmpersandToken && + lowerer.armTag(target.unionId, left.type) < 0)) + ) { + const and = op === ts.SyntaxKind.AmpersandAmpersandToken; + const answer = and ? right : left; + const answerT = target ?? answer.type; + if (answerT.kind !== "void") { + const value = lowerer.coerceInto(and ? expr.right : expr.left, answer, answerT); + if (!and) return value; + return { + kind: "seqExpr", + stmts: [{ kind: "exprStmt", expr: left, loc }], + result: value, + type: answerT, + loc, + }; + } + } if (target?.kind === "union") { lowerer.requireTruthyUnion(target.unionId, expr); return { @@ -12332,11 +12371,11 @@ export function lowerInstanceOf(lowerer: Lowerer, expr: ts.BinaryExpression, loc // h2 compat 'connect' narrowing (lower-server.ts): a union tag test. const sockTest = lowerSocketInstanceOf(lowerer, expr, loc); if (sockTest !== null) return sockTest; - const zlibMode = zlibClassModeOfExpr(lowerer, expr.right); - if (zlibMode !== null) return lowerZlibInstanceOf(lowerer, expr, zlibMode, loc); + const rightClass = builtinClassRefOf(lowerer, expr.right); + if (rightClass?.kind === "zlib") return lowerZlibInstanceOf(lowerer, expr, rightClass.mode, loc); - if (lowerer.isStdlibGlobal(expr.right, "Set") || lowerer.isStdlibGlobal(expr.right, "Map")) { - const collection = lowerer.isStdlibGlobal(expr.right, "Map") ? "map" : "set"; + if (rightClass?.kind === "collection") { + const collection = rightClass.which === "Map" ? "map" : "set"; const value = lowerer.lowerExpr(expr.left); if (value.type.kind === "union") { const union = lowerer.unions.get(value.type.unionId); @@ -15465,20 +15504,6 @@ export function fieldGetExpr( const helper = lowerer.deferredReadHelper(target.fieldType.unionId, inner, loc); if (helper) return { kind: "call", callee: helper, args: [read], type: inner, loc }; } - // An early-read field (ClassInfo.earlyReadFields) yields its undefined - // where the use reads it as Node does; elsewhere the declared type is - // extracted with a check. - if ( - target.fieldType.kind === "union" && - lowerer.classes.get(target.className)?.earlyReadFields?.has(target.field) === true && - !observesEarlyRead(blame) - ) { - const declared = lowerer.mapTypeOf(lowerer.typeOf(blame)); - if (declared && declared.kind !== "union") { - const helper = lowerer.narrowedArmHelper(target.fieldType.unionId, declared, loc); - if (helper) return lowerer.checkedNarrowCall(helper, read, declared, loc); - } - } return read; } return { @@ -16019,22 +16044,6 @@ function lowerFieldCompoundValue( if (op === "+" && read.type.kind === "string") { appended = lowerer.ensureString(rhs, rhsNode ?? access); value = { kind: "strConcat", left: read, right: appended, type: STRING, loc }; - } else if ( - op === "+" && - read.type.kind === "union" && - lowerer.stripUndefinedArm(read.type).kind === "string" && - target.container === "class" && - lowerer.classes.get(target.className)?.earlyReadFields?.has(target.field) === true - ) { - // A string field read before its assignment holds undefined, which - // concatenates as "undefined". - value = { - kind: "strConcat", - left: lowerer.ensureString(read, access), - right: lowerer.ensureString(rhs, rhsNode ?? access), - type: STRING, - loc, - }; } else if (numericRead.type.kind === "f64" && numericRhs.type.kind === "f64") { value = { kind: "bin", op, left: numericRead, right: numericRhs, type: F64, loc }; } else if (bigintValue) { diff --git a/packages/compiler/src/frontend/lowering/lower-modules.ts b/packages/compiler/src/frontend/lowering/lower-modules.ts index c5802b07e..3c035cd45 100644 --- a/packages/compiler/src/frontend/lowering/lower-modules.ts +++ b/packages/compiler/src/frontend/lowering/lower-modules.ts @@ -80,7 +80,7 @@ import { createRequireProgramModuleDecl, } from "./builtins/module-bindings.js"; import { isPromisifyCall, registerBuiltinCallableAlias } from "./builtins/callable-aliases.js"; -import { registerZlibClassAlias } from "./builtins/compression.js"; +import { registerBuiltinClassAlias } from "./builtins/class-values.js"; import { stripTypeCasts } from "./builtins/arguments.js"; import { bindingContextualGenericFnNodeOf, @@ -1701,7 +1701,7 @@ export function collectGlobals( // resolution and value reads reuse the source's interned closure. if (isConst && registerOverloadedCallableAlias(lowerer, decl)) continue; if (isConst && registerBuiltinCallableAlias(lowerer, decl.name, decl.initializer)) continue; - if (registerZlibClassAlias(lowerer, decl.name, decl.initializer)) continue; + if (registerBuiltinClassAlias(lowerer, decl.name, decl.initializer)) continue; // `const f = (x) => ...` at file scope of an npm-static JS file // whose params are implicit-any: the implicit-monomorphization // twin of the generic binding above — no global exists; calls and @@ -2664,6 +2664,18 @@ export function lowerFileInit( loc: loc0, }); } + if (sf === lowerer.entry) header.push(...lowerer.punycodeLinkWarning()); + // Top-level `this` of a CommonJS module: its object exists before the + // module's first statement (lowerCjsTopLevelThis). + const exportsThis = lowerer.cjsExportsThis.get(sf); + if (exportsThis) { + header.push({ + kind: "assign", + localId: exportsThis.id, + value: { kind: "dynObjLit", fields: [], type: DYN, loc: loc0 }, + loc: loc0, + }); + } for (const { stmt, dep } of orderedImportsOf(lowerer.program, sf)) { const npm = lowerer.npmInitActions.get(sf)?.get(stmt); if (npm !== undefined && npm.length > 0) { diff --git a/packages/compiler/src/frontend/lowering/lower-node-module.ts b/packages/compiler/src/frontend/lowering/lower-node-module.ts index ad3f86dee..c15749601 100644 --- a/packages/compiler/src/frontend/lowering/lower-node-module.ts +++ b/packages/compiler/src/frontend/lowering/lower-node-module.ts @@ -8,10 +8,11 @@ import * as win32 from "node:path/win32"; import * as ts from "../ts7/adapter.js"; import type { Lowerer } from "./lowerer.js"; import type { FileParts } from "./lower-modules.js"; -import { isNodeEsmFile, locOf } from "../program.js"; +import { isCjsJsFile, isNodeEsmFile, locOf } from "../program.js"; import { runtimePathForTarget } from "../runtime-resolve.js"; import { BOOL, + DYN, F64, type IrExpr, type IrStmt, @@ -224,6 +225,135 @@ function nodeModuleType(lowerer: Lowerer, node: ts.Expression): boolean { return rendered === "ScriptcModule" || rendered === "NodeModule" || rendered === "NodeJS.Module"; } +/** `module.exports` used only for its truthiness (the UMD test `typeof + * module !== "undefined" && module.exports`): the exports object starts as + * a fresh object, so it stays truthy unless the module replaces it with a + * value that can be falsy. True when every `module.exports = value` in the + * file assigns an object or function. */ +function moduleExportsAlwaysTruthy(lowerer: Lowerer, expr: ts.PropertyAccessExpression): boolean { + if (!truthinessOnly(expr)) return false; + const sf = expr.getSourceFile(); + if (isNodeEsmFile(sf, lowerer.program)) return false; + let truthy = true; + ts.walkPreorder(sf, (node) => { + if ( + !ts.isBinaryExpression(node) || + node.operatorToken.kind < ts.SyntaxKind.FirstAssignment || + node.operatorToken.kind > ts.SyntaxKind.LastAssignment + ) + return undefined; + let target = node.left; + while (ts.isParenthesizedExpression(target)) target = target.expression; + if ( + !ts.isPropertyAccessExpression(target) || + target.name.text !== "exports" || + !isCjsModuleGlobal(lowerer, target.expression) + ) + return undefined; + const type = lowerer.typeOf(node.right); + const objectLike = (t: ts.Type): boolean => + t.isUnionType() + ? ts.constituentTypes(t).every(objectLike) + : (t.flags & ts.TypeFlags.Object) !== 0 || (t.flags & ts.TypeFlags.NonPrimitive) !== 0; + if (node.operatorToken.kind !== ts.SyntaxKind.EqualsToken || !objectLike(type)) { + truthy = false; + return "stop"; + } + return undefined; + }); + return truthy; +} + +/** Whether only the truthiness of `node` can be observed: a condition, a + * `!` operand, or a `&&`/`||` operand of an expression that is itself used + * only for its truthiness. */ +function truthinessOnly(node: ts.Expression): boolean { + let child: ts.Node = node; + for (let parent = node.parent; ; child = parent, parent = parent.parent) { + if (ts.isParenthesizedExpression(parent)) continue; + if (ts.isPrefixUnaryExpression(parent)) + return parent.operator === ts.SyntaxKind.ExclamationToken; + if ( + ts.isBinaryExpression(parent) && + (parent.operatorToken.kind === ts.SyntaxKind.AmpersandAmpersandToken || + parent.operatorToken.kind === ts.SyntaxKind.BarBarToken) + ) + continue; + if (ts.isIfStatement(parent) || ts.isWhileStatement(parent) || ts.isDoStatement(parent)) + return parent.expression === child; + if (ts.isForStatement(parent)) return parent.condition === child; + if (ts.isConditionalExpression(parent)) return parent.condition === child; + return false; + } +} + +/** `this` at the top level of a CommonJS module (directly, or inside + * arrow functions there): Node calls the module wrapper with `this` bound + * to the module's original exports object. */ +export function isCjsTopLevelThis(lowerer: Lowerer, node: ts.Node): boolean { + if (node.kind !== ts.SyntaxKind.ThisKeyword) return false; + for (let p: ts.Node | undefined = node.parent; p && !ts.isSourceFile(p); p = p.parent) { + if (ts.isArrowFunction(p)) continue; + if ( + ts.isFunctionLike(p) || + ts.isClassDeclaration(p) || + ts.isClassExpression(p) || + ts.isClassStaticBlockDeclaration(p) + ) + return false; + } + return isCjsJsFile(node.getSourceFile(), lowerer.program); +} + +export function usesCjsTopLevelThis(lowerer: Lowerer, sf: ts.SourceFile): boolean { + if (!isCjsJsFile(sf, lowerer.program)) return false; + let found = false; + ts.walkPreorder(sf, (node) => { + if ( + node !== sf && + !ts.isArrowFunction(node) && + (ts.isFunctionLike(node) || + ts.isClassDeclaration(node) || + ts.isClassExpression(node) || + ts.isClassStaticBlockDeclaration(node)) + ) + return "skip"; + if (node.kind === ts.SyntaxKind.ThisKeyword) { + found = true; + return "stop"; + } + return undefined; + }); + return found; +} + +/** The value of top-level `this` in a CommonJS module: the module's own + * object standing for its original exports, created when the module starts + * (lowerFileInit). Exports themselves are the static export table, so a + * property access through `this` would read or write an object no requirer + * sees; it keeps a named refusal instead. Typeof, truthiness, identity and + * passing the value along (the UMD wrapper `(function (root) { ... })(this)`) + * behave as in Node. */ +export function lowerCjsTopLevelThis(lowerer: Lowerer, expr: ts.Expression): IrExpr | null { + if (!isCjsTopLevelThis(lowerer, expr)) return null; + const global = lowerer.cjsExportsThis.get(expr.getSourceFile()); + if (!global) return null; + let receiver: ts.Node = expr; + while (ts.isParenthesizedExpression(receiver.parent)) receiver = receiver.parent; + const access = receiver.parent; + if ( + (ts.isPropertyAccessExpression(access) || ts.isElementAccessExpression(access)) && + access.expression === receiver + ) { + lowerer.unsupported( + "SC1090", + access, + "property access through top-level 'this' in a CommonJS module (it is module.exports; spell exports.)", + ); + } + return varRef(global.id, DYN, locOf(expr)); +} + function optionalModule( lowerer: Lowerer, node: ts.Expression, @@ -309,6 +439,8 @@ export function lowerNodeModuleProperty( if (!direct && !nodeModuleType(lowerer, expr.expression)) return null; if (expr.questionDotToken) return null; const member = expr.name.text; + if (member === "exports" && direct && moduleExportsAlwaysTruthy(lowerer, expr)) + return boolLit(true, locOf(expr)); if (member === "exports" || member === "require") return null; let moduleRef = direct ? cjsModuleRef(lowerer, expr.expression) diff --git a/packages/compiler/src/frontend/lowering/lower-stmts.ts b/packages/compiler/src/frontend/lowering/lower-stmts.ts index 9fd770a23..b5e79ad72 100644 --- a/packages/compiler/src/frontend/lowering/lower-stmts.ts +++ b/packages/compiler/src/frontend/lowering/lower-stmts.ts @@ -10,6 +10,7 @@ import { InternalCompilerError } from "../../errors.js"; * flow (if/while/for/for-of/do, switch, try/catch, abrupt completions), * explicit resource management, and blocked-binding poisoning. */ import * as ts from "../ts7/adapter.js"; +import { processDataMemberValue } from "./builtins/process.js"; import { requireInitFlagStmt } from "./require-initialization.js"; import { isJsonRequireDecl } from "./json-require.js"; import { bindingSource } from "../binding-source.js"; @@ -147,7 +148,7 @@ import { } from "./builtins/module-bindings.js"; import { lowerNodeModuleCall } from "./builtins/module-resolution.js"; import { registerBuiltinCallableAlias } from "./builtins/callable-aliases.js"; -import { registerZlibClassAlias } from "./builtins/compression.js"; +import { builtinClassAliasOwnsNoSlot, registerBuiltinClassAlias } from "./builtins/class-values.js"; import { lowerEnumDeclaration } from "./lower-enums.js"; import { abstractPropertyDeclOf, @@ -675,6 +676,7 @@ export function lowerStmts(lowerer: Lowerer, stmts: readonly ts.Statement[]): Ir if (sf !== undefined) lowerer.bumpFileStat(sf.fileName, "total"); } const diagsBefore = lowerer.diags.length; + const sinkMark = lowerer.diagSinkMark(); try { let lowered: IrStmt | IrStmt[] | null; let appended = false; @@ -759,6 +761,11 @@ export function lowerStmts(lowerer: Lowerer, stmts: readonly ts.Statement[]): Ir }, }); if (fence) out.push(fence); + } else if (deferFences) { + // Inside a capture buffer the owner keeps the diagnostics, but + // the statement still must not vanish from a result it keeps. + const fence = lowerer.capturedRuntimeFence(sinkMark, stmt); + if (fence) out.push(fence); } } } @@ -898,6 +905,9 @@ export function predeclareForwardCapture(lowerer: Lowerer, symbol: ts.Symbol): b if (!decl.initializer) return false; const flags = ts.getCombinedNodeFlags(decl); if ((flags & ts.NodeFlags.BlockScoped) === 0 || (flags & ts.NodeFlags.Using) !== 0) return false; + // `const E = EventEmitter` has no storage to box: uses resolve at + // compile time (builtinClassAliasOf). + if (builtinClassAliasOwnsNoSlot(lowerer, symbol)) return false; // MODULE-scope bindings are pre-registered globals (collectGlobals): // references resolve through globalOf after the local search fails, so // a TDZ box here would SHADOW the global and never fill (the top-level @@ -1628,7 +1638,9 @@ function lowerVarDeclarator( // Relative requires keep the fence: their module-init-at-position // semantics needs storage this model doesn't represent. if (canonicalBuiltinModule(spec) !== null) { - return []; + return canonicalBuiltinModule(spec) === "punycode" + ? [lowerer.punycodeRequireWarning(decl)] + : []; } lowerer.unsupported( "SC1090", @@ -3329,6 +3341,9 @@ function stdlibGlobalTokenDestructure( token: string; alias: string | null; g: IrGlobal | undefined; + /** A data member's value (`const { env } = process`), bound instead + * of a token. */ + value?: IrExpr; }[] = []; for (const el of decl.name.elements) { if (el.name === undefined) continue; @@ -3364,8 +3379,9 @@ function stdlibGlobalTokenDestructure( "destructuring with computed keys that do not fold to one property name", ); } - let token: string; + let token = ""; let alias: string | null = null; + let value: IrExpr | null = null; if (globalName === "globalThis") { // The member must BE a stdlib global (user script vars are // properties of `typeof globalThis` too — their values are real @@ -3385,6 +3401,13 @@ function stdlibGlobalTokenDestructure( token = `[builtin console.${propName}]`; } else if (TOKEN_OPAQUE_GLOBALS.has(globalName)) { token = `[builtin ${globalName}.${propName}]`; + } else if ( + globalName === "process" && + ts.isIdentifier(el.name) && + (value = processDataMemberValue(lowerer, propName, el.name, locOf(el))) !== null + ) { + // A data member reads its value once, exactly like the spelled + // `process.` read the destructuring performs. } else { lowerer.unsupported( "SC1031", @@ -3401,6 +3424,10 @@ function stdlibGlobalTokenDestructure( ); } const g = lowerer.globalsBySymbol.get(symbol); + if (value !== null) { + binds.push({ name: el.name, token, alias, g, value }); + continue; + } if (g !== undefined && g.type.kind !== "string" && g.type.kind !== "dyn") { lowerer.unsupported( "SC1031", @@ -3413,6 +3440,21 @@ function stdlibGlobalTokenDestructure( const out: IrStmt[] = []; for (const b of binds) { const loc = locOf(b.name); + if (b.value !== undefined) { + if (b.g !== undefined) { + out.push({ + kind: "assign", + localId: b.g.id, + value: lowerer.coerceInto(b.name, b.value, b.g.type), + ...(b.g.tdz ? { initializes: true as const } : {}), + loc, + }); + } else { + const local = lowerer.declareLocal(b.name, b.name.text, b.value.type, isLet); + out.push({ kind: "varDecl", localId: local.id, init: b.value, loc }); + } + continue; + } const token: IrExpr = { kind: "strLit", value: b.token, type: STRING, loc }; if (b.alias !== null) { const symbol = lowerer.checker.getSymbolAtLocation(b.name); @@ -4444,7 +4486,7 @@ export function lowerVarDecl( // reads reuse its interned closure identity. if (!isLet && registerOverloadedCallableAlias(lowerer, decl)) return null; if (!isLet && registerBuiltinCallableAlias(lowerer, decl.name, decl.initializer)) return null; - if (registerZlibClassAlias(lowerer, decl.name, decl.initializer)) return null; + if (registerBuiltinClassAlias(lowerer, decl.name, decl.initializer)) return null; // `const knownBy = (cmd) => ...` — an IMPLICIT-ANY function-value // binding in npm-static JS (function-scope bindings register here, in @@ -4533,16 +4575,21 @@ export function lowerVarDecl( if (wrapped) return { kind: "assign", localId: g.id, value: wrapped, loc: locOf(decl) }; return null; } - const raw = lowerVariableInitializer( - lowerer, - decl.initializer, - g.type.kind === "dyn" && - (isNativeBuiltinValueInitializer(lowerer, decl.initializer) || - ts.isArrayLiteralExpression(decl.initializer) || - ts.isObjectLiteralExpression(decl.initializer)) - ? DYN - : undefined, - ); + const guardedRead = isGuardedRecordRead(lowerer, decl) + ? lowerAbsenceProbe(lowerer, decl.initializer) + : null; + const raw = + guardedRead ?? + lowerVariableInitializer( + lowerer, + decl.initializer, + g.type.kind === "dyn" && + (isNativeBuiltinValueInitializer(lowerer, decl.initializer) || + ts.isArrayLiteralExpression(decl.initializer) || + ts.isObjectLiteralExpression(decl.initializer)) + ? DYN + : undefined, + ); if ( isJsSourceFile(decl.getSourceFile()) && !decl.type && @@ -5338,29 +5385,111 @@ export function jsBindingHasOpenWrites(lowerer: Lowerer, decl: ts.VariableDeclar } function immediatelyGuardedAbsenceProbe(lowerer: Lowerer, decl: ts.VariableDeclaration): boolean { - if (decl.type || !ts.isIdentifier(decl.name) || !decl.initializer) return false; + const guard = immediateAbsenceGuard(lowerer, decl); + // `if (!x)`: the falsy test sees a missing value. + if (guard === "falsy") return true; + // `if (x === undefined)` and its spellings compare the binding with the + // missing value itself, so the read must be able to produce it. Only + // index-signature record reads take this form: array reads already + // carry their undefined arm, and their ordinary lowering keeps its + // bounds proofs. + return guard === "compare" && isIndexSignatureRecordRead(lowerer, decl); +} + +/** A file-scope or block-scope `let e = map[k]` over an index-signature + * record whose next statement guards `e` for absence (`if (e === + * undefined)`, `if (!e)`, ...): the get-or-create idiom. Its read keeps the + * missing key as undefined even without noUncheckedIndexedAccess. */ +export function isGuardedRecordRead(lowerer: Lowerer, decl: ts.VariableDeclaration): boolean { + return immediateAbsenceGuard(lowerer, decl) !== null && isIndexSignatureRecordRead(lowerer, decl); +} + +function isIndexSignatureRecordRead(lowerer: Lowerer, decl: ts.VariableDeclaration): boolean { + if (!decl.initializer) return false; + let init = decl.initializer; + while (ts.isParenthesizedExpression(init)) init = init.expression; + if (!ts.isElementAccessExpression(init)) return false; + // Under noUncheckedIndexedAccess the checker already spells the missing + // arm, and the ordinary read carries it. + const read = lowerer.typeOf(init); + if ( + (read.flags & (ts.TypeFlags.Undefined | ts.TypeFlags.Any | ts.TypeFlags.Unknown)) !== 0 || + (read.isUnionType() && + ts.constituentTypes(read).some((t) => (t.flags & ts.TypeFlags.Undefined) !== 0)) + ) + return false; + const receiver = lowerer.mapTypeOf(lowerer.typeOf(init.expression)); + const shape = receiver?.kind === "record" ? lowerer.shapes.get(receiver.shapeId) : undefined; + return !!shape && !shape.tuple && !!shape.indexValue && shape.indexValue.kind !== "dyn"; +} + +/** How the statement right after an unannotated declaration tests the + * declared binding for absence: `if (!x)` ("falsy"), or a comparison with + * the missing value — `x === undefined`, `x == null`, `x !== void 0`, + * `typeof x === "undefined"` and their mirrored and negated spellings + * ("compare"). Null for anything else. */ +function immediateAbsenceGuard( + lowerer: Lowerer, + decl: ts.VariableDeclaration, +): "falsy" | "compare" | null { + if (decl.type || !ts.isIdentifier(decl.name) || !decl.initializer) return null; const statement = decl.parent?.parent; - if (!statement || !ts.isVariableStatement(statement)) return false; + if (!statement || !ts.isVariableStatement(statement)) return null; const container = statement.parent; const statements = ts.isBlock(container) || ts.isSourceFile(container) ? container.statements : null; - if (!statements) return false; + if (!statements) return null; const index = statements.indexOf(statement); const next = index >= 0 ? statements[index + 1] : undefined; - if (!next || !ts.isIfStatement(next)) return false; - let condition = next.expression; - while (ts.isParenthesizedExpression(condition)) condition = condition.expression; - if ( - !ts.isPrefixUnaryExpression(condition) || - condition.operator !== ts.SyntaxKind.ExclamationToken - ) - return false; - let operand = condition.operand; - while (ts.isParenthesizedExpression(operand)) operand = operand.expression; - return ( - ts.isIdentifier(operand) && - lowerer.resolveValueSymbol(operand) === lowerer.resolveValueSymbol(decl.name) - ); + if (!next || !ts.isIfStatement(next)) return null; + const binding = lowerer.resolveValueSymbol(decl.name); + const peel = (node: ts.Expression): ts.Expression => { + while (ts.isParenthesizedExpression(node)) node = node.expression; + return node; + }; + const isBinding = (node: ts.Expression): boolean => { + const operand = peel(node); + return ts.isIdentifier(operand) && lowerer.resolveValueSymbol(operand) === binding; + }; + const condition = peel(next.expression); + if (ts.isPrefixUnaryExpression(condition)) + return condition.operator === ts.SyntaxKind.ExclamationToken && isBinding(condition.operand) + ? "falsy" + : null; + if (!ts.isBinaryExpression(condition)) return null; + const op = condition.operatorToken.kind; + const strict = + op === ts.SyntaxKind.EqualsEqualsEqualsToken || + op === ts.SyntaxKind.ExclamationEqualsEqualsToken; + const loose = + op === ts.SyntaxKind.EqualsEqualsToken || op === ts.SyntaxKind.ExclamationEqualsToken; + if (!strict && !loose) return null; + const isUndefined = (node: ts.Expression): boolean => { + const operand = peel(node); + if (ts.isVoidExpression(operand)) return true; + return ( + ts.isIdentifier(operand) && + operand.text === "undefined" && + (lowerer.typeOf(operand).flags & ts.TypeFlags.Undefined) !== 0 + ); + }; + const isNull = (node: ts.Expression): boolean => peel(node).kind === ts.SyntaxKind.NullKeyword; + const missing = (node: ts.Expression): boolean => isUndefined(node) || (loose && isNull(node)); + const isTypeofBinding = (node: ts.Expression): boolean => { + const operand = peel(node); + return ts.isTypeOfExpression(operand) && isBinding(operand.expression); + }; + const isUndefinedTag = (node: ts.Expression): boolean => { + const operand = peel(node); + return ts.isStringLiteralLike(operand) && operand.text === "undefined"; + }; + const { left, right } = condition; + return (isBinding(left) && missing(right)) || + (missing(left) && isBinding(right)) || + (isTypeofBinding(left) && isUndefinedTag(right)) || + (isUndefinedTag(left) && isTypeofBinding(right)) + ? "compare" + : null; } function falsyExitRuntimeOptionalLocals(lowerer: Lowerer, stmt: ts.IfStatement): IrLocal[] { diff --git a/packages/compiler/src/frontend/lowering/lower-test.ts b/packages/compiler/src/frontend/lowering/lower-test.ts index b9d4e8df1..e36518559 100644 --- a/packages/compiler/src/frontend/lowering/lower-test.ts +++ b/packages/compiler/src/frontend/lowering/lower-test.ts @@ -473,7 +473,7 @@ function lowerHook( return { kind: "libCall", fn: "test.hook", - args: [numLit(which, loc), cb, numLit(flags, loc)], + args: [numLit(which, loc), cb, numLit(flags, loc), strLit(atStringOf(expr), loc)], type: VOID, loc, }; diff --git a/packages/compiler/src/frontend/lowering/lowerer.ts b/packages/compiler/src/frontend/lowering/lowerer.ts index b77768c28..a1ded6aee 100644 --- a/packages/compiler/src/frontend/lowering/lowerer.ts +++ b/packages/compiler/src/frontend/lowering/lowerer.ts @@ -1,3 +1,4 @@ +import { jsDynamicDeclarations } from "./js-dynamic-objects.js"; import { lifetimeRegionType, lowerLifetimeRegionCall, @@ -110,6 +111,13 @@ import { ClassDynamicDispatch, trackClassFieldCreation } from "./class-dynamic-d import { ClassProtocols } from "./class-protocols.js"; import { earlyReadVarSymbols } from "./var-hoisting.js"; import { recordTextCodecClass } from "../../ir/ir.js"; +import { assignLegacyTiers, type DynamicFunction } from "../../ir/tiers.js"; +import { + analyzeTiers, + dynamicStatementCount, + type TierMode, + type TierReport, +} from "../tier-analysis.js"; import { DeferredModuleInitializers } from "./deferred-module-initializers.js"; import { finalizeClassMethodValues } from "./class-method-values.js"; import type { ClassSymbolKey } from "./symbol-fields.js"; @@ -134,6 +142,7 @@ import { resolve } from "node:path"; import { bindingSource } from "../binding-source.js"; import { tsgoPath } from "../dts-paths.js"; import * as ts from "../ts7/adapter.js"; +import { erasedTypeOnlyImport } from "../type-only-imports.js"; import { literalValues } from "../literal-values.js"; import { discriminantOwners, literalUnionArm, type UnionLiteral } from "../union-discriminants.js"; import type { ScrDiagnostic } from "../../diagnostics/diagnostic.js"; @@ -253,6 +262,7 @@ import { stdlibGlobalMember, nodeTypesOnlySymbol, } from "./surfaces.js"; +import { type BuiltinClassRef, builtinClassAliasOf } from "./builtins/class-values.js"; import { type FileParts, splitFiles, @@ -270,7 +280,7 @@ import { appendForkModules, appendWorkerModules, } from "./lower-modules.js"; -import { prepareCjsModuleGraph } from "./lower-node-module.js"; +import { prepareCjsModuleGraph, usesCjsTopLevelThis } from "./lower-node-module.js"; import { registerBuiltinEventTargetClasses } from "./lower-event-target.js"; import { type ClassInfo, @@ -532,6 +542,7 @@ import { lowerForOf, lowerForStatement, matchGroupsPatternSource, + isGuardedRecordRead, } from "./lower-stmts.js"; import { type FieldTarget, @@ -629,6 +640,9 @@ export interface LowerStats { /** Functions whose signature couldn't be analyzed (bodies not counted). */ functionsSkipped: number; + /** Coverage only: reached statements that compile into dynamic-tier + * functions (tier-analysis.ts), a subset of the compiled statements. */ + statementsDynamic?: number; } /** Every identifier a binding name binds: the identifier itself, or all @@ -703,6 +717,11 @@ export interface LowerResult { /** Compile-time-detectable behavior differences from Node (SC6xxx * divergence warnings) in the program's own modules. Never failures. */ divergences?: ScrDiagnostic[]; + /** Functions lowering placed in the dynamic tier (ir/tiers.ts), present + * when there are any. */ + dynamicFunctions?: DynamicFunction[]; + /** LowerOptions.tierReport: the report-only tier assignment. */ + tiers?: TierReport; } export interface LowerOptions { @@ -752,6 +771,12 @@ export interface LowerOptions { /** The mapped entries plus relative declaration dependencies, attributed * to their owning external specifier. */ externalTypeSpecifiersByFile?: ReadonlyMap; + /** `--tier` (tier-analysis.ts); "auto" when absent. */ + tierMode?: TierMode; + /** Coverage only: compute the tier-assignment report (`--explain-dynamic`). + * `preflight` holds the import and package refusals coverage reports + * beside the lowering's own blockers; they request module demotions. */ + tierReport?: { preflight: readonly ScrDiagnostic[] }; } interface RuntimeFenceFallback { @@ -827,6 +852,8 @@ export interface LowererMode { externalTypes?: ReadonlyMap; /** LowerOptions.externalTypeSpecifiersByFile, shared by every pass. */ externalTypeSpecifiersByFile?: ReadonlyMap; + /** LowerOptions.tierMode, shared by every pass. */ + tierMode?: TierMode; } function directExternalTypeSpecifiersByFile( @@ -918,6 +945,7 @@ export function lowerToIr( const externalTypes = options.externalTypes ?? new Map(); const externalTypeSpecifiersByFile = options.externalTypeSpecifiersByFile ?? directExternalTypeSpecifiersByFile(externalTypes); + const tierMode = options.tierMode ?? "auto"; const validation = new Lowerer(program, entry, moduleOrder, { frontendServices: options.frontendServices, projectCheckJs: options.projectCheckJs ?? false, @@ -929,6 +957,7 @@ export function lowerToIr( workerTargets, externalTypes, externalTypeSpecifiersByFile, + tierMode, }); const ffiValidation = validateFfiImports(validation); timing("ffi-validate"); @@ -951,6 +980,7 @@ export function lowerToIr( workerTargets, externalTypes, externalTypeSpecifiersByFile, + tierMode, ffiBindingSymbols: ffiValidation.symbolsByName, }); for (const d of extraModuleCycleDiags) reachableEmit.pushDiag(d); @@ -981,6 +1011,7 @@ export function lowerToIr( ffiBindingSymbols: ffiValidation.symbolsByName, externalTypes, externalTypeSpecifiersByFile, + tierMode, }); for (const d of extraModuleCycleDiags) emit.pushDiag(d); for (const d of ffiValidation.diagnostics) emit.pushDiag(d); @@ -1039,19 +1070,44 @@ export function lowerToIr( ffiBindingSymbols: ffiValidation.symbolsByName, externalTypes, externalTypeSpecifiersByFile, + tierMode, }); const rem = remainder.run(); + timing("coverage-remainder"); + // Tiers (tier-analysis.ts): reached statements in dynamic-tier functions, + // and with tierReport the report-only demotion analysis. + const dynamicFunctions = resultLowerer.dynamicBodies; + const tiers = + options.tierReport === undefined + ? undefined + : analyzeTiers(program, { + mode: tierMode, + dynamicFunctions, + reached: [...options.tierReport.preflight, ...result.diagnostics], + deferred: result.runtimeFences, + unreached: [...rem.diagnostics, ...rem.runtimeFences], + }); + // A dynamic function's own statements can include ones that failed. + const statementsDynamic = Math.min( + tiers?.statementsDynamic ?? dynamicStatementCount(program, dynamicFunctions), + result.stats.statementsTotal - result.stats.statementsFailed, + ); + if (tiers !== undefined) tiers.statementsDynamic = statementsDynamic; + timing("tier-analysis", { dynamic: dynamicFunctions.length }); // Unreached JS statements whose fences deferred to runtime count as // failed statements in the remainder's stats, so their diagnostics must // travel with them: a failed statement is never left unexplained. - return { + const covered: LowerResult = { ...result, + stats: { ...result.stats, statementsDynamic }, unreached: { diagnostics: rem.diagnostics, stats: rem.stats, ...(rem.runtimeFences.length > 0 ? { runtimeFences: rem.runtimeFences } : {}), }, }; + if (tiers !== undefined) covered.tiers = tiers; + return covered; } /** The native namespace type produced by a literal static import() of a @@ -1804,11 +1860,11 @@ export class Lowerer { * typeParamBindings — the checker has no `T` to substitute, so the * binding rides the node-type accessor instead of mapType. */ implicitParamTypes: Map | null = null; - /** Immutable bindings that name a zlib stream class (`const C = - * zlib.Gzip`), keyed by the binding's symbol: the class's - * node_zlib_mode. No runtime slot exists: new, typeof, .name and === - * resolve at compile time, and instanceof checks the stream's mode. */ - readonly zlibClassAliases = new Map(); + /** The builtin class each binding names (`const E = EventEmitter`, + * `const C = zlib.Gzip`), keyed by the binding's symbol; null for + * bindings that name none (builtinClassAliasOf). new, instanceof, + * typeof, .name and === resolve at compile time. */ + readonly builtinClassAliases = new Map(); /** Whether some net.Socket in this program may get an encoding: a * setEncoding call whose receiver is a socket or an untyped value. * Computed once on first use (null until then). */ @@ -1858,6 +1914,9 @@ export class Lowerer { * carry undefined at runtime. Identifier reads keep the union tag so * typeof and later boundaries observe the real value. */ readonly runtimeOptionalGlobals = new Set(); + /** Per CommonJS file using top-level `this`: the global holding the + * object that stands for its original exports (lowerCjsTopLevelThis). */ + readonly cjsExportsThis = new Map(); /** Bindings whose unchecked string arithmetic can produce either a number * (NaN for an absent read) or a string. Preserve that result union at bare * reads instead of applying the checker-bare string narrowing adapter. */ @@ -2692,6 +2751,14 @@ export class Lowerer { /** Coverage remainder mode: the reachability gate inverts (see wantBody) * and no module is built. */ readonly remainder: boolean; + /** `--tier` (tier-analysis.ts). Stage S2 is report-only: both modes + * refuse and fence exactly as before. Stage S6 demotes the units of + * committed refusals in "auto" and keeps them refusals in "static-only". */ + readonly tierMode: TierMode; + /** Functions finishModule placed in the dynamic tier (ir/tiers.ts). */ + dynamicFunctions: DynamicFunction[] = []; + /** Their IR, for coverage's dynamic statement count. */ + dynamicBodies: IrFunction[] = []; /** Deferred collection diagnostics (failed signatures/class shapes) by * declaration symbol: an unreached declaration must not fail the build, * so its diagnostics wait until a reference makes them relevant. */ @@ -2778,6 +2845,7 @@ export class Lowerer { this.frontendServices = mode.frontendServices; this.reachable = mode.reachable ?? null; this.remainder = mode.remainder ?? false; + this.tierMode = mode.tierMode ?? "auto"; this.alreadyFlushed = mode.alreadyFlushed ?? new Set(); this.targetPlatform = mode.targetPlatform ?? process.platform; this.startupCrash = mode.startupCrash ?? null; @@ -2867,7 +2935,8 @@ export class Lowerer { return null; } const className = this.classNamer(decl); - if (!this.collectingExprClasses.has(decl)) this.lowerClassExpressionInfo(decl); + if (!this.collectingExprClasses.has(decl) && !this.collectLocalClassForType(decl)) + return null; return { kind: "object", className }; }, mixinIntersectionInstance: (widened) => mixinIntersectionInstanceType(this, widened), @@ -2902,6 +2971,8 @@ export class Lowerer { this.fileTag.set(sf, sf === entry ? "" : `%m${i}.`); }); if (this.moduleOrder.length === 0) this.fileTag.set(entry, ""); + const jsDynamic = jsDynamicDeclarations(this.moduleOrder, this.checker); + if (jsDynamic.size > 0) this.typeCtx.jsDynamicDeclaration = (decl) => jsDynamic.has(decl); this.registerBuiltinErrorClasses(); registerBuiltinEmitterClass(this); registerBuiltinStreamClasses(this); @@ -3004,6 +3075,10 @@ export class Lowerer { return builtinStreamInfoOf(this, symbol); } + builtinClassAliasOf(symbol: ts.Symbol): BuiltinClassRef | null { + return builtinClassAliasOf(this, symbol); + } + /** Program-wide qualified name for a top-level declaration. */ qualify(sf: ts.SourceFile, name: string): string { return `${this.fileTag.get(sf) ?? ""}${name}`; @@ -3698,6 +3773,16 @@ export class Lowerer { const id = `%g.${tag}%loaded`; this.moduleGuardOf.set(fp.sf, id); this.globalsList.push({ id, name: "%loaded", type: BOOL, mutable: true }); + if (usesCjsTopLevelThis(this, fp.sf)) { + const exportsThis: IrGlobal = { + id: `%g.${tag}%exportsThis`, + name: "%exportsThis", + type: DYN, + mutable: true, + }; + this.globalsList.push(exportsThis); + this.cjsExportsThis.set(fp.sf, exportsThis); + } } // A module is intrinsically async when an await/for-await occurs @@ -3817,6 +3902,65 @@ export class Lowerer { for (const fp of parts) if (!indexOf.has(fp.sf)) visit(fp.sf); } + /** node:punycode's load at a require() call: the module body reports + * DEP0040 when it first runs, unless isInsideNodeModules() finds the + * requiring frame in a dependency, and process.emitWarning delivers the + * report on the next tick. Later loads are cache hits either way. */ + punycodeRequireWarning(node: ts.Node): IrStmt { + const loc = locOf(node); + const fromDependency = /[\\/]node_modules[\\/]/.test(node.getSourceFile().fileName); + return { + kind: "exprStmt", + expr: { + kind: "libCall", + fn: "punycode.warn", + args: [ + { kind: "boolLit", value: false, type: BOOL, loc }, + { kind: "boolLit", value: !fromDependency, type: BOOL, loc }, + ], + type: VOID, + loc, + }, + loc, + }; + } + + /** node:punycode's DEP0040 for a static ES import anywhere in the module + * graph: Node links builtins before any module evaluates, so the report + * comes first, before the program can register a 'warning' listener — + * whichever file imports it (a dependency's ES import has no requiring + * frame either). Prepended to the entry's %init. */ + punycodeLinkWarning(): IrStmt[] { + const imports = this.moduleOrder.some((sf) => + sf.statements.some( + (stmt) => + ((ts.isImportDeclaration(stmt) && !erasedTypeOnlyImport(stmt)) || + (ts.isExportDeclaration(stmt) && !stmt.isTypeOnly)) && + stmt.moduleSpecifier !== undefined && + ts.isStringLiteral(stmt.moduleSpecifier) && + canonicalBuiltinModule(stmt.moduleSpecifier.text) === "punycode", + ), + ); + if (!imports) return []; + const loc: SrcLoc = { file: this.entry.fileName, start: 0, end: 0 }; + return [ + { + kind: "exprStmt", + expr: { + kind: "libCall", + fn: "punycode.warn", + args: [ + { kind: "boolLit", value: true, type: BOOL, loc }, + { kind: "boolLit", value: true, type: BOOL, loc }, + ], + type: VOID, + loc, + }, + loc, + }, + ]; + } + /** The lowering of a CommonJS `require("./local")` occurrence: a call of * the required module's run-once %init at exactly this statement's * position — Node's inline evaluation, with the guard supplying the @@ -3826,6 +3970,7 @@ export class Lowerer { * their external dependencies by name). Null for everything else * (builtins load nothing; the rest kept its preflight fence). */ requireInitStmt(spec: string, node: ts.Node): IrStmt | null { + if (canonicalBuiltinModule(spec) === "punycode") return this.punycodeRequireWarning(node); // Project requires resolve within the program; another bare require can // be a program-module edge when it names an opted-in --npm-static package // (one package requiring another — the resolution answered its shipped @@ -3952,6 +4097,33 @@ export class Lowerer { } } + /** A file-scope get-or-create read (`let e = map[k]; if (e === + * undefined) ...` over an index-signature record) keeps a missing key as + * undefined (isGuardedRecordRead). Its pre-registered global takes the + * undefined arm before any body reads it; the declaration lowers its + * initializer as the matching absence probe. */ + analyzeGuardedRecordReadGlobals(parts: FileParts[]): void { + for (const { sf } of parts) { + for (const statement of sf.statements) { + if (!ts.isVariableStatement(statement)) continue; + for (const decl of statement.declarationList.declarations) { + if (!ts.isIdentifier(decl.name) || !isGuardedRecordRead(this, decl)) continue; + const global = this.globalOf(decl.name); + const symbol = this.resolveValueSymbol(decl.name); + if (!global || !symbol || global.tdz || this.runtimeOptionalBindingTypes.has(symbol)) + continue; + const widened = + global.type.kind === "union" + ? this.withUndefinedArmOf(global.type) + : this.withUndefinedArm(global.type); + if (!widened || typeEquals(widened, global.type)) continue; + global.type = widened; + this.runtimeOptionalBindingTypes.set(symbol, widened); + } + } + } + } + analyzeRuntimeOptionalArrayReads(parts: FileParts[]): void { const provenance = RuntimeOptionalProvenance.fromEnvironment(); // The site whose value the current promotion follows (debug only). @@ -3976,18 +4148,6 @@ export class Lowerer { if (!s) return null; return s.flags & ts.SymbolFlags.Alias ? this.checker.getAliasedSymbol(s) : s; }; - // Fields that can be read before their assignment hold undefined then; - // their reads are runtime-optional values like missing array reads. - for (const info of this.classes.values()) { - if (!info.earlyReadFields || !info.decl?.name) continue; - const classSymbol = symbolOf(info.decl.name); - if (!classSymbol) continue; - const instance = this.checker.getDeclaredTypeOfSymbol(classSymbol); - for (const name of info.earlyReadFields) { - const property = this.checker.getPropertyOfType(instance, name); - if (property) optionalSymbols.add(property); - } - } const signatureOf = (node: ts.Identifier): FnSig | null => { const symbol = symbolOf(node); if (!symbol) return null; @@ -6206,8 +6366,11 @@ export class Lowerer { for (const fp of parts) this.checker.prefetchSourceFile(fp.sf); } this.collectProgram(parts); - this.analyzeRuntimeOptionalArrayReads(parts); - this.analyzeDateJsonResults(parts); + this.speculativeTypeQueries(() => { + this.analyzeRuntimeOptionalArrayReads(parts); + this.analyzeDateJsonResults(parts); + this.analyzeGuardedRecordReadGlobals(parts); + }); // Decorated classes analyze AFTER the whole collection pass: a // decorator's return type may name a subclass declared below the // class, and reference lowering needs each class's rebindability @@ -6458,11 +6621,18 @@ export class Lowerer { }; if (module) sanitizeUnregisteredClassTypes(module, (name) => this.classes.has(name)); if (module) checkCycleBindingAccesses(this, module); + // Execution tiers from the final bodies (ir/tiers.ts). Assigned with or + // without a module: coverage reports the dynamic tier of programs that + // still have blockers. + this.dynamicFunctions = assignLegacyTiers(functions); + if (this.dynamicFunctions.length > 0) + this.dynamicBodies = functions.filter((fn) => fn.tier === "dynamic"); return { module, diagnostics: this.diags, runtimeFences: this.runtimeFences, stats: this.stats, + ...(this.dynamicFunctions.length > 0 ? { dynamicFunctions: this.dynamicFunctions } : {}), ...(this.statsByFile.size > 0 ? { statsByFile: this.statsByFile } : {}), ...(this.provenanceElided.length > 0 ? { provenanceElided: this.provenanceElided } : {}), }; @@ -6592,8 +6762,11 @@ export class Lowerer { collectionEdges.add(name); }; this.collectProgram(parts); - this.analyzeRuntimeOptionalArrayReads(parts); - this.analyzeDateJsonResults(parts); + this.speculativeTypeQueries(() => { + this.analyzeRuntimeOptionalArrayReads(parts); + this.analyzeDateJsonResults(parts); + this.analyzeGuardedRecordReadGlobals(parts); + }); // Decorated classes analyze post-collection here too: the %init seeds // lower the decoration calls, whose edges (decorator bodies, construct // thunks) reachable emit must see. @@ -7087,23 +7260,8 @@ export class Lowerer { } this.runtimeFences.push(...captured); - const first = captured[0]; + const fence = this.runtimeFenceStmt(captured[0], node, target.fallback, target.bareMessage); const loc = locOf(node); - const code = first?.code ?? target.fallback!.code; - const baseMessage = first?.message ?? target.fallback!.message; - const messageLoc = first?.loc ?? loc; - const source = first - ? (this.program.getSourceFile(messageLoc.file) ?? node.getSourceFile()) - : node.getSourceFile(); - const pos = ts.getLineAndCharacterOfPosition(source, messageLoc.start); - const fence: IrStmt = { - kind: "runtimeFence", - code, - message: target.bareMessage - ? baseMessage - : `${baseMessage} [${code} at ${messageLoc.file}:${pos.line + 1}]`, - loc, - }; if (target.kind === "statement") return fence; const params = target.params ?? []; @@ -7131,6 +7289,54 @@ export class Lowerer { return { kind: "closure", fnName: fn.name, captures: [], type: target.type, loc }; } + /** The runtimeFence statement that throws `first`'s message (or the + * fallback's) at `node`. */ + private runtimeFenceStmt( + first: ScrDiagnostic | undefined, + node: ts.Node, + fallback: { code: string; message: string } | undefined, + bareMessage = false, + ): IrStmt & { kind: "runtimeFence" } { + const loc = locOf(node); + const code = first?.code ?? fallback?.code ?? "SC1090"; + const baseMessage = + first?.message ?? + fallback?.message ?? + "this statement uses a construct with no static lowering"; + const messageLoc = first?.loc ?? loc; + const source = first + ? (this.program.getSourceFile(messageLoc.file) ?? node.getSourceFile()) + : node.getSourceFile(); + const pos = ts.getLineAndCharacterOfPosition(source, messageLoc.start); + return { + kind: "runtimeFence", + code, + message: bareMessage + ? baseMessage + : `${baseMessage} [${code} at ${messageLoc.file}:${pos.line + 1}]`, + loc, + }; + } + + /** The current length of the capture buffer (diagSink), or 0 without one: + * the mark capturedRuntimeFence reads the refusal from. */ + diagSinkMark(): number { + return this.diagSink?.length ?? 0; + } + + /** The runtimeFence for a JavaScript statement refused while a capture + * buffer (diagSink) is active. The buffer's owner decides whether its + * diagnostics report, and some owners keep the lowered result (a + * successful tryLowerExpression candidate, a computed class base whose + * diagnostics defer to the class). Without a fence the refused statement + * would vanish from that result; with one it throws the refusal when it + * runs, exactly like the fence lowerStmts emits outside a buffer. The + * diagnostics stay in the buffer for its owner. Null without a buffer. */ + capturedRuntimeFence(sinkMark: number, node: ts.Node): IrStmt | null { + if (this.diagSink === null) return null; + return this.runtimeFenceStmt(this.diagSink[sinkMark], node, undefined); + } + /** All diagnostics land here; while a generic instance body is lowering, * the instantiation context is appended so the user knows which concrete * types made the (source-anchored) construct fail. */ @@ -9745,6 +9951,43 @@ export class Lowerer { return lowerClassExpressionInfo(this, expr); } + /** Depth of the whole-program analyses that query types across every + * file before reachability is known (speculativeTypeQueries). */ + private speculativeTypeQueryDepth = 0; + + /** Runs a whole-program analysis whose type queries are speculative: they + * visit every file, reached or not, and no statement or declaration owns + * a refusal they trigger (collectLocalClassForType). */ + speculativeTypeQueries(analysis: () => void): void { + this.speculativeTypeQueryDepth++; + try { + analysis(); + } finally { + this.speculativeTypeQueryDepth--; + } + } + + /** Collects a function-local class because a type query named its + * instance type. Collection and body lowering own a refusal of that + * class: the poison reaches the declaration or statement that asked, as + * before. A speculative whole-program analysis owns none, so there the + * refusal stays reported (the build fails with it, as it did where such + * an analysis already caught the poison) and the class is unmappable for + * the query instead of the poison escaping the compiler. */ + collectLocalClassForType(decl: ts.ClassLikeDeclaration): boolean { + if (this.speculativeTypeQueryDepth === 0 || this.fnStack.length > 0) { + this.lowerClassExpressionInfo(decl); + return true; + } + try { + this.lowerClassExpressionInfo(decl); + return true; + } catch (e) { + if (e instanceof PoisonError) return false; + throw e; + } + } + lowerClassExpression(expr: ts.ClassLikeDeclaration): IrExpr { return lowerClassExpression(this, expr); } @@ -11549,7 +11792,23 @@ export class Lowerer { return { module: "timers/promises", member: `scheduler.${access.name.text}` }; } } + // punycode.ucs2.decode / encode: methods of the module's ucs2 object, + // dispatched as module members. + if (access.name.text === "decode" || access.name.text === "encode") { + const owner = access.expression; + const imported = ts.isIdentifier(owner) + ? this.builtinImportOf(owner) + : ts.isPropertyAccessExpression(owner) + ? this.builtinMemberOf(owner) + : null; + if (imported?.module === "punycode" && imported.member === "ucs2") { + return { module: "punycode", member: `ucs2.${access.name.text}` }; + } + } const module = this.builtinNamespaceModuleOf(access.expression); + // node:constants is one checked-dynamic object: its members read + // through the object itself (builtinConstantsObject), not the tables. + if (module === "constants") return null; if (module !== null) return { module, member: access.name.text }; // The events module object IS the EventEmitter class, so the class's // defaultMaxListeners is the module property. diff --git a/packages/compiler/src/frontend/lowering/surfaces.ts b/packages/compiler/src/frontend/lowering/surfaces.ts index d394c1960..44ed37e3c 100644 --- a/packages/compiler/src/frontend/lowering/surfaces.ts +++ b/packages/compiler/src/frontend/lowering/surfaces.ts @@ -1822,6 +1822,22 @@ export const BUILTIN_MODULE_FNS: Record< valueParams: [{ mode: "required", type: STRING }], }, }, + // node:punycode (deprecated): punycode.js 2.1.0's conversions over + // strings (scr_url.c). ucs2.decode/encode are members of the module's + // ucs2 object, dispatched as module members (builtinMemberOf). The + // DEP0040 report rides the module's load sites (punycodeLoadWarning). + punycode: { + decode: { fn: "punycode.decode", params: [STRING], result: STRING }, + encode: { fn: "punycode.encode", params: [STRING], result: STRING }, + toASCII: { fn: "punycode.toASCII", params: [STRING], result: STRING }, + toUnicode: { fn: "punycode.toUnicode", params: [STRING], result: STRING }, + "ucs2.decode": { fn: "punycode.ucs2Decode", params: [STRING], result: arrayOf(F64) }, + "ucs2.encode": { fn: "punycode.ucs2Encode", params: [arrayOf(F64)], result: STRING }, + }, + // node:constants (deprecated) is one frozen object (Object.assign of + // os.constants' groups, fs.constants and crypto.constants): bindings of + // the module read it as a checked-dynamic value (builtinConstantsObject). + constants: {}, // node:readline: createInterface's options are entirely special-cased // (exactly { input: process.stdin, output?: process.stdout } — see // lowerBuiltinModuleCall); the entry carries the canonical shape. The @@ -1951,6 +1967,7 @@ export const BUILTIN_MODULE_CONSTS: Record< os: { EOL: "\n", devNull: "/dev/null" }, worker_threads: { isMainThread: true, threadId: 0 }, cluster: { isPrimary: true, isMaster: true, isWorker: false }, + punycode: { version: "2.1.0" }, // zlib's module-level copies of its constants (every member outside // BROTLI_*, Node's own loop over zlib.constants). zlib: Object.fromEntries( @@ -3207,7 +3224,9 @@ export function isStdlibSymbol(lowerer: Lowerer, symbol: ts.Symbol | undefined): * `global.process`) — `declare var` globals ARE properties of * `typeof globalThis`, same symbol either way; * - a local alias binding (`const process = globalThis.process`, the - * tamper-guard prologue) registered in stdlibGlobalAliases. */ + * tamper-guard prologue) registered in stdlibGlobalAliases; + * - a const binding of the Map or Set constructor (`const M = Map`, + * builtinClassAliasOf). */ export function stdlibGlobalNameOf(lowerer: Lowerer, expr: ts.Expression): string | null { if (ts.isParenthesizedExpression(expr)) return stdlibGlobalNameOf(lowerer, expr.expression); if (ts.isIdentifier(expr)) { @@ -3225,6 +3244,8 @@ export function stdlibGlobalNameOf(lowerer: Lowerer, expr: ts.Expression): strin if (!symbol) return null; const alias = lowerer.stdlibGlobalAliases.get(symbol); if (alias !== undefined) return alias; + const classAlias = lowerer.builtinClassAliasOf(symbol); + if (classAlias?.kind === "collection") return classAlias.which; const decl = lowerer.checker.declarationsOf(symbol)[0]; if (decl !== undefined && ts.isImportClause(decl) && decl.name !== undefined) { const importDecl = decl.parent; diff --git a/packages/compiler/src/frontend/npm-static-prune.ts b/packages/compiler/src/frontend/npm-static-prune.ts index c6c1f9caf..b71d6ca33 100644 --- a/packages/compiler/src/frontend/npm-static-prune.ts +++ b/packages/compiler/src/frontend/npm-static-prune.ts @@ -10,12 +10,27 @@ import { nearestPackageType, resolveBareModule } from "./resolve.js"; import { npmStaticPackageOfPath } from "./npm-static.js"; import { trackedReadFile } from "./input-tracker.js"; import { npmPackageNameOf } from "./workspace-registry.js"; +import { typeOnlyNamedImport } from "./type-only-imports.js"; export function isPrunedNpmReexport(program: ts.Program, stmt: ts.ExportDeclaration): boolean { return program.analysis.prunedNpmReexports.has(stmt); } -function packageJson(path: string): Record | null { +type PackageJson = Record | null; + +/** Parsed package.json files by path, kept for one plan: every purity check + * reads the metadata of its file's package again. */ +type PackageJsonCache = Map; + +function packageJson(path: string, cache: PackageJsonCache): PackageJson { + const cached = cache.get(path); + if (cached !== undefined) return cached; + const json = readPackageJson(path); + cache.set(path, json); + return json; +} + +function readPackageJson(path: string): PackageJson { const source = trackedReadFile(path); if (source === null) return null; try { @@ -58,12 +73,16 @@ function sideEffectPattern(pattern: string): RegExp { /** A nested format scope such as dist/esm/package.json is not the package's * published metadata. Stay inside the resolved package while finding its * named root, including pnpm's realpath and workspace-linked installs. */ -function packageRootJsonPath(fromFile: string, packageName: string): string | null { +function packageRootJsonPath( + fromFile: string, + packageName: string, + cache: PackageJsonCache, +): string | null { let dir = dirname(resolve(fromFile)); let root: string | null = null; while (npmPackageNameOf(dir.replaceAll("\\", "/")) === packageName) { const path = join(dir, "package.json"); - if (packageJson(path)?.["name"] === packageName) root = path; + if (packageJson(path, cache)?.["name"] === packageName) root = path; const parent = dirname(dir); if (parent === dir) break; dir = parent; @@ -73,9 +92,13 @@ function packageRootJsonPath(fromFile: string, packageName: string): string | nu // An external package's executable sources may not be in this program. // Preserve the conservative whole-dependency promise in that case. -function pureExternalPackage(path: string, visiting = new Set()): boolean { +function pureExternalPackage( + path: string, + cache: PackageJsonCache, + visiting = new Set(), +): boolean { if (visiting.has(path)) return true; - const json = packageJson(path); + const json = packageJson(path, cache); const value = json?.["sideEffects"]; if (json === null || !(value === false || (Array.isArray(value) && value.length === 0))) return false; @@ -85,8 +108,8 @@ function pureExternalPackage(path: string, visiting = new Set()): boolea if (!dependencies || typeof dependencies !== "object" || Array.isArray(dependencies)) continue; for (const name of Object.keys(dependencies)) { const resolved = resolveBareModule(path, name, "js-only"); - const root = resolved === null ? null : packageRootJsonPath(resolved.typesFile, name); - if (root === null || !pureExternalPackage(root, visiting)) return false; + const root = resolved === null ? null : packageRootJsonPath(resolved.typesFile, name, cache); + if (root === null || !pureExternalPackage(root, cache, visiting)) return false; } } return true; @@ -135,10 +158,11 @@ export function planNpmStaticReexports( for (const path of extraRoots) request(program.getSourceFile(path) ?? null, null); const pureMemo = new Map(); + const packageJsons: PackageJsonCache = new Map(); const filePure = (sf: ts.SourceFile): boolean => { const pkg = npmPackageNameOf(sf.fileName) ?? npmStaticPackageOfPath(sf.fileName); - const path = pkg === null ? null : packageRootJsonPath(sf.fileName, pkg); - const json = path === null ? null : packageJson(path); + const path = pkg === null ? null : packageRootJsonPath(sf.fileName, pkg, packageJsons); + const json = path === null ? null : packageJson(path, packageJsons); const value = json?.["sideEffects"]; if (value === false) return true; if ( @@ -195,8 +219,10 @@ export function planNpmStaticReexports( const resolved = resolveBareModule(sf.fileName, edge.spec, "js-only"); const pkg = resolved === null ? null : npmPackageNameOf(resolved.typesFile); const root = - pkg === null || resolved === null ? null : packageRootJsonPath(resolved.typesFile, pkg); - return root !== null && pureExternalPackage(root); + pkg === null || resolved === null + ? null + : packageRootJsonPath(resolved.typesFile, pkg, packageJsons); + return root !== null && pureExternalPackage(root, packageJsons); }); }; const pure = visit(root); @@ -232,7 +258,9 @@ export function planNpmStaticReexports( (clause.namedBindings !== undefined && (ts.isNamespaceImport(clause.namedBindings) || (ts.isNamedImports(clause.namedBindings) && - clause.namedBindings.elements.length === 0))) + clause.namedBindings.elements.length === 0))) || + // `import { type A }` still loads the module, like `import {}`. + typeOnlyNamedImport(stmt) ) { request(dep, null); } else { @@ -244,12 +272,10 @@ export function planNpmStaticReexports( } } } else if (ts.isExportDeclaration(stmt)) { + // `export { type A } from` still loads the module under Node's + // type stripping; only `export type … from` erases. if ( stmt.isTypeOnly || - (stmt.exportClause !== undefined && - ts.isNamedExports(stmt.exportClause) && - stmt.exportClause.elements.length > 0 && - stmt.exportClause.elements.every((element) => element.isTypeOnly)) || stmt.moduleSpecifier === undefined || !ts.isStringLiteral(stmt.moduleSpecifier) ) diff --git a/packages/compiler/src/frontend/npm-static.ts b/packages/compiler/src/frontend/npm-static.ts index 1f83a12d4..d2f4ea3e8 100644 --- a/packages/compiler/src/frontend/npm-static.ts +++ b/packages/compiler/src/frontend/npm-static.ts @@ -190,6 +190,10 @@ export function npmStaticTransformPkgJson(pkg: Record): void { for (const value of Object.values(obj)) transform(value); }; if (pkg["exports"] !== undefined) transform(pkg["exports"]); + // The "imports" field maps "#" specifiers with the same condition objects + // (chalk 5's `"#supports-color": { node, default }`), so Node's "node" + // target hoists there too. + if (pkg["imports"] !== undefined) transform(pkg["imports"]); } /** npmStaticTransformPkgJson over a package.json TEXT (the fs shadow's diff --git a/packages/compiler/src/frontend/pipeline.ts b/packages/compiler/src/frontend/pipeline.ts index 2ec9553f6..c769ed99e 100644 --- a/packages/compiler/src/frontend/pipeline.ts +++ b/packages/compiler/src/frontend/pipeline.ts @@ -25,9 +25,18 @@ import { isJsSourceFileName } from "./tsc-codes.js"; import { isRelativeSpecifier, packageNameOfSpecifier } from "./workspace-registry.js"; import { lowerToIr, type LowerOptions, type LowerResult } from "./lowering/lowerer.js"; import type { FrontendServices } from "./services.js"; -import { isImportDeclaration, isExportDeclaration, isStringLiteral } from "./ts7/adapter.js"; +import { + isIdentifier, + isImportDeclaration, + isExportDeclaration, + isStringLiteral, + isStringLiteralLike, + SyntaxKind, +} from "./ts7/adapter.js"; +import { moduleSourceCandidates } from "./module-source-candidates.js"; import { CONTRACT_TYPES_MODULE } from "./contract-types.js"; import { THREADS_MODULE } from "./threads-import.js"; +import { erasedTypeOnlyImport } from "./type-only-imports.js"; import { EXPERIMENTAL_MODULE } from "./experimental-import.js"; export interface ProgramLoadOptions { @@ -91,15 +100,36 @@ function detectAutoPackages( mode: "auto" | "lib" = "auto", judged?: Set, sites?: Map, + transitive: Set = new Set(), ): string[] { + const chosen: string[] = []; // package → the resolved types file AND the file whose import found it: // the runtime-JS probe below must resolve from the SAME importing file, // or a package visible only to a nested package.json realm (a pnpm // monorepo's packages/*/node_modules, unreachable from the entry's own // walk-up) answers "no runtime JS" for perfectly ordinary installs. - const seen = new Map(); - for (const sf of [...load.moduleOrder, load.entry]) { + const seen = new Map< + string, + { typesFile: string; fromFile: string; spec: string; owner: string | null } + >(); + // A transitive package's files outside the module order (reached through + // nested require() calls the graph walk does not model) still name its + // dependencies. + const scanned = new Set([...load.moduleOrder, load.entry]); + const transitiveFiles = + transitive.size === 0 + ? [] + : load.program.getImplementationSourceFiles().filter((sf) => { + const pkg = npmStaticPackageOfPath(sf.fileName); + return !scanned.has(sf) && pkg !== null && transitive.has(pkg); + }); + for (const sf of [...scanned, ...transitiveFiles]) { if (mode === "auto" && sf.fileName.includes("/node_modules/")) continue; + // `--npm-static=pkg...` opts pkg's whole dependency closure in: every + // package its compiled files import is selected the same explicit way, + // without the declaration eligibility bar. + const owner = npmStaticPackageOfPath(sf.fileName); + const viaTransitive = owner !== null && transitive.has(owner); const edges: { spec: string; loc: SrcLoc }[] = []; for (const stmt of sf.statements) { if ( @@ -107,13 +137,31 @@ function detectAutoPackages( stmt.moduleSpecifier && isStringLiteral(stmt.moduleSpecifier) ) { + // `import type` and `export type … from` erase under Node's type + // stripping: Node never loads the package, so it is no dependency + // to compile (its declarations still type the program). + if (isImportDeclaration(stmt) ? erasedTypeOnlyImport(stmt) : stmt.isTypeOnly) continue; edges.push({ spec: stmt.moduleSpecifier.text, loc: locOf(stmt) }); - } else if (mode === "lib") { + } else if (mode === "lib" && !viaTransitive) { // CJS packages spell their dep edges as top-level requires; the // import-statement scan alone would miss every one of them. for (const req of requiresOf(stmt)) edges.push({ spec: req.spec, loc: locOf(req.node) }); } } + // Transitive selection counts every literal require() and import() + // anywhere in the package, nested ones included. + if (viaTransitive) { + for (const call of moduleSourceCandidates(load.program, sf).calls) { + const arg = call.arguments[0]; + if ( + arg !== undefined && + isStringLiteralLike(arg) && + (call.expression.kind === SyntaxKind.ImportKeyword || + (isIdentifier(call.expression) && call.expression.text === "require")) + ) + edges.push({ spec: arg.text, loc: locOf(call) }); + } + } for (const { spec, loc } of edges) { if (load.externalTypes?.has(spec)) continue; // scriptc's own packages are not npm dependencies: contract types are @@ -137,28 +185,51 @@ function detectAutoPackages( if (judged?.has(packageNameOfSpecifier(spec))) continue; const npm = resolveNpmImport(sf.fileName, spec); if (npm !== null && isNodeTypesPath(npm.typesFile)) continue; - if (npm === null) { - if (mode !== "lib") continue; - const js = resolveBareModule(sf.fileName, spec, "js-only"); - if (js === null || judged!.has(js.packageName)) continue; - judged!.add(js.packageName); - sites!.set(js.packageName, loc); + // The runtime package Node loads. Types-first resolution answers an + // @types twin's declarations for a package that ships none, but + // selection and every refusal name the package that has the code. + const js = + mode === "lib" && (npm === null || npm.packageName.startsWith("@types/")) + ? resolveBareModule(sf.fileName, spec, "js-only") + : null; + const runtimePkg = js !== null && js.packageName !== npm?.packageName ? js.packageName : null; + if (npm === null || runtimePkg !== null) { + if (mode !== "lib" || runtimePkg === null || judged!.has(runtimePkg)) continue; + judged!.add(runtimePkg); + sites!.set(runtimePkg, loc); + if (viaTransitive) { + transitive.add(runtimePkg); + chosen.push(runtimePkg); + continue; + } statuses.push({ - package: js.packageName, + package: runtimePkg, status: "refused", - detail: "it ships no own .d.ts declaration surface", + detail: + npm === null + ? "it ships no own .d.ts declaration surface" + : `its declared types come from the separate ${npm.packageName} package, not the package itself`, + hint: npmStaticOptInHint(runtimePkg, owner), }); continue; } if (judged?.has(npm.packageName)) continue; + // An @types twin with no runtime package installed has nothing to + // compile (Node cannot load it either). + if (viaTransitive && !npm.packageName.startsWith("@types/")) { + judged!.add(npm.packageName); + sites!.set(npm.packageName, loc); + transitive.add(npm.packageName); + chosen.push(npm.packageName); + continue; + } if (!seen.has(npm.packageName)) { - seen.set(npm.packageName, { typesFile: npm.typesFile, fromFile: sf.fileName, spec }); + seen.set(npm.packageName, { typesFile: npm.typesFile, fromFile: sf.fileName, spec, owner }); sites?.set(npm.packageName, loc); } } } - const chosen: string[] = []; - for (const [pkg, { typesFile, fromFile, spec }] of seen) { + for (const [pkg, { typesFile, fromFile, spec, owner }] of seen) { judged?.add(pkg); // A package without a root entry (no "main", "exports" or index file) // can still serve the subpath the program imports. @@ -175,11 +246,23 @@ function detectAutoPackages( package: pkg, status: "refused", detail: mode === "lib" ? reason : `auto: ${reason}`, + // Naming a package bypasses the declaration and shipped-JS + // heuristics; it cannot supply JavaScript that does not resolve. + ...(mode === "lib" && jsEntry !== null ? { hint: npmStaticOptInHint(pkg, owner) } : {}), }); } return chosen; } +/** How to compile a package the declaration bar refused anyway: name it, + * or (for a dependency of an opted-in package) opt in that package's + * whole dependency closure. */ +function npmStaticOptInHint(pkg: string, importer: string | null): string { + return importer === null + ? `to compile it anyway, name it with --npm-static=${pkg} (or --npm-static=${pkg}... to include its dependencies)` + : `to compile it anyway, name it with --npm-static=${pkg}, or use --npm-static=${importer}... to include all of ${importer}'s dependencies`; +} + /** The opted-in packages a consumer-anchored tsc message NAMES: module * specifiers in `Module '"spec"'` phrasings, and resolved file paths in * `import("…")` type spellings — the two ways the checker points at an @@ -227,9 +310,23 @@ export function runFrontend( // selections bypass eligibility heuristics, but never compilation checks. const explicit: readonly string[] = npmStatic !== undefined && typeof npmStatic !== "string" ? npmStatic : []; - return loadFrontend(entryPath, trackedLoader, "lib", externalTypes, [ - ...new Set([...libraryNpmStatic, ...explicit]), - ]); + // A trailing "..." (`--npm-static=fs-extra...`) selects the package and + // its whole dependency closure, which the library fixpoint discovers. + const transitive = new Set(); + const names = [...libraryNpmStatic, ...explicit].map((name) => { + if (!name.endsWith("...") || name.length === 3) return name; + const base = name.slice(0, -3); + transitive.add(base); + return base; + }); + return loadFrontend( + entryPath, + trackedLoader, + "lib", + externalTypes, + [...new Set(names)], + transitive, + ); } catch (error) { for (const load of active) { // Preserve the original failure and attempt every remaining cleanup. @@ -249,6 +346,7 @@ function loadFrontend( npmStatic?: readonly string[] | "auto" | "lib", externalTypes?: Readonly>, libraryNpmStatic: readonly string[] = [], + transitive: Set = new Set(), ): Frontend { // Resolver package/workspace metadata is intentionally shared across the // several load attempts of ONE auto-detection fixpoint, but never across @@ -270,7 +368,10 @@ function loadFrontend( const scoutPreflight = checkPreflight(scout); requested = npmStatic === "lib" - ? [...libraryNpmStatic, ...detectAutoPackages(scout, statuses, "lib", judged, npmSites)] + ? [ + ...libraryNpmStatic, + ...detectAutoPackages(scout, statuses, "lib", judged, npmSites, transitive), + ] : detectAutoPackages(scout, statuses); // With no package to opt in, the scout already IS the final frontend: // same roots, resolution posture, preflight, and module order. Retain it @@ -328,7 +429,7 @@ function loadFrontend( // at least one new package for good). if (npmStatic === "lib") { for (;;) { - const grown = detectAutoPackages(load, statuses, "lib", judged, npmSites); + const grown = detectAutoPackages(load, statuses, "lib", judged, npmSites, transitive); if (grown.length === 0) break; requested = [...requested, ...grown]; load.dispose(); diff --git a/packages/compiler/src/frontend/program.ts b/packages/compiler/src/frontend/program.ts index df0cb673e..dcdaf9040 100644 --- a/packages/compiler/src/frontend/program.ts +++ b/packages/compiler/src/frontend/program.ts @@ -119,6 +119,11 @@ import { import { trackedFileExists, trackedReadFile, trackedRealpath } from "./input-tracker.js"; import { forkTargetPaths } from "./fork-target.js"; import { workerTargetPaths } from "./worker-target.js"; +import { + erasedTypeOnlyImport, + typeOnlyNamedImport, + typeOnlyReachedFiles, +} from "./type-only-imports.js"; import { moduleSourceCandidates } from "./module-source-candidates.js"; import { inferredJsDiagnosticSuppressed } from "./inferred-js-diagnostics.js"; @@ -2146,7 +2151,7 @@ function insideFunctionLike7(node: ts.Node): boolean { function nodeEsmSyntaxMarker7(sf: ts.SourceFile): ts.Node | null { for (const stmt of sf.statements) { if (ts.isImportDeclaration(stmt) && !erasedTypeOnlyImport(stmt)) return stmt; - if (ts.isExportDeclaration(stmt) && !stmt.isTypeOnly && !erasedTypeOnlyReexport(stmt)) { + if (ts.isExportDeclaration(stmt) && !stmt.isTypeOnly) { return stmt; } if (ts.isExportAssignment(stmt)) return stmt; @@ -2829,6 +2834,7 @@ export function makeCycleAdmission( const clause = e.stmt.importClause; if ( clause === undefined || + typeOnlyNamedImport(e.stmt) || (clause.name === undefined && (clause.namedBindings === undefined || (ts.isNamespaceImport(clause.namedBindings) && @@ -4299,21 +4305,41 @@ function preflight7(load: LoadResult): { npmStaticPackageOfPath(sf.fileName) !== null) && !unselectedPackageJsFile(sf.fileName), ); + // Files reached only through `import type` / `export type … from` are + // checker input, never program modules: Node does not load them, so + // their imports and syntax must not fence the program. + const resolveRuntimeEdge = ( + sf: ts.SourceFile, + spec: string, + resolutionKind?: "import" | "require", + ): ts.SourceFile | null => + resolveImport7(program, sf, spec, resolutionKind) ?? + npmStaticDepSf7(program, sf, spec, resolutionKind); + const typeOnlyFiles = typeOnlyReachedFiles( + program, + entry, + programFiles, + () => [ + ...forkTargetPaths(program, program.getImplementationSourceFiles()), + ...workerTargetPaths(program, program.getImplementationSourceFiles()), + ], + resolveRuntimeEdge, + ); + const runtimeFiles = + typeOnlyFiles.size === 0 ? programFiles : programFiles.filter((sf) => !typeOnlyFiles.has(sf)); const userFiles = npmStaticActive() ? planNpmStaticReexports( program, entry, - programFiles, + runtimeFiles, [ ...createRequireProgramRoots7(program), ...forkTargetPaths(program, program.getImplementationSourceFiles()), ...workerTargetPaths(program, program.getImplementationSourceFiles()), ], - (sf, spec, resolutionKind) => - resolveImport7(program, sf, spec, resolutionKind) ?? - npmStaticDepSf7(program, sf, spec, resolutionKind), + resolveRuntimeEdge, ) - : programFiles; + : runtimeFiles; program.getTypeChecker().prefetchSourceFileStructures(userFiles); // Unselected node_modules JS is not native program source even when @@ -4438,8 +4464,7 @@ function preflight7(load: LoadResult): { continue; } if (ts.isExportDeclaration(stmt)) { - if (stmt.isTypeOnly || erasedTypeOnlyReexport(stmt) || isPrunedNpmReexport(program, stmt)) - continue; + if (stmt.isTypeOnly || isPrunedNpmReexport(program, stmt)) continue; if (!stmt.moduleSpecifier) continue; const fromSpec = ts.isStringLiteral(stmt.moduleSpecifier) ? stmt.moduleSpecifier.text : ""; if (load.externalTypes.has(fromSpec)) { @@ -5708,42 +5733,6 @@ export function isCheckedJsSourceFile(sf: ts.SourceFile, projectCheckJs: boolean return checked; } -/** `import type …` declarations and named clauses whose every element is - * `type`-qualified are ERASED by tsc at emit — Node's module graph has no - * edge there, so module order, cycle detection, init calls, and the - * startup-refusal probe must not count them either (zod v4's core modules - * are "cyclic" only through `import type * as ns` namespaces — 27 phantom - * SC1016 sites against 1 real value-level cycle; and a type-only import - * of a module Node cannot resolve is NOT a startup crash — Node never - * resolves it). Originated as the provenance-sources pilot's flag-gated - * prototype (SCRIPTC_TYPE_ONLY_EDGES), promoted to always-on with the - * full-suite differential green. Side-effect imports (no clause) stay - * real edges. */ -function erasedTypeOnlyImport(stmt: ts.ImportDeclaration): boolean { - const clause = stmt.importClause; - if (clause === undefined) return false; // side-effect import: a real edge - if (clause.phaseModifier === ts.SyntaxKind.TypeKeyword) return true; - return ( - clause.name === undefined && - clause.namedBindings !== undefined && - ts.isNamedImports(clause.namedBindings) && - clause.namedBindings.elements.length > 0 && - clause.namedBindings.elements.every((e) => e.isTypeOnly) - ); -} - -/** The re-export twin: `export { type X } from "./m"` with every element - * `type`-qualified erases at emit exactly like `export type { X } from` - * (which stmt.isTypeOnly already covers) — no module edge. */ -function erasedTypeOnlyReexport(stmt: ts.ExportDeclaration): boolean { - return ( - stmt.exportClause !== undefined && - ts.isNamedExports(stmt.exportClause) && - stmt.exportClause.elements.length > 0 && - stmt.exportClause.elements.every((e) => e.isTypeOnly) - ); -} - export function orderedImportsOf( program: ts.Program, sf: ts.SourceFile, @@ -5751,11 +5740,7 @@ export function orderedImportsOf( const out: { stmt: ts.Statement; dep: ts.SourceFile | null }[] = []; for (const stmt of sf.statements) { if (ts.isExportDeclaration(stmt) && isPrunedNpmReexport(program, stmt)) continue; - if ( - ts.isExportDeclaration(stmt) && - (stmt.isTypeOnly || erasedTypeOnlyReexport(stmt) || !stmt.moduleSpecifier) - ) - continue; + if (ts.isExportDeclaration(stmt) && (stmt.isTypeOnly || !stmt.moduleSpecifier)) continue; if (!ts.isImportDeclaration(stmt) && !ts.isExportDeclaration(stmt)) continue; if (ts.isImportDeclaration(stmt) && erasedTypeOnlyImport(stmt)) continue; if (!stmt.moduleSpecifier || !ts.isStringLiteral(stmt.moduleSpecifier)) continue; diff --git a/packages/compiler/src/frontend/resolve.test.ts b/packages/compiler/src/frontend/resolve.test.ts index 0d135b3c2..0dd09e73d 100644 --- a/packages/compiler/src/frontend/resolve.test.ts +++ b/packages/compiler/src/frontend/resolve.test.ts @@ -11,6 +11,7 @@ import { setProjectPathMappings, setProjectRealm, } from "./resolve.js"; +import { setNpmStaticPackages } from "./npm-static.js"; const fixturesRoot = join(import.meta.dirname, "../../../..", "tests/fixtures"); @@ -60,3 +61,56 @@ test("source-only workspace packages are project modules while JavaScript packag clearResolveCaches(); } }); + +test("package imports inside a --npm-static package answer the shipped JavaScript", async () => { + // chalk 5's shape: a declaration twin beside every runtime file, and a + // `{ node, default }` imports entry whose node target is what Node runs. + const dir = realpathSync(await mkdtemp(join(tmpdir(), "scriptc-resolve-imports-"))); + try { + const pkg = join(dir, "node_modules", "colors"); + const vendor = join(pkg, "source", "vendor"); + await mkdir(join(vendor, "styles"), { recursive: true }); + await mkdir(join(vendor, "detect"), { recursive: true }); + await Promise.all([ + writeFile( + join(pkg, "package.json"), + JSON.stringify({ + name: "colors", + version: "1.0.0", + type: "module", + exports: "./source/index.js", + imports: { + "#styles": "./source/vendor/styles/index.js", + "#detect": { + node: "./source/vendor/detect/index.js", + default: "./source/vendor/detect/browser.js", + }, + }, + }), + ), + writeFile(join(pkg, "source", "index.js"), 'import styles from "#styles";\n'), + writeFile(join(pkg, "source", "index.d.ts"), "export {};\n"), + writeFile(join(vendor, "styles", "index.js"), "export default {};\n"), + writeFile(join(vendor, "styles", "index.d.ts"), "export {};\n"), + writeFile(join(vendor, "detect", "index.js"), "export default 1;\n"), + writeFile(join(vendor, "detect", "index.d.ts"), "export {};\n"), + writeFile(join(vendor, "detect", "browser.js"), "export default 0;\n"), + writeFile(join(vendor, "detect", "browser.d.ts"), "export {};\n"), + ]); + const from = join(pkg, "source", "index.js"); + + clearResolveCaches(); + setNpmStaticPackages([]); + expect(resolveProjectModule(from, "#styles")).toBe(join(vendor, "styles", "index.d.ts")); + + clearResolveCaches(); + setNpmStaticPackages(["colors"]); + expect(resolveProjectModule(from, "#styles")).toBe(join(vendor, "styles", "index.js")); + expect(resolveProjectModule(from, "#detect")).toBe(join(vendor, "detect", "index.js")); + expect(resolveProjectModule(from, "#missing")).toBeNull(); + } finally { + setNpmStaticPackages([]); + clearResolveCaches(); + await rm(dir, { recursive: true, force: true }); + } +}); diff --git a/packages/compiler/src/frontend/resolve.ts b/packages/compiler/src/frontend/resolve.ts index db440c199..463400475 100644 --- a/packages/compiler/src/frontend/resolve.ts +++ b/packages/compiler/src/frontend/resolve.ts @@ -617,6 +617,18 @@ export function resolveProjectImport(fromFile: string, specifier: string): strin // Node's validity rule: only the bare "#" is never resolvable ("#/..." // matches "#/*"-pattern keys on current Node). if (specifier === "#") return null; + // Inside an opted-in --npm-static package, an imports entry resolves + // the way the tsgo host's fs shadow does: the transformed document + // (the "node" condition hoisted), the runtime JS conditions, and the + // shipped JS rather than a declaration twin (see resolveRelativeModule). + if (npmStaticPackageOfPath(resolve(fromFile)) !== null) { + const transformed = JSON.parse(JSON.stringify(pkg)) as PkgJson & Record; + npmStaticTransformPkgJson(transformed); + const jsTarget = resolvePackageImports(transformed.imports, specifier, JS_ONLY_CONDITIONS); + if (jsTarget === null || !jsTarget.startsWith("./")) return null; + const jsPath = join(pkgDir, jsTarget); + return normalizeResolvedAnswer(loadAsJsFile(jsPath) ?? loadAsJsDirectory(jsPath)); + } target = resolvePackageImports(pkg.imports, specifier, EXPORT_CONDITIONS); // An imports target that is not "./"-relative is PACKAGE_RESOLVE'd as // a bare specifier (Node's PACKAGE_IMPORTS_EXPORTS_RESOLVE): the diff --git a/packages/compiler/src/frontend/tier-analysis.ts b/packages/compiler/src/frontend/tier-analysis.ts new file mode 100644 index 000000000..ded7c5225 --- /dev/null +++ b/packages/compiler/src/frontend/tier-analysis.ts @@ -0,0 +1,629 @@ +/* Tier assignment, report-only (two-tier design, stage S2). + * + * The design assigns every binding, field, function instance, class and + * module a tier. Static is today's compiler; dynamic code operates on + * dynamic values. Demotion from static to dynamic is monotone and is seeded + * by `any`, untyped JavaScript, types and operations without a static + * lowering, reflection a static layout cannot answer, and dynamic modules + * (DESIGN.md §6). + * + * Stage S2 computes what WOULD be demoted without demoting anything: + * + * 1. Existing dynamic code. Functions whose bodies already use the `dyn` + * vocabulary are the dynamic tier today (ir/tiers.ts assigns them). The + * source statements whose lowering performs a dynamic operation are the + * coverage report's statementsDynamic. + * 2. Demotion requests. Every refusal or runtime fence the lowerer commits + * (today's poison sites: the SC1xxx/SC2xxx families) and every refused + * import or package becomes a request, attributed to the unit whose + * static representation could not express the construct. The lowerer + * still refuses or fences exactly as before; the requests are recorded + * from the committed diagnostics, so speculative probes that roll back + * record nothing. Under `--tier=static-only` every request stays today's + * refusal; stage S6 turns requests into demotions in the default mode. + * 3. Demoted units. A request demotes its unit, and the function instance + * containing the site turns dynamic (it would hold dynamic operations, + * invariant I1). Units are deduplicated; each keeps its requests, which + * is the chain `--explain-dynamic` prints. + * + * Attribution is syntactic and approximate: the site's innermost node, the + * operand it acts on, and that operand's declaration. Stage S6 replaces it + * with requests the failing lowering names exactly. */ +import type { ScrDiagnostic } from "../diagnostics/diagnostic.js"; +import type { IrFunction, SrcLoc } from "../ir/ir.js"; +import { dynamicOperationSites, type DynamicFunction } from "../ir/tiers.js"; +import * as ts from "./ts7/adapter.js"; + +/** `--tier`: "auto" (the default) will demote; "static-only" keeps every + * demotion request a refusal (design invariant I2). Identical in S2. */ +export type TierMode = "auto" | "static-only"; + +export const TIER_MODES: readonly TierMode[] = ["auto", "static-only"]; + +export type DemotionUnitKind = "binding" | "field" | "function" | "class" | "module"; + +export interface DemotionUnit { + kind: DemotionUnitKind; + /** Display name: `x`, `Point.x`, `parse`, `Parser.next`, `Point`, or the + * module's file path. */ + name: string; + /** Bindings: the function declaring a local, absent for module scope. */ + owner?: string; + /** The unit's declaration (the site itself when none is known). */ + loc: SrcLoc; +} + +/** Why a unit is demoted (DESIGN.md §6.2): a type with no static + * representation, an operation with no static lowering, reflection on a + * class, a use of a binding whose declaration was refused, a module or + * package that cannot be static, or a construct excluded by policy (no + * demotion: it stays a refusal). */ +export type DemotionSeed = "type" | "operation" | "reflection" | "cascade" | "module" | "excluded"; + +export type DemotionScope = "reached" | "deferred" | "unreached"; + +export interface DemotionRequest { + unit: DemotionUnit; + /** The function instance (or module top level) containing the site. */ + enclosing: DemotionUnit; + seed: DemotionSeed; + scope: DemotionScope; + code: string; + message: string; + loc: SrcLoc; +} + +export interface DemotedUnit extends DemotionUnit { + /** Requests naming this unit directly. */ + requests: number; + /** Requests whose sites this function instance or module contains. */ + sites: number; +} + +export interface TierDynamicFunction extends DynamicFunction { + /** Source statements with a dynamic operation in this function. */ + statements: number; +} + +export interface TierReport { + mode: TierMode; + /** Reached source statements that perform a dynamic operation. */ + statementsDynamic: number; + dynamicFunctions: TierDynamicFunction[]; + requests: DemotionRequest[]; + /** Units the requests would demote (reached and deferred scopes), + * deduplicated, by kind then source order. */ + units: DemotedUnit[]; +} + +/* ── statements of dynamic functions ─────────────────────────────────── */ + +function isClassLike(node: ts.Node): boolean { + return ts.isClassDeclaration(node) || ts.isClassExpression(node); +} + +/** The innermost node spanning exactly or containing [start, end). */ +function nodeAt(sf: ts.SourceFile, start: number, end: number): ts.Node { + let node: ts.Node = sf; + for (;;) { + let next: ts.Node | undefined; + node.forEachChild((child) => { + if (child.getStart(sf) <= start && end <= child.getEnd()) { + next = child; + return true; + } + return undefined; + }); + if (next === undefined) return node; + node = next; + } +} + +/** The source statement an IR location belongs to: the innermost + * statement containing it, or an arrow function's expression body. Null + * for synthetic locations. */ +function statementAt(program: ts.Program, loc: SrcLoc): ts.Node | null { + if (loc.start === 0 && loc.end === 0) return null; + const sf = program.getSourceFile(loc.file); + if (sf === undefined) return null; + let node: ts.Node | undefined = nodeAt(sf, loc.start, loc.end); + while (node !== undefined && !ts.isSourceFile(node)) { + const parent: ts.Node | undefined = node.parent; + if (parent !== undefined && ts.isArrowFunction(parent) && parent.body === node) return node; + if (ts.isStatement(node) && !ts.isBlock(node)) return node; + node = parent; + } + return null; +} + +/** Dynamic statements are the source statements whose lowering performs a + * dynamic operation (in the function itself or in a generated helper it + * calls). They count once however many functions and operations they + * involve. A function's tier is the unit of I1, but static operations + * inside a dynamic function stay static, so the statement is the honest + * measure of how much code is dynamic. */ +function dynamicFunctionReports( + program: ts.Program, + bodies: readonly IrFunction[], +): { functions: TierDynamicFunction[]; statements: number } { + const byLoc = new Map(); + const all = new Set(); + const functions: TierDynamicFunction[] = []; + for (const fn of bodies) { + const sites = dynamicOperationSites(fn); + if (sites.length === 0) continue; + const own = new Set(); + for (const site of sites) { + const key = `${site.loc.file}\0${site.loc.start}\0${site.loc.end}`; + let statement = byLoc.get(key); + if (statement === undefined) { + statement = statementAt(program, site.loc); + byLoc.set(key, statement); + } + if (statement === null) continue; + own.add(statement); + all.add(statement); + } + functions.push({ + name: fn.name, + loc: fn.loc, + operation: sites[0]!.operation, + operationLoc: sites[0]!.loc, + statements: own.size, + }); + } + return { functions, statements: all.size }; +} + +/* ── demotion requests ───────────────────────────────────────────────── */ + +const TYPE_CODES = new Set([ + "SC2001", + "SC2002", + "SC2003", + "SC2005", + "SC2006", + "SC2007", + "SC2008", + "SC2009", + "SC2011", + "SC1063", + "SC1100", + "SC1101", +]); +const MODULE_CODES = new Set([ + "SC1010", + "SC1012", + "SC1013", + "SC1014", + "SC1015", + "SC1016", + "SC2013", +]); +/** Runtime code generation (`eval`, `new Function`): excluded by policy, + * since the dynamic tier compiles ahead of time (DESIGN.md §12 class X). */ +const EXCLUDED_CODES = new Set(["SC2010"]); +const EXCLUDED_SURFACE = /^'(?:new Function|Function|eval)'/; + +/** Refusals and fences that name a static-tier gap. Type errors, + * environment and target refusals, divergences and internal errors are + * not demotion requests. */ +export function isDemotionCode(code: string): boolean { + return /^SC[12]\d{3}$/.test(code); +} + +function isFunctionLikeOrFile(node: ts.Node): boolean { + return ( + ts.isSourceFile(node) || + ts.isFunctionLikeDeclaration(node) || + ts.isClassStaticBlockDeclaration(node) + ); +} + +function nameText(name: ts.Node | undefined): string | null { + if (name === undefined) return null; + if (ts.isIdentifier(name) || ts.isPrivateIdentifier(name)) return name.text; + if (ts.isStringLiteral(name) || ts.isNumericLiteral(name)) return name.text; + return null; +} + +function className(node: ts.Node | undefined): string { + if (node === undefined) return ""; + if ((ts.isClassDeclaration(node) || ts.isClassExpression(node)) && node.name !== undefined) + return node.name.text; + const parent = node.parent; + if (parent !== undefined && ts.isVariableDeclaration(parent) && ts.isIdentifier(parent.name)) + return parent.name.text; + return ""; +} + +/** The display name of a function-like node, as a user would look it up. */ +function functionName(node: ts.Node): string { + if (ts.isSourceFile(node)) return ""; + if (ts.isClassStaticBlockDeclaration(node)) return `${className(node.parent)}.`; + if (ts.isConstructorDeclaration(node)) return `${className(node.parent)}.constructor`; + const own = ts.isFunctionLikeDeclaration(node) ? nameText(node.name) : null; + const parent = node.parent; + if (parent !== undefined && isClassLike(parent) && own !== null) + return `${className(parent)}.${own}`; + if (own !== null) return own; + if (parent !== undefined) { + if (ts.isVariableDeclaration(parent) || ts.isPropertyAssignment(parent)) { + const bound = nameText(parent.name); + if (bound !== null) return bound; + } + if ( + ts.isPropertyDeclaration(parent) && + parent.parent !== undefined && + isClassLike(parent.parent) + ) { + const field = nameText(parent.name); + if (field !== null) return `${className(parent.parent)}.${field}`; + } + if (ts.isBinaryExpression(parent) && parent.right === node) { + const left = parent.left; + if (ts.isIdentifier(left)) return left.text; + if (ts.isPropertyAccessExpression(left)) return left.getText(); + } + } + const sf = node.getSourceFile(); + const pos = ts.getLineAndCharacterOfPosition(sf, node.getStart(sf)); + return ``; +} + +function locOfNode(node: ts.Node): SrcLoc { + const sf = node.getSourceFile(); + return { file: sf.fileName, start: node.getStart(sf), end: node.getEnd() }; +} + +function enclosingFunction(node: ts.Node): ts.Node { + let current: ts.Node | undefined = node.parent; + while (current !== undefined && !isFunctionLikeOrFile(current)) current = current.parent; + return current ?? node.getSourceFile(); +} + +function functionUnit(node: ts.Node, instance: string | null): DemotionUnit { + // Top-level code runs in the module's initializer function. + if (ts.isSourceFile(node)) + return { + kind: "function", + name: "", + loc: { file: node.fileName, start: 0, end: 0 }, + }; + const name = functionName(node); + return { + kind: "function", + name: instance === null ? name : `${name} ${instance}`, + loc: locOfNode(node), + }; +} + +/** Generic instances append their instantiation context to diagnostics + * (Lowerer.pushDiag: ` (instantiating 'f' with )`); the instance, not + * the source function, is the unit. Returns the type-argument text. */ +function instanceOf(message: string): string | null { + const at = message.lastIndexOf(" (instantiating "); + if (at < 0 || !message.endsWith(")")) return null; + const context = message.slice(at + 2, -1); + const withAt = context.indexOf(" with "); + return withAt < 0 ? context : context.slice(withAt + 6); +} + +/** Strip wrappers and walk receivers down to the value an operation acts + * on: `a.b.c(…)` → `a`, `(x as T)[k]` → `x`. Records whether the chain + * passes through `.prototype`. */ +function rootOf(expr: ts.Node): { root: ts.Node; viaPrototype: boolean } { + let node = expr; + let viaPrototype = false; + for (;;) { + if ( + ts.isParenthesizedExpression(node) || + ts.isAsExpression(node) || + ts.isNonNullExpression(node) || + ts.isAssertionExpression(node) || + ts.isSatisfiesExpression(node) || + ts.isSpreadElement(node) || + ts.isAwaitExpression(node) + ) { + node = node.expression!; + } else if (ts.isPropertyAccessExpression(node)) { + if (node.name.text === "prototype") viaPrototype = true; + node = node.expression; + } else if (ts.isElementAccessExpression(node) || ts.isCallExpression(node)) { + node = node.expression; + } else if ( + ts.isDeleteExpression(node) || + ts.isTypeOfExpression(node) || + ts.isVoidExpression(node) + ) { + node = node.expression; + } else { + return { root: node, viaPrototype }; + } + } +} + +const REFLECTION_NAMESPACES = new Set(["Object", "Reflect"]); + +/** The expression an operation at `node` acts on. Reflection calls + * (`Object.defineProperty(x, …)`, `Reflect.set(x, …)`) act on their first + * argument; `new C()` and `x instanceof C` on the class value. */ +function subjectOf(node: ts.Node): ts.Node { + let current = node; + // A refused callee or member name: the whole call or access is the site. + const parent = current.parent; + if (parent !== undefined && ts.isPropertyAccessExpression(parent) && parent.name === current) + current = parent; + if (ts.isCallExpression(current)) { + const callee = current.expression; + if ( + ts.isPropertyAccessExpression(callee) && + ts.isIdentifier(callee.expression) && + REFLECTION_NAMESPACES.has(callee.expression.text) && + current.arguments.length > 0 + ) { + return current.arguments[0]!; + } + return callee; + } + if (ts.isNewExpression(current)) return current.expression; + if (ts.isBinaryExpression(current)) { + if (current.operatorToken.kind === ts.SyntaxKind.InstanceOfKeyword) return current.right; + if (current.operatorToken.kind === ts.SyntaxKind.EqualsToken) return current.left; + } + return current; +} + +/** The declaration owning a type-refused node: the binding, parameter, + * field or function whose declared type has no static representation. */ +function typeOwnerOf(node: ts.Node): ts.Node | null { + let current: ts.Node | undefined = node; + while (current !== undefined && !ts.isSourceFile(current)) { + if ( + ts.isVariableDeclaration(current) || + ts.isParameter(current) || + ts.isBindingElement(current) || + ts.isPropertyDeclaration(current) || + ts.isPropertySignature(current) || + ts.isCatchClause(current) + ) { + return current; + } + if (ts.isFunctionLikeDeclaration(current) || isClassLike(current)) return current; + if (ts.isStatement(current)) return null; + current = current.parent; + } + return null; +} + +function isModuleObjectName(node: ts.Node): boolean { + return ts.isIdentifier(node) && (node.text === "exports" || node.text === "module"); +} + +class UnitResolver { + constructor(private readonly checker: ts.TypeChecker) {} + + /** The unit a declaration node denotes. */ + declarationUnit(decl: ts.Node): DemotionUnit | null { + if (isClassLike(decl)) return { kind: "class", name: className(decl), loc: locOfNode(decl) }; + if (ts.isFunctionLikeDeclaration(decl)) return functionUnit(decl, null); + if (ts.isPropertyDeclaration(decl) || ts.isPropertySignature(decl)) { + const field = nameText(decl.name) ?? ""; + const owner = decl.parent; + const ownerName = owner !== undefined && isClassLike(owner) ? className(owner) : null; + return { + kind: ownerName === null ? "binding" : "field", + name: ownerName === null ? field : `${ownerName}.${field}`, + loc: locOfNode(decl), + }; + } + if (ts.isCatchClause(decl)) { + const binding = decl.variableDeclaration; + if (binding === undefined) return null; + return this.declarationUnit(binding); + } + if (ts.isVariableDeclaration(decl) || ts.isParameter(decl) || ts.isBindingElement(decl)) { + const name = nameText(decl.name) ?? decl.name?.getText() ?? ""; + const scope = enclosingFunction(decl); + const unit: DemotionUnit = { kind: "binding", name, loc: locOfNode(decl) }; + if (ts.isParameter(decl) && ts.isFunctionLikeDeclaration(decl.parent)) + unit.owner = functionName(decl.parent); + else if (!ts.isSourceFile(scope)) unit.owner = functionName(scope); + return unit; + } + return null; + } + + /** The unit an operand expression denotes. */ + valueUnit(expr: ts.Node): { unit: DemotionUnit; reflection: boolean } | null { + const { root, viaPrototype } = rootOf(expr); + const sf = root.getSourceFile(); + if (root.kind === ts.SyntaxKind.ThisKeyword) { + const owner = enclosingFunction(root); + if (ts.isSourceFile(owner)) + return { + unit: { kind: "module", name: sf.fileName, loc: locOfNode(sf) }, + reflection: false, + }; + const cls = owner.parent; + if (cls !== undefined && isClassLike(cls)) { + // `this.f` names a field; `this` itself is the instance. + const access = root.parent; + if ( + access !== undefined && + ts.isPropertyAccessExpression(access) && + access.expression === root + ) { + return { + unit: { + kind: "field", + name: `${className(cls)}.${access.name.text}`, + loc: locOfNode(access), + }, + reflection: false, + }; + } + return { + unit: { kind: "class", name: className(cls), loc: locOfNode(cls) }, + reflection: viaPrototype, + }; + } + return null; + } + if (!ts.isIdentifier(root)) return null; + let symbol = this.checker.getSymbolAtLocation(root); + if (symbol !== undefined && (symbol.flags & ts.SymbolFlags.Alias) !== 0) + symbol = this.checker.getAliasedSymbol(symbol); + const decls = symbol === undefined ? [] : this.checker.declarationsOf(symbol); + const decl = decls.find((d) => !d.getSourceFile().isDeclarationFile); + if (decl === undefined) { + // CommonJS `exports`/`module` as values: the module object. + if (isModuleObjectName(root)) + return { + unit: { kind: "module", name: sf.fileName, loc: locOfNode(sf) }, + reflection: false, + }; + return null; + } + if (ts.isSourceFile(decl)) + return { + unit: { kind: "module", name: decl.fileName, loc: locOfNode(decl) }, + reflection: false, + }; + const unit = this.declarationUnit(decl); + if (unit === null) return null; + // `C.prototype.m = …`, `Object.defineProperty(C.prototype, …)`: the + // class (or the constructor function's prototype object). + if (viaPrototype && (unit.kind === "class" || unit.kind === "function")) + return { unit: { ...unit, kind: "class" }, reflection: true }; + return { unit, reflection: unit.kind === "class" }; + } +} + +function seedOf(code: string, message: string): DemotionSeed { + if (EXCLUDED_CODES.has(code) || EXCLUDED_SURFACE.test(message)) return "excluded"; + if (code === "SC2004") return "cascade"; + if (MODULE_CODES.has(code)) return "module"; + if (TYPE_CODES.has(code)) return "type"; + return "operation"; +} + +/** One request for a committed refusal or fence diagnostic. */ +function requestFor( + program: ts.Program, + resolver: UnitResolver, + diag: ScrDiagnostic, + scope: DemotionScope, +): DemotionRequest | null { + if (!isDemotionCode(diag.code)) return null; + let seed = seedOf(diag.code, diag.message); + const base = { code: diag.code, message: diag.message, loc: diag.loc, scope }; + const sf = program.getSourceFile(diag.loc.file); + if (sf === undefined) { + const unit: DemotionUnit = { kind: "module", name: diag.loc.file, loc: diag.loc }; + return { ...base, unit, enclosing: unit, seed }; + } + const moduleUnit: DemotionUnit = { kind: "module", name: sf.fileName, loc: locOfNode(sf) }; + const node = nodeAt(sf, diag.loc.start, diag.loc.end); + const instance = instanceOf(diag.message); + const owner = ts.isSourceFile(node) ? sf : enclosingFunction(node); + const enclosing = functionUnit(owner, instance); + if (seed === "module" || ts.isSourceFile(node)) + return { ...base, unit: moduleUnit, enclosing, seed }; + let unit: DemotionUnit | null = null; + if (seed === "type") { + const decl = typeOwnerOf(node); + if (decl !== null) unit = resolver.declarationUnit(decl); + } + if (unit === null) { + const value = resolver.valueUnit(subjectOf(node)); + if (value !== null) { + unit = value.unit; + if (value.reflection && seed === "operation") seed = "reflection"; + } + } + return { ...base, unit: unit ?? enclosing, enclosing, seed }; +} + +function unitKey(unit: DemotionUnit): string { + return `${unit.kind}\0${unit.name}\0${unit.owner ?? ""}\0${unit.loc.file}\0${unit.loc.start}`; +} + +const KIND_ORDER: readonly DemotionUnitKind[] = ["module", "class", "field", "function", "binding"]; + +function demotedUnits(requests: readonly DemotionRequest[]): DemotedUnit[] { + const units = new Map(); + const entry = (unit: DemotionUnit): DemotedUnit => { + const key = unitKey(unit); + let found = units.get(key); + if (found === undefined) units.set(key, (found = { ...unit, requests: 0, sites: 0 })); + return found; + }; + for (const request of requests) { + if (request.scope === "unreached" || request.seed === "excluded") continue; + entry(request.unit).requests++; + entry(request.enclosing).sites++; + } + return [...units.values()].sort( + (a, b) => + KIND_ORDER.indexOf(a.kind) - KIND_ORDER.indexOf(b.kind) || + a.loc.file.localeCompare(b.loc.file) || + a.loc.start - b.loc.start || + a.name.localeCompare(b.name), + ); +} + +export interface TierAnalysisInput { + mode: TierMode; + /** The dynamic-tier IR functions of the reached program. */ + dynamicFunctions: readonly IrFunction[]; + reached: readonly ScrDiagnostic[]; + deferred: readonly ScrDiagnostic[]; + unreached: readonly ScrDiagnostic[]; +} + +/** Coverage's statementsDynamic (see dynamicFunctionReports). */ +export function dynamicStatementCount( + program: ts.Program, + dynamicFunctions: readonly IrFunction[], +): number { + return dynamicFunctionReports(program, dynamicFunctions).statements; +} + +export function analyzeTiers(program: ts.Program, input: TierAnalysisInput): TierReport { + const resolver = new UnitResolver(program.getTypeChecker()); + const requests: DemotionRequest[] = []; + const add = (diags: readonly ScrDiagnostic[], scope: DemotionScope): void => { + for (const diag of diags) { + let request: DemotionRequest | null; + try { + request = requestFor(program, resolver, diag, scope); + } catch { + // Attribution is best effort; a report never fails an analysis. + if (!isDemotionCode(diag.code)) continue; + const unit: DemotionUnit = { kind: "module", name: diag.loc.file, loc: diag.loc }; + request = { + unit, + enclosing: unit, + seed: seedOf(diag.code, diag.message), + scope, + code: diag.code, + message: diag.message, + loc: diag.loc, + }; + } + if (request !== null) requests.push(request); + } + }; + add(input.reached, "reached"); + add(input.deferred, "deferred"); + add(input.unreached, "unreached"); + const dynamic = dynamicFunctionReports(program, input.dynamicFunctions); + return { + mode: input.mode, + statementsDynamic: dynamic.statements, + dynamicFunctions: dynamic.functions, + requests, + units: demotedUnits(requests), + }; +} diff --git a/packages/compiler/src/frontend/type-mapper.ts b/packages/compiler/src/frontend/type-mapper.ts index 5ecfc60ae..7ecf44659 100644 --- a/packages/compiler/src/frontend/type-mapper.ts +++ b/packages/compiler/src/frontend/type-mapper.ts @@ -996,6 +996,10 @@ export type TypeParamTsResolver = (t: ts.Type) => ts.Type | null; * one new field here, not a new parameter at every call site. */ export interface TypeMapperCtx { checker: ts.TypeChecker; + /** JavaScript declarations whose instance or object-literal types only + * the checked-dynamic representation holds (js-dynamic-objects.ts). + * Absent when the program has none: every mapping is unchanged. */ + jsDynamicDeclaration?: (decl: ts.Node) => boolean; /** Set while mapping a generic callable member's erased type * (erasedGenericMemberType): the signature's own type parameters * resolve to `unknown` instead of refusing the mapping. */ @@ -1368,6 +1372,16 @@ function mapTypeInner(type: ts.Type, ctx: TypeMapperCtx): IrType | null { const intrinsic = type.getSymbol(); if (intrinsic?.name === "globalThis" && checker.declarationsOf(intrinsic).length === 0) return DYN; + if (ctx.jsDynamicDeclaration !== undefined && intrinsic !== undefined) { + const objectLiteral = (intrinsic.flags & ts.SymbolFlags.ObjectLiteral) !== 0; + const classInstance = + (intrinsic.flags & ts.SymbolFlags.Class) !== 0 && type.isClassOrInterface(); + if ( + (objectLiteral || classInstance) && + checker.declarationsOf(intrinsic).some((decl) => ctx.jsDynamicDeclaration!(decl)) + ) + return DYN; + } { const moduleId = ctx.moduleNamespaceId?.(type) ?? null; if (moduleId !== null) return { kind: "moduleNs", moduleId }; @@ -1687,16 +1701,38 @@ function mapTypeInner(type: ts.Type, ctx: TypeMapperCtx): IrType | null { // Declaration-only package types have no compiled implementation. // Primitive and container types retain their structural mappings; types // owned by the standard library or explicit host mappings are handled - // by their dedicated native rules. - const npmSym = widened.getAliasSymbol() ?? widened.getSymbol(); - const npmDecls = npmSym ? checker.declarationsOf(npmSym) : undefined; + // by their dedicated native rules. Pure type shapes — interfaces, type + // aliases, and object or mapped type literals with no value declaration + // merged in — describe data, not an implementation: a program value of + // such a type is one the program built itself (`import type` from a + // types-only package), so it maps structurally like a local declaration. + // Values the package itself would produce stay refused at their import. + const packageOnly = (sym: ts.Symbol | undefined): readonly ts.Node[] | null => { + const decls = sym ? checker.declarationsOf(sym) : undefined; + return decls && + decls.length > 0 && + decls.every((d) => { + const sf = d.getSourceFile(); + return sf.isDeclarationFile && !ctx.isStdlibFile(sf) && !ctx.isExternalTypeFile(sf); + }) + ? decls + : null; + }; + const pureTypeShape = (decls: readonly ts.Node[] | null): boolean => + decls === null || + decls.every( + (d) => + packageTypeFile(d.getSourceFile(), ctx) && + (ts.isInterfaceDeclaration(d) || + ts.isTypeAliasDeclaration(d) || + ts.isTypeLiteralNode(d) || + ts.isMappedTypeNode(d)), + ); + const aliasDecls = packageOnly(widened.getAliasSymbol()); + const ownDecls = packageOnly(widened.getSymbol()); if ( - npmDecls && - npmDecls.length > 0 && - npmDecls.every((d) => { - const sf = d.getSourceFile(); - return sf.isDeclarationFile && !ctx.isStdlibFile(sf) && !ctx.isExternalTypeFile(sf); - }) + (aliasDecls !== null || (widened.getAliasSymbol() === undefined && ownDecls !== null)) && + !(pureTypeShape(aliasDecls) && pureTypeShape(ownDecls)) ) { return null; } @@ -5407,6 +5443,17 @@ function isMappedShape(t: ts.Type): boolean { ); } +/** A declaration file an installed package ships (its own or an @types + * twin), outside the standard library and @types/node: the type surface + * of a package the program does not compile. Its pure type shapes — + * interfaces, aliases, type literals — describe data a program can build + * itself (`import type` from a types-only package such as type-fest). */ +function packageTypeFile(sf: ts.SourceFile, ctx: Pick): boolean { + return ( + sf.isDeclarationFile && /[\\/]node_modules[\\/]/.test(sf.fileName) && !ctx.isStdlibFile(sf) + ); +} + /** The record path's provenance fence. Declared shapes (object literals, * interfaces, type literals) must come from user code or an explicitly * mapped external type surface, never an arbitrary .d.ts — the empty @@ -5445,13 +5492,14 @@ function recordProvenanceOk(t: ts.Type, ctx: TypeMapperCtx): boolean { .every( (decl) => !decl.getSourceFile().isDeclarationFile || - ctx.isExternalTypeFile(decl.getSourceFile()), + ctx.isExternalTypeFile(decl.getSourceFile()) || + packageTypeFile(decl.getSourceFile(), ctx), ), ); } return !decls.some((d) => { const sf = d.getSourceFile(); - return sf.isDeclarationFile && !ctx.isExternalTypeFile(sf); + return sf.isDeclarationFile && !ctx.isExternalTypeFile(sf) && !packageTypeFile(sf, ctx); }); } @@ -5908,7 +5956,7 @@ function mapRecordTypeInner( computed && checker.declarationsOf(p).some((d) => { const sf = d.getSourceFile(); - return sf.isDeclarationFile && !ctx.isExternalTypeFile(sf); + return sf.isDeclarationFile && !ctx.isExternalTypeFile(sf) && !packageTypeFile(sf, ctx); }) ) { return null; @@ -6235,7 +6283,7 @@ export function describeRecordMemberBlocker(widened: ts.Type, ctx: TypeMapperCtx computed && checker.declarationsOf(p).some((d) => { const sf = d.getSourceFile(); - return sf.isDeclarationFile && !ctx.isExternalTypeFile(sf); + return sf.isDeclarationFile && !ctx.isExternalTypeFile(sf) && !packageTypeFile(sf, ctx); }) ) { return null; diff --git a/packages/compiler/src/frontend/type-only-imports.ts b/packages/compiler/src/frontend/type-only-imports.ts new file mode 100644 index 000000000..003d4ccf7 --- /dev/null +++ b/packages/compiler/src/frontend/type-only-imports.ts @@ -0,0 +1,143 @@ +/** Type-only module edges under Node's type stripping. + * + * Node runs TypeScript by erasing types in place, with the semantics of + * `verbatimModuleSyntax`: `import type …` and `export type … from` vanish, + * but `import { type A } from "./m"` and `export { type A } from "./m"` + * keep an empty clause (`import {} from "./m"`) and still load and + * evaluate `./m`. Only the erased forms are absent from Node's module + * graph. */ +import * as ts from "./ts7/adapter.js"; +import { moduleSourceCandidates } from "./module-source-candidates.js"; + +/** `import type …` (default, named, or namespace): erased by Node, so + * module order, cycle detection, init calls, package selection, and the + * startup-refusal probe must not count it (zod v4's core modules are + * "cyclic" only through `import type * as ns` namespaces, and a + * type-only import of a module Node cannot resolve is not a startup + * crash). Side-effect imports and inline `type` specifiers stay real + * edges. */ +export function erasedTypeOnlyImport(stmt: ts.ImportDeclaration): boolean { + return stmt.importClause?.phaseModifier === ts.SyntaxKind.TypeKeyword; +} + +/** `import { type A, type B } from "./m"`: a real module edge that binds + * nothing at runtime, exactly like a side-effect import. */ +export function typeOnlyNamedImport(stmt: ts.ImportDeclaration): boolean { + const clause = stmt.importClause; + return ( + clause !== undefined && + clause.phaseModifier !== ts.SyntaxKind.TypeKeyword && + clause.name === undefined && + clause.namedBindings !== undefined && + ts.isNamedImports(clause.namedBindings) && + clause.namedBindings.elements.length > 0 && + clause.namedBindings.elements.every((e) => e.isTypeOnly) + ); +} + +/** One module reference of a program file: its specifier and whether + * Node erases it. */ +interface ModuleReference { + spec: string; + erased: boolean; + kind?: "import" | "require"; +} + +function declarationReferencesOf(sf: ts.SourceFile): ModuleReference[] { + const refs: ModuleReference[] = []; + for (const stmt of sf.statements) { + if (ts.isImportDeclaration(stmt)) { + if (ts.isStringLiteral(stmt.moduleSpecifier)) + refs.push({ spec: stmt.moduleSpecifier.text, erased: erasedTypeOnlyImport(stmt) }); + } else if (ts.isExportDeclaration(stmt)) { + if (stmt.moduleSpecifier !== undefined && ts.isStringLiteral(stmt.moduleSpecifier)) + refs.push({ spec: stmt.moduleSpecifier.text, erased: stmt.isTypeOnly }); + } else if ( + ts.isImportEqualsDeclaration(stmt) && + ts.isExternalModuleReference(stmt.moduleReference) && + ts.isStringLiteralLike(stmt.moduleReference.expression) + ) { + refs.push({ + spec: stmt.moduleReference.expression.text, + erased: stmt.isTypeOnly, + kind: "require", + }); + } + } + return refs; +} + +/** Every call with a leading string literal counts as a loading edge — + * require(), import(), createRequire'd require functions, and loader + * helpers alike. Over-counting only keeps a file in the graph. */ +function callReferencesOf(program: ts.Program, sf: ts.SourceFile): ModuleReference[] { + const refs: ModuleReference[] = []; + for (const node of moduleSourceCandidates(program, sf).calls) { + const arg = node.arguments[0]; + if (arg === undefined || !ts.isStringLiteralLike(arg)) continue; + refs.push({ + spec: arg.text, + erased: false, + kind: node.expression.kind === ts.SyntaxKind.ImportKeyword ? "import" : "require", + }); + } + return refs; +} + +/** The program files Node never loads because the only references that + * reach them are erased type-only forms (directly or through other such + * files). A file referenced by nothing at all (a program root, or a file + * reached through a reference this walk does not model, such as an + * `import("./m")` type query) keeps its place, as do `extraRoots` (fork + * and worker targets). Empty when no erased reference resolves to a + * program file — the common case pays one statement scan per file. */ +export function typeOnlyReachedFiles( + program: ts.Program, + entry: ts.SourceFile, + files: readonly ts.SourceFile[], + extraRoots: () => readonly string[], + resolveEdge: ( + from: ts.SourceFile, + spec: string, + resolutionKind?: "import" | "require", + ) => ts.SourceFile | null, +): Set { + const available = new Set(files); + const declarationRefs = new Map(files.map((sf) => [sf, declarationReferencesOf(sf)] as const)); + const target = (sf: ts.SourceFile, ref: ModuleReference): ts.SourceFile | null => { + const dep = resolveEdge(sf, ref.spec, ref.kind); + return dep === null || dep === sf || !available.has(dep) ? null : dep; + }; + if ( + !files.some((sf) => + declarationRefs.get(sf)!.some((ref) => ref.erased && target(sf, ref) !== null), + ) + ) + return new Set(); + const valueEdges = new Map(); + const referenced = new Set(); + for (const sf of files) { + const deps: ts.SourceFile[] = []; + valueEdges.set(sf, deps); + for (const ref of [...declarationRefs.get(sf)!, ...callReferencesOf(program, sf)]) { + const dep = target(sf, ref); + if (dep === null) continue; + referenced.add(dep); + if (!ref.erased) deps.push(dep); + } + } + const roots = files.filter((sf) => sf === entry || !referenced.has(sf)); + for (const path of extraRoots()) { + const sf = program.getSourceFile(path); + if (sf !== undefined && available.has(sf)) roots.push(sf); + } + const reached = new Set(); + const queue = [...roots]; + while (queue.length > 0) { + const sf = queue.pop()!; + if (reached.has(sf)) continue; + reached.add(sf); + for (const dep of valueEdges.get(sf) ?? []) if (!reached.has(dep)) queue.push(dep); + } + return new Set(files.filter((sf) => !reached.has(sf))); +} diff --git a/packages/compiler/src/index.ts b/packages/compiler/src/index.ts index c3009665a..56b787e18 100644 --- a/packages/compiler/src/index.ts +++ b/packages/compiler/src/index.ts @@ -172,11 +172,22 @@ export { } from "./diagnostics/render.js"; export { renderCoverage, + renderExplainDynamic, coverageEnvelope, coveragePasses, type CoverageInput, type CoverageFailOn, } from "./coverage/report.js"; +export { + TIER_MODES, + type DemotedUnit, + type DemotionRequest, + type DemotionSeed, + type DemotionUnit, + type DemotionUnitKind, + type TierMode, + type TierReport, +} from "./frontend/tier-analysis.js"; export { buildEnvelope, DIAGNOSTICS_SCHEMA_VERSION, @@ -208,6 +219,13 @@ export { } from "./compat/fetch-profile.js"; export { LIB_FN_SIGS } from "./ir/builtin-signatures.js"; export { validateModule } from "./ir/validate.js"; +export { + assignLegacyTiers, + dynamicOperationOf, + functionTier, + type DynamicFunction, + type IrTier, +} from "./ir/tiers.js"; export { deserializeModule, deserializeModuleBytes, diff --git a/packages/compiler/src/ir/array-length-stability.test.ts b/packages/compiler/src/ir/array-length-stability.test.ts new file mode 100644 index 000000000..44e461ca2 --- /dev/null +++ b/packages/compiler/src/ir/array-length-stability.test.ts @@ -0,0 +1,141 @@ +import { expect, test } from "vitest"; +import { findUnaliasedLocalArrays, preservesArrayLength } from "./array-length-stability.js"; +import { + BOOL, + F64, + arrayOf, + type IrExpr, + type IrFunction, + type IrLocal, + type IrStmt, +} from "./ir.js"; + +const loc = { file: "array-length-stability.ts", start: 0, end: 0 }; +const ARR = arrayOf(F64); +const num = (value: number): IrExpr => ({ kind: "numLit", value, type: F64, loc }); +const ref = (localId: string): IrExpr => ({ kind: "varRef", localId, type: F64, loc }); +const arr = (localId: string): IrExpr => ({ kind: "varRef", localId, type: ARR, loc }); +const method = ( + receiver: string, + name: Extract["method"], + args: IrExpr[] = [], +): IrExpr => ({ + kind: "arrIntrinsic", + method: name, + receiver: arr(receiver), + args, + type: F64, + loc, +}); +const length = (receiver: string): IrExpr => method(receiver, "length"); +const read = (receiver: string, index: IrExpr): IrExpr => method(receiver, "getNumber", [index]); +const plus = (left: IrExpr, right: IrExpr): IrExpr => ({ + kind: "bin", + op: "+", + left, + right, + type: F64, + loc, +}); +const stmt = (expr: IrExpr): IrStmt => ({ kind: "exprStmt", expr, loc }); +const set = (receiver: string, index: IrExpr, value: IrExpr): IrStmt => ({ + kind: "arraySet", + arr: arr(receiver), + index, + value, + loc, +}); +const cond: IrExpr = { kind: "bin", op: "<", left: ref("i"), right: length("a"), type: BOOL, loc }; +const sum: IrStmt = { + kind: "assign", + localId: "s", + value: plus(ref("s"), read("a", ref("i"))), + loc, +}; +const locals = new Map([ + ["i", { id: "i", name: "i", type: F64, mutable: true }], + ["s", { id: "s", name: "s", type: F64, mutable: true }], + ["a", { id: "a", name: "a", type: ARR, mutable: false }], + ["b", { id: "b", name: "b", type: ARR, mutable: false }], + ["out", { id: "out", name: "out", type: ARR, mutable: false }], +]); +const preserves = ( + body: IrStmt[], + inRange: string | null = "i", + unaliased: ReadonlySet = new Set(), + scope = locals, +): boolean => preservesArrayLength([cond, ...body], { receiver: "a", inRange }, scope, unaliased); + +test("reads, local arithmetic and element writes at the in-range counter keep the length", () => { + expect(preserves([sum])).toBe(true); + expect(preserves([set("a", ref("i"), num(1)), set("b", ref("i"), read("a", ref("i")))])).toBe( + true, + ); + expect(preserves([stmt(method("a", "indexOf", [num(3)]))])).toBe(true); +}); + +test("writes past the counter, without an in-range counter, or to the counter can grow it", () => { + expect(preserves([set("a", plus(ref("i"), num(1)), num(1))])).toBe(false); + expect(preserves([set("b", ref("s"), num(1))])).toBe(false); + expect(preserves([set("a", ref("i"), num(1))], null)).toBe(false); + expect(preserves([{ kind: "assign", localId: "i", value: num(0), loc }, sum])).toBe(false); +}); + +test("resizing methods, length stores, calls and escapes refuse the proof", () => { + for (const name of ["push", "pop", "shift", "unshift", "splice", "fill"] as const) + expect(preserves([stmt(method("a", name, [num(1)]))])).toBe(false); + // An unrelated array may alias the receiver. + expect(preserves([stmt(method("b", "push", [num(1)]))])).toBe(false); + expect(preserves([{ kind: "arraySetLength", arr: arr("a"), length: num(0), loc }])).toBe(false); + expect(preserves([{ kind: "arrayDelete", arr: arr("a"), index: ref("i"), loc }])).toBe(false); + expect(preserves([stmt({ kind: "call", callee: "f", args: [], type: F64, loc })])).toBe(false); + expect(preserves([stmt({ kind: "call", callee: "f", args: [arr("a")], type: F64, loc })])).toBe( + false, + ); + expect(preserves([{ kind: "varDecl", localId: "b", init: arr("a"), loc }])).toBe(false); + expect(preserves([{ kind: "assign", localId: "a", value: arr("b"), loc }])).toBe(false); +}); + +test("an unaliased local array other than the receiver may grow", () => { + const push = stmt(method("out", "push", [read("a", ref("i"))])); + expect(preserves([push], "i", new Set(["out"]))).toBe(true); + expect(preserves([set("out", ref("s"), num(1))], "i", new Set(["out"]))).toBe(true); + expect(preserves([push])).toBe(false); + // The receiver itself is never resized, even when no alias exists. + expect(preserves([stmt(method("a", "push", [num(1)]))], "i", new Set(["a"]))).toBe(false); +}); + +test("captured receivers have no local proof", () => { + const boxed = new Map(locals); + boxed.set("a", { ...locals.get("a")!, boxed: true }); + expect(preserves([sum], "i", new Set(), boxed)).toBe(false); +}); + +function fn(body: IrStmt[], params: string[] = []): IrFunction { + return { + loc, + name: "f", + params: params.map((localId) => ({ localId, type: ARR })), + returnType: ARR, + locals: [...locals.values()], + body, + } as IrFunction; +} +const fresh: IrExpr = { kind: "arrayLit", elems: [], type: ARR, loc }; + +test("fresh local arrays read only as receivers have no alias", () => { + const body: IrStmt[] = [ + { kind: "varDecl", localId: "out", init: fresh, loc }, + stmt(method("out", "push", [num(1)])), + { kind: "return", value: arr("out"), loc }, + ]; + expect([...findUnaliasedLocalArrays(fn(body))]).toEqual(["out"]); + // Parameters, copies into another binding, and arguments escape. + expect(findUnaliasedLocalArrays(fn(body, ["out"])).size).toBe(0); + const copied = [...body, { kind: "varDecl", localId: "b", init: arr("out"), loc } as IrStmt]; + expect(findUnaliasedLocalArrays(fn(copied)).has("out")).toBe(false); + const passed = [...body, stmt({ kind: "call", callee: "g", args: [arr("out")], type: F64, loc })]; + expect(findUnaliasedLocalArrays(fn(passed)).has("out")).toBe(false); + const rebound = [...body, { kind: "assign", localId: "out", value: arr("a"), loc } as IrStmt]; + expect(findUnaliasedLocalArrays(fn(rebound)).has("out")).toBe(false); +}); diff --git a/packages/compiler/src/ir/array-length-stability.ts b/packages/compiler/src/ir/array-length-stability.ts new file mode 100644 index 000000000..2ac196d30 --- /dev/null +++ b/packages/compiler/src/ir/array-length-stability.ts @@ -0,0 +1,259 @@ +import type { IrExpr, IrFunction, IrLocal, IrStmt } from "./ir.js"; +import { isBorrowSafeMath } from "./analysis.js"; +import { byteNumberAccess } from "./byte-numbers.js"; +import { everyExprChild, everyStmtChild } from "./traverse.js"; + +/** Array methods that read the receiver without changing its length or its + * slot states and run no user code. */ +const READING_ARRAY_METHODS = new Set([ + "length", + "getNumber", + "indexEq", + "indexOf", + "includes", + "nextPresent", +]); + +/** String methods that read an immutable string and run no user code. */ +const READING_STRING_METHODS = new Set(["length", "charCodeAt", "codePointAt"]); + +/** Expressions that compute from their operands and run no user code. Their + * children are checked separately. */ +const PLAIN_EXPRESSIONS = new Set([ + "numLit", + "strLit", + "boolLit", + "unitLit", + "varRef", + "bin", + "unary", + "incDec", + "toBool", + "logical", + "ternary", + "strEq", + "strCmp", + "fieldGet", + "recordGet", + "unionNarrow", + "unionIsTag", + "unionDisc", + "unionWrap", + "upcast", + "downcast", + "instanceOf", + "nullish", + "seqExpr", +]); + +/** Statements whose own effect is a local write or control flow inside the + * loop. Their children are checked separately. */ +const PLAIN_STATEMENTS = new Set([ + "varDecl", + "assign", + "exprStmt", + "if", + "block", + "break", + "continue", + "for", + "while", + "doWhile", +]); + +/** Array methods whose result is never the receiver itself (unlike + * `reverse`, `fill` or `copyWithin`, which return it). */ +const CONTAINED_RECEIVER_METHODS = new Set([ + ...READING_ARRAY_METHODS, + "push", + "pop", + "shift", + "unshift", + "splice", + "join", + "slice", +]); + +/** Local arrays that no other binding can hold: every write of the local + * stores a fresh array literal and every read of it is the receiver of an + * element access or array method (or the value of a `return`, which ends the + * activation). No alias of such an array exists, so mutating it cannot change + * the length of an array held by any other binding. */ +export function findUnaliasedLocalArrays(fn: IrFunction): ReadonlySet { + const candidates = new Set(); + const captured = new Set( + [...(fn.captures ?? []), ...(fn.classCaptures ?? [])].map((c) => c.localId), + ); + for (const local of fn.locals) + if (local.type.kind === "array" && !local.boxed && !local.tdz && !captured.has(local.id)) + candidates.add(local.id); + for (const param of fn.params) candidates.delete(param.localId); + const written = new Set(); + const fresh = (value: IrExpr | null): boolean => + value !== null && (value.kind === "arrayLit" || value.kind === "arrayNewLen"); + const receiver = (e: IrExpr): void => { + if (e.kind !== "varRef") expr(e); + }; + const expr = (e: IrExpr): boolean => { + switch (e.kind) { + case "varRef": + candidates.delete(e.localId); + return true; + case "assignExpr": + candidates.delete(e.localId); + break; + case "arrIntrinsic": + if (!CONTAINED_RECEIVER_METHODS.has(e.method)) break; + receiver(e.receiver); + e.args.forEach(expr); + return true; + case "arrayGet": + case "arrayHas": + case "arrayState": + receiver(e.arr); + expr(e.index); + return true; + } + return everyExprChild(e, expr, stmt); + }; + const stmt = (s: IrStmt): boolean => { + switch (s.kind) { + case "varDecl": + case "assign": { + const value = s.kind === "varDecl" ? s.init : s.value; + if (fresh(value)) written.add(s.localId); + else if (value) candidates.delete(s.localId); + if (value) expr(value); + return true; + } + case "arraySet": + receiver(s.arr); + expr(s.index); + expr(s.value); + return true; + case "return": + if (s.value) receiver(s.value); + return true; + case "forOf": + case "tryCatch": + if (s.kind === "forOf") candidates.delete(s.localId); + if (s.kind === "tryCatch" && s.catchLocalId) candidates.delete(s.catchLocalId); + break; + } + return everyStmtChild(s, expr, stmt); + }; + fn.body.every(stmt); + // A binding with no visible write takes its value some other way. + for (const id of candidates) if (!written.has(id)) candidates.delete(id); + return candidates; +} + +export interface ArrayLengthLoop { + /** The local holding the array whose length must not change. */ + receiver: string; + /** A binding that holds a value below the array's length wherever the + * loop's condition, body or update reads it (the induction variable of + * `for (let i = 0; i < a.length; i++)` when the body never writes it, as + * long as the length itself has not changed). An element write at this + * index replaces an existing element or sets a non-index property. */ + inRange: string | null; +} + +/** Whether running `nodes` (a loop's condition, body and update) can never + * change the length of the array held by local `loop.receiver`. The proof is + * syntactic and conservative: every expression and statement must be one + * that runs no user code (no calls, callbacks, closures, iterators, + * suspension or exception regions); the binding is never written and only + * ever read as the receiver of an element access or a reading array method, + * so the array cannot escape to code that resizes it; and nothing in the + * nodes resizes any array that may be the same one. Element writes are + * accepted at the in-range binding (no array grows by them: an alias of the + * receiver has the same length) and anywhere in an unaliased local array. + * Array methods that resize an array are accepted only on unaliased local + * arrays other than the receiver. */ +export function preservesArrayLength( + nodes: readonly (IrExpr | IrStmt)[], + loop: ArrayLengthLoop, + locals: ReadonlyMap, + unaliased: ReadonlySet, +): boolean { + const local = locals.get(loop.receiver); + if (!local || local.type.kind !== "array" || local.boxed || local.tdz) return false; + const separate = (e: IrExpr): boolean => + e.kind === "varRef" && e.localId !== loop.receiver && unaliased.has(e.localId); + // A receiver position: the receiver binding itself is read there without + // escaping. + const receiver = (e: IrExpr): boolean => + (e.kind === "varRef" && e.localId === loop.receiver) || expr(e); + const written = (id: string): boolean => id === loop.receiver || id === loop.inRange; + const expr = (e: IrExpr): boolean => { + switch (e.kind) { + case "varRef": + return e.localId !== loop.receiver; + case "assignExpr": + case "incDec": + return !written(e.localId) && everyExprChild(e, expr, stmt); + case "arrayGet": + case "arrayHas": + case "arrayState": + return receiver(e.arr) && expr(e.index); + case "arrIntrinsic": + if (READING_ARRAY_METHODS.has(e.method)) return receiver(e.receiver) && e.args.every(expr); + return separate(e.receiver) && e.args.every(expr); + case "strIntrinsic": + return ( + READING_STRING_METHODS.has(e.method) && + e.receiver.type.kind === "string" && + expr(e.receiver) && + e.args.every(expr) + ); + case "bytesIntrinsic": { + const numeric = byteNumberAccess(e); + return ( + (numeric ? !numeric.write : e.method === "get" || e.method === "length") && + expr(e.receiver) && + e.args.every(expr) + ); + } + case "libCall": + return ( + isBorrowSafeMath(e.fn) && + e.args.every((arg) => arg.type.kind === "f64") && + e.args.every(expr) + ); + case "strConcat": + return ( + e.left.type.kind === "string" && + e.right.type.kind === "string" && + expr(e.left) && + expr(e.right) + ); + default: + return PLAIN_EXPRESSIONS.has(e.kind) && everyExprChild(e, expr, stmt); + } + }; + const stmt = (s: IrStmt): boolean => { + switch (s.kind) { + case "varDecl": + case "assign": + if (written(s.localId)) return false; + break; + case "arraySet": + return ( + (separate(s.arr) || + (s.index.kind === "varRef" && + loop.inRange !== null && + s.index.localId === loop.inRange)) && + receiver(s.arr) && + expr(s.index) && + expr(s.value) + ); + case "return": + return s.value === null || expr(s.value); + default: + if (!PLAIN_STATEMENTS.has(s.kind)) return false; + } + return everyStmtChild(s, expr, stmt); + }; + return nodes.every((node) => ("type" in node ? expr(node as IrExpr) : stmt(node as IrStmt))); +} diff --git a/packages/compiler/src/ir/builtin-effects.ts b/packages/compiler/src/ir/builtin-effects.ts index 1bb9cbd14..155c097d5 100644 --- a/packages/compiler/src/ir/builtin-effects.ts +++ b/packages/compiler/src/ir/builtin-effects.ts @@ -489,6 +489,13 @@ export const MAY_THROW_LIB_FNS: ReadonlySet = new Set([ "str.encodeUri", "qs.escape", "qs.stringify", + // node:punycode's RangeErrors (encode/toASCII only past 32-bit deltas) + // and String.fromCodePoint's for ucs2.encode. + "punycode.decode", + "punycode.encode", + "punycode.toASCII", + "punycode.toUnicode", + "punycode.ucs2Encode", // The base64 globals: atob/btoa throw the catchable DOMException // InvalidCharacterError on malformed input; the zero-argument form // always throws Node's TypeError [ERR_MISSING_ARGS]. diff --git a/packages/compiler/src/ir/builtin-signatures.ts b/packages/compiler/src/ir/builtin-signatures.ts index 780918333..8015c104a 100644 --- a/packages/compiler/src/ir/builtin-signatures.ts +++ b/packages/compiler/src/ir/builtin-signatures.ts @@ -563,6 +563,10 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ /** os.constants: the per-thread null-prototype constants object (dlopen, * errno, signals, priority, UV_UDP_REUSEADDR). Never throws. */ "os.constants": { argTypes: [], result: DYN }, + /** The deprecated node:constants object (scr_os.c): the os groups and + * fs.constants merged with the borrowed baked crypto.constants table, + * frozen, one per thread. */ + "constants.object": { argTypes: [DYN], result: DYN }, /** umask(2): arg < 0 reads without setting (umask has no read-only form * — set 0, restore); otherwise sets and answers the PREVIOUS mask. * Never throws. */ @@ -861,6 +865,19 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ "url.legacyResolve": { argTypes: [DYN, DYN, BOOL], result: STRING }, "url.domainToASCII": { argTypes: [DYN, BOOL], result: STRING }, "url.domainToUnicode": { argTypes: [DYN, BOOL], result: STRING }, + /** node:punycode (scr_url.c): punycode.js 2.1.0's string conversions; + * decode/toUnicode (and, past 32-bit deltas, encode/toASCII) throw the + * module's RangeErrors, ucs2Encode String.fromCodePoint's. warn marks the + * module's first load (scr_async_dyn.c) and reports DEP0040 then unless + * the second argument is false: at once for a static ES import, on the + * next tick for a require() call. */ + "punycode.decode": { argTypes: [STRING], result: STRING }, + "punycode.encode": { argTypes: [STRING], result: STRING }, + "punycode.toASCII": { argTypes: [STRING], result: STRING }, + "punycode.toUnicode": { argTypes: [STRING], result: STRING }, + "punycode.ucs2Decode": { argTypes: [STRING], result: arrayOf(F64) }, + "punycode.ucs2Encode": { argTypes: [arrayOf(F64)], result: STRING }, + "punycode.warn": { argTypes: [BOOL, BOOL], result: VOID }, "url.fileURLToPathBuffer": { argTypes: [DYN, DYN], result: BYTES_U8 }, "url.pathToFileURLChecked": { argTypes: [DYN, DYN], result: URL_T }, "url.fileURLToPathStr": { argTypes: [STRING], result: STRING }, @@ -1438,7 +1455,8 @@ export const LIB_FN_SIGS = defineLibFnSignatures({ }, // The suite body takes no parameters or its SuiteContext (flag 2). "test.suite": { argTypes: [STRING, F64, STRING, null, STRING, F64, F64], result: VOID }, - "test.hook": { argTypes: [F64, null, F64], result: VOID }, + // The trailing string is the hook call's "file:line:col". + "test.hook": { argTypes: [F64, null, F64, STRING], result: VOID }, "test.sub": { argTypes: [TESTCTX_T, STRING, F64, STRING, null, F64, STRING, F64, F64, F64], result: { kind: "promise", inner: VOID }, diff --git a/packages/compiler/src/ir/integer-loop-facts.ts b/packages/compiler/src/ir/integer-loop-facts.ts index 4c1a1ef1f..84fabf956 100644 --- a/packages/compiler/src/ir/integer-loop-facts.ts +++ b/packages/compiler/src/ir/integer-loop-facts.ts @@ -118,6 +118,9 @@ export function boundedIntegerLoopFacts( stable(e.receiver.localId) ) return { min: 0, max: Number.MAX_SAFE_INTEGER }; + // Every evaluation of an array length lies in its range, so the + // condition bounds each pass even when the body resizes the array. + if (isArrayLength(e)) return ARRAY_LENGTH_RANGE; return null; } let step: IntegerRange; @@ -134,12 +137,15 @@ export function boundedIntegerLoopFacts( if (step.min <= 0 || step.max > 2147483647) return result; } const limit = bound(loop.cond.right); - if (!limit || limit.max < start.min) return result; - const distance = limit.max - start.min; + // The counter is an integer, so a strict bound admits passes up to one + // below the limit. + const last = limit && (loop.cond.op === "<" ? limit.max - 1 : limit.max); + if (!limit || last === null || last < start.min) return result; + const distance = last - start.min; const iterations = Math.floor(distance / step.min) + 1; if (!Number.isSafeInteger(distance) || !Number.isSafeInteger(iterations)) return result; // Header facts also cover the failing final condition and zero iterations. - const maximum = Math.max(start.max, limit.max + step.max); + const maximum = Math.max(start.max, last + step.max); if (Number.isSafeInteger(maximum)) result.set(counter, { min: start.min, max: maximum }); for (const [id, hi] of upper) { const range = initial.get(id); @@ -154,6 +160,20 @@ export function boundedIntegerLoopFacts( return result; } +/** Every length an array can have: SCR_ARR_MAX_LENGTH (scr_array.c). */ +export const ARRAY_LENGTH_RANGE: IntegerRange = { min: 0, max: 4294967295 }; + +/** `a.length` of a typed array value (an ordinary JS array). */ +function isArrayLength(e: IrExpr): boolean { + return ( + e.kind === "arrIntrinsic" && + e.method === "length" && + e.args.length === 0 && + e.type.kind === "f64" && + e.receiver.type.kind === "array" + ); +} + /** The most UTF-16 code units a runtime string holds: SCR_STR_MAX_CAP * (scr_runtime.h) caps a string's UTF-8 bytes, and a UTF-16 length never * exceeds the UTF-8 byte length. */ @@ -241,6 +261,9 @@ export function guardedLoopCounterFacts( stable(e.receiver.localId) ) return { min: 0, max: STRING_MAX_UNITS }; + // An array length can change on any pass, but never leaves its range, + // which is all a ceiling needs. + if (isArrayLength(e)) return ARRAY_LENGTH_RANGE; return null; }; // The bounds a condition's conjuncts put on counters while it holds. diff --git a/packages/compiler/src/ir/integer-loops.test.ts b/packages/compiler/src/ir/integer-loops.test.ts index 0f7b9ae13..75968b262 100644 --- a/packages/compiler/src/ir/integer-loops.test.ts +++ b/packages/compiler/src/ir/integer-loops.test.ts @@ -208,3 +208,111 @@ test("a string read at the counter is an integer consumer", () => { s.body = [read(ref("limit"))]; expect(matchIntegerCountedForLoop(s, withText)).toBeNull(); }); + +test("a stable array length bounds a counted loop; a resizing body keeps it a condition", () => { + const ARR = arrayOf(F64); + const array = (localId: string): IrExpr => ({ kind: "varRef", localId, type: ARR, loc }); + const length: IrExpr = { + kind: "arrIntrinsic", + method: "length", + receiver: array("a"), + args: [], + type: F64, + loc, + }; + const read = (index: IrExpr): IrExpr => ({ + kind: "arrIntrinsic", + method: "getNumber", + receiver: array("a"), + args: [index], + type: F64, + loc, + }); + // for (let i = 0; i < a.length; i += 2) a[i]; + const strided = (body: IrStmt[]): IrStmt & { kind: "for" } => { + const s = counted(length); + s.update = { + kind: "assign", + localId: "i", + value: { kind: "bin", op: "+", left: ref("i"), right: number(2), type: F64, loc }, + loc, + }; + s.body = [{ kind: "exprStmt", expr: read(ref("i")), loc }, ...body]; + return s; + }; + const ranges = new Map([[length, { min: 0, max: 4294967295 }]]); + expect(matchIntegerCountedForLoop(strided([]), locals, ranges, new Set())).toMatchObject({ + localId: "i", + guarded: false, + range: { min: 0, max: 4294967294 }, + }); + // Without the alias facts there is no stability proof. + expect(matchIntegerCountedForLoop(strided([]), locals, ranges)).toBeNull(); + const push: IrStmt = { + kind: "exprStmt", + expr: { + kind: "arrIntrinsic", + method: "push", + receiver: array("a"), + args: [number(1)], + type: F64, + loc, + }, + loc, + }; + expect(matchIntegerCountedForLoop(strided([push]), locals, ranges, new Set())).toBeNull(); + // An element write at the counter stays below the length; one past it can grow the array. + const write = (index: IrExpr): IrStmt => ({ + kind: "arraySet", + arr: array("a"), + index, + value: number(0), + loc, + }); + expect( + matchIntegerCountedForLoop(strided([write(ref("i"))]), locals, ranges, new Set()), + ).not.toBeNull(); + const past = { kind: "bin", op: "+", left: ref("i"), right: number(1), type: F64, loc } as const; + expect(matchIntegerCountedForLoop(strided([write(past)]), locals, ranges, new Set())).toBeNull(); +}); + +test("a descending loop from an array's last index takes an integer counter for its reads", () => { + const ARR = arrayOf(F64); + const length: IrExpr = { + kind: "arrIntrinsic", + method: "length", + receiver: { kind: "varRef", localId: "a", type: ARR, loc }, + args: [], + type: F64, + loc, + }; + const start: IrExpr = { kind: "bin", op: "-", left: length, right: number(1), type: F64, loc }; + const s = loop(start); + s.cond = { kind: "bin", op: ">=", left: ref("i"), right: number(0), type: BOOL, loc }; + s.update = { + kind: "assign", + localId: "i", + value: { kind: "bin", op: "-", left: ref("i"), right: number(1), type: F64, loc }, + loc, + }; + s.body = [ + { + kind: "exprStmt", + expr: { + kind: "arrIntrinsic", + method: "getNumber", + receiver: { kind: "varRef", localId: "a", type: ARR, loc }, + args: [ref("i")], + type: F64, + loc, + }, + loc, + }, + ]; + const ranges = new Map([[start, { min: -1, max: 4294967294 }]]); + expect(matchIntegerCountedForLoop(s, locals, ranges)).toMatchObject({ + step: -1, + guarded: false, + range: { min: 0, max: 4294967294 }, + }); +}); diff --git a/packages/compiler/src/ir/integer-loops.ts b/packages/compiler/src/ir/integer-loops.ts index c854a99c5..3ba186376 100644 --- a/packages/compiler/src/ir/integer-loops.ts +++ b/packages/compiler/src/ir/integer-loops.ts @@ -3,6 +3,7 @@ import type { IntegerRange, IntegerRanges } from "./integer-ranges.js"; import { everyExprChild, everyStmtChild, everyStmtList } from "./traverse.js"; import { byteNumberAccess } from "./byte-numbers.js"; +import { preservesArrayLength } from "./array-length-stability.js"; /** A canonical byte loop whose induction variable is mathematically an * unsigned integer for every body entry. Backends may keep this binding in @@ -47,15 +48,17 @@ function constantStep(update: IrStmt | null, id: string): number | null { } /** Pure initial values may be evaluated again on a fallback path. Bounds - * additionally have to remain invariant throughout the loop. Array length - * is allowed only in the initializer; its mutable value is never hoisted - * from a condition. Typed-array lengths are fixed for a given receiver. */ + * additionally have to remain invariant throughout the loop. An array length + * is hoisted from a condition only when `stableLength` proves the loop never + * changes it; otherwise it is allowed only in the initializer. Typed-array + * lengths are fixed for a given receiver. */ function stableLoopValue( value: IrExpr, locals: ReadonlyMap, body: IrStmt[], invariant: boolean, counter: string, + stableLength: (receiver: string) => boolean = () => false, ): boolean { switch (value.kind) { case "numLit": @@ -77,16 +80,16 @@ function stableLoopValue( value.op === "&" || value.op === ">>>" || value.op === "|") && - stableLoopValue(value.left, locals, body, invariant, counter) && - stableLoopValue(value.right, locals, body, invariant, counter) + stableLoopValue(value.left, locals, body, invariant, counter, stableLength) && + stableLoopValue(value.right, locals, body, invariant, counter, stableLength) ); case "bytesIntrinsic": case "arrIntrinsic": return ( (value.method === "length" || (value.kind === "bytesIntrinsic" && value.method === "byteLength")) && - (value.kind === "bytesIntrinsic" || !invariant) && value.receiver.kind === "varRef" && + (value.kind === "bytesIntrinsic" || !invariant || stableLength(value.receiver.localId)) && stableLoopValue(value.receiver, locals, body, invariant, counter) ); default: @@ -103,6 +106,7 @@ export function matchIntegerCountedForLoop( stmt: IrStmt & { kind: "for" }, locals: ReadonlyMap, ranges: IntegerRanges = new Map(), + unaliased: ReadonlySet | null = null, ): IntegerCountedForLoop | null { const init = stmt.init; if (init?.kind !== "varDecl" || !init.init) return null; @@ -135,7 +139,38 @@ export function matchIntegerCountedForLoop( op = ({ "<": ">", "<=": ">=", ">": "<", ">=": "<=" } as Record)[op] ?? ""; } if (step > 0 ? op !== "<" && op !== "<=" : op !== ">" && op !== ">=") return null; - if (!stableLoopValue(limit, locals, stmt.body, true, local.id)) return null; + // An ascending counter compared below `a.length - c` holds an index below + // the length on every pass while that length is unchanged. + const belowLength = (receiver: string): boolean => { + let offset = 0, + length = limit; + if ( + length.kind === "bin" && + length.op === "-" && + length.right.kind === "numLit" && + Number.isFinite(length.right.value) + ) { + offset = length.right.value; + length = length.left; + } + return ( + step > 0 && + length.kind === "arrIntrinsic" && + length.method === "length" && + length.receiver.kind === "varRef" && + length.receiver.localId === receiver && + (op === "<" ? offset >= 0 : offset >= 1) + ); + }; + const stableLength = (receiver: string): boolean => + unaliased !== null && + preservesArrayLength( + [cond, ...stmt.body], + { receiver, inRange: belowLength(receiver) ? local.id : null }, + locals, + unaliased, + ); + if (!stableLoopValue(limit, locals, stmt.body, true, local.id, stableLength)) return null; const inclusive = op === "<=" || op === ">="; const guardLimit = step > 0 ? 2 ** 53 - (inclusive ? 1 : 0) : -(2 ** 53) + (inclusive ? 1 : 0); const boundRange = @@ -178,10 +213,19 @@ export function matchIntegerCountedForLoop( // A string read at the counter indexes with the integer directly. if (e.kind === "strIntrinsic" && e.method === "charCodeAt" && e.args[0] && counter(e.args[0])) useful = true; + // So does an array element read; only a loop that needs no versioned + // copy takes the integer counter for it. + if ( + !guarded && + ((e.kind === "arrayGet" && counter(e.index)) || + (e.kind === "arrIntrinsic" && e.method === "getNumber" && e.args[0] && counter(e.args[0]))) + ) + useful = true; return (!guarded || ++cost <= 80) && everyExprChild(e, expr, bodyStmt); }; const bodyStmt = (s: IrStmt): boolean => { if (s.kind === "bytesSet" && counter(s.index)) useful = true; + if (!guarded && s.kind === "arraySet" && counter(s.index)) useful = true; if ( guarded && (s.kind === "for" || s.kind === "forOf" || s.kind === "while" || s.kind === "doWhile") diff --git a/packages/compiler/src/ir/integer-ranges.test.ts b/packages/compiler/src/ir/integer-ranges.test.ts index e9d1797d1..8e1c97f66 100644 --- a/packages/compiler/src/ir/integer-ranges.test.ts +++ b/packages/compiler/src/ir/integer-ranges.test.ts @@ -334,7 +334,8 @@ test("filtered strided loops retain decoded values and bound secondary cursors", const ranges = analyzeIntegerRanges(f); expect(ranges.get(cursor)).toEqual({ min: 0, max: 80 }); expect(ranges.get(decoded)).toEqual({ min: 0, max: 4294967295 }); - expect(ranges.get(conditionRead)).toEqual({ min: 0, max: 124 }); + // The last pass enters at most at 99, so the failing test reads at most 123. + expect(ranges.get(conditionRead)).toEqual({ min: 0, max: 123 }); for (const mutation of ["fraction", "repeat", "overflow", "counter"] as const) { const changed = structuredClone(f); const body = (changed.body[1] as IrStmt & { kind: "for" }).body; @@ -795,6 +796,77 @@ test("a while condition on a string's length bounds a counter, and guarded incre expect(run((inside) => [observe(inside), bump, rebound]).after).toBeNull(); }); +test("an array length bounds while and for counters even when the body resizes the array", () => { + // let i = 0; while (i < items.length) { observe(i); items.push(i); i++; } observe(i); + const ARR = { kind: "array", elem: F64 } as const; + const items: IrExpr = { loc, kind: "varRef", localId: "items", type: ARR }; + const length: IrExpr = { + loc, + kind: "arrIntrinsic", + method: "length", + receiver: items, + args: [], + type: F64, + }; + const push: IrStmt = { + loc, + kind: "exprStmt", + expr: { + loc, + kind: "arrIntrinsic", + method: "push", + receiver: items, + args: [ref("i")], + type: F64, + }, + }; + const bump: IrStmt = { + loc, + kind: "exprStmt", + expr: { loc, kind: "incDec", op: "+", prefix: false, localId: "i", type: F64 }, + }; + const inside = ref("i"), + after = ref("i"); + const f = fn([ + { loc, kind: "assign", localId: "i", value: num(0) }, + { + loc, + kind: "while", + cond: condition("<", ref("i"), length), + body: [observe(inside), push, bump], + }, + observe(after), + ]); + f.locals.push( + { id: "i", name: "i", type: F64, mutable: true }, + { id: "items", name: "items", type: ARR, mutable: false }, + ); + const ranges = analyzeIntegerRanges(f); + expect(ranges.get(inside)).toEqual({ min: 0, max: 4294967294 }); + expect(ranges.get(after)).toEqual({ min: 0, max: 4294967295 }); + // A secondary cursor of `for (let j = 0; j < items.length; j++) k++` stays a uint32. + const cursor = ref("k"); + const loop: IrStmt = { + loc, + kind: "for", + init: { loc, kind: "varDecl", localId: "j", init: num(0) }, + cond: condition("<", ref("j"), length), + update: { loc, kind: "assign", localId: "j", value: bin("+", ref("j"), num(1)) }, + body: [ + push, + observe(cursor), + { loc, kind: "assign", localId: "k", value: bin("+", ref("k"), num(1)) }, + ], + }; + const g = fn([{ loc, kind: "assign", localId: "k", value: num(0) }, loop]); + g.locals.push( + { id: "j", name: "j", type: F64, mutable: true }, + { id: "k", name: "k", type: F64, mutable: true }, + { id: "items", name: "items", type: ARR, mutable: false }, + ); + expect(analyzeIntegerRanges(g).get(cursor)).toEqual({ min: 0, max: 4294967295 }); +}); + test("whole-program slot facts seed parameters, field reads and call results", () => { const field: IrExpr = { loc, diff --git a/packages/compiler/src/ir/ir.ts b/packages/compiler/src/ir/ir.ts index 5382a6735..68a4776f5 100644 --- a/packages/compiler/src/ir/ir.ts +++ b/packages/compiler/src/ir/ir.ts @@ -1656,6 +1656,14 @@ export interface IrFunction { sourceName?: string; /** Ordinary authored function declarations/expressions own a prototype. */ ownsPrototype?: true; + /** The function's execution tier (ir/tiers.ts). Absent means static: the + * function may use dynamic values only through boundary operations + * (dynFrom, dynCheck, kind tests) and moves, which the validator enforces + * (invariant I1). "dynamic" admits the dynamic operation vocabulary + * (dynKeyGet, dynCall, dyn.* builtins, ...). Lowering assigns the field + * from the body (assignLegacyTiers); stage S2 of the two-tier design + * keeps it out of serialized IR, so loaders derive it again. */ + tier?: "dynamic"; body: IrStmt[]; loc: SrcLoc; } diff --git a/packages/compiler/src/ir/runtime-features.ts b/packages/compiler/src/ir/runtime-features.ts index d16edfabd..7df85c738 100644 --- a/packages/compiler/src/ir/runtime-features.ts +++ b/packages/compiler/src/ir/runtime-features.ts @@ -46,6 +46,8 @@ export interface RuntimeFeatures { } const DYN_ASYNC_LIB_FNS: ReadonlySet = new Set([ + // DEP0040 rides the dyn-async unit's warning machinery. + "punycode.warn", "fs.callbackValue", "fs.callbackCall", "fs.callbackGlob", diff --git a/packages/compiler/src/ir/serialize.ts b/packages/compiler/src/ir/serialize.ts index 54e210ea3..1d0e5364e 100644 --- a/packages/compiler/src/ir/serialize.ts +++ b/packages/compiler/src/ir/serialize.ts @@ -7,6 +7,7 @@ * text parts and decoded from bytes one function at a time; neither direction * materializes the complete document as a single string. */ import type { IrExpr, IrFunction, IrModule, IrStmt } from "./ir.js"; +import { deriveModuleTiers } from "./tiers.js"; // Version 16 removes embedded-engine types, operations, and module metadata. // Older modules can require runtime support that no longer exists. @@ -42,6 +43,19 @@ function encodeNumber(_key: string, value: unknown): unknown { return value; } +/** Stage S2 of the two-tier design keeps function tiers out of IR + * artifacts, so a report-only release changes no serialized byte; loaders + * derive them again (deriveModuleTiers). No IR node other than a function + * carries a `tier` key. */ +function encodeWithoutTier(key: string, value: unknown): unknown { + return key === "tier" ? undefined : encodeNumber(key, value); +} + +function encodeFunction(fn: IrFunction): string | undefined { + if (fn.tier !== undefined) return JSON.stringify(fn, encodeWithoutTier); + return ordinaryNumbers(fn) ? JSON.stringify(fn) : JSON.stringify(fn, encodeNumber); +} + /** The compact encoding as ordered parts: the module header up to the * function array, one part per function (with separate comma parts), and the * header's remainder. Concatenating the parts yields exactly @@ -58,9 +72,7 @@ export function serializeModuleParts(mod: IrModule): string[] { for (let index = 0; index < mod.functions.length; index++) { const fn = mod.functions[index]!; if (index > 0) parts.push(","); - parts.push( - (ordinaryNumbers(fn) ? JSON.stringify(fn) : JSON.stringify(fn, encodeNumber)) ?? "null", - ); + parts.push(encodeFunction(fn) ?? "null"); } parts.push("]" + header.slice(offset + slot.length)); return parts; @@ -70,7 +82,7 @@ export function serializeModuleParts(mod: IrModule): string[] { * readable default. Both return one string, so modules whose encoding * exceeds the host's string limit must use `serializeModuleParts`. */ export function serializeModule(mod: IrModule, compact = false): string { - if (!compact) return JSON.stringify(mod, encodeNumber, 2); + if (!compact) return JSON.stringify(mod, encodeWithoutTier, 2); return serializeModuleParts(mod).join(""); } @@ -96,7 +108,7 @@ function checkedVersion(mod: IrModule): IrModule { } export function deserializeModule(json: string): IrModule { - return checkedVersion(JSON.parse(json, decodeNumber) as IrModule); + return deriveModuleTiers(checkedVersion(JSON.parse(json, decodeNumber) as IrModule)); } /** A node as it reads back from the compact encoding: a fresh object, as @@ -249,5 +261,5 @@ export function deserializeModuleBytes(bytes: Buffer): IrModule { ); } mod.functions = functions; - return mod; + return deriveModuleTiers(mod); } diff --git a/packages/compiler/src/ir/tiers.test.ts b/packages/compiler/src/ir/tiers.test.ts new file mode 100644 index 000000000..c5f1ad4b8 --- /dev/null +++ b/packages/compiler/src/ir/tiers.test.ts @@ -0,0 +1,142 @@ +import { expect, test } from "vitest"; +import { BOOL, DYN, F64, STRING, VOID, type IrExpr, type IrFunction, type IrModule } from "./ir.js"; +import { + deserializeModule, + deserializeModuleBytes, + serializeModule, + serializeModuleParts, +} from "./serialize.js"; +import { + assignLegacyTiers, + dynamicOperationOf, + firstDynamicOperation, + functionTier, +} from "./tiers.js"; +import { validateModule } from "./validate.js"; + +const loc = { file: "tiers.ts", start: 0, end: 0 }; +const dynParam = { kind: "varRef", localId: "v", type: DYN, loc } as const satisfies IrExpr; + +function fn(name: string, value: IrExpr): IrFunction { + return { + name, + params: [{ localId: "v", name: "v", type: DYN }], + returnType: value.type, + locals: [{ id: "v", name: "v", type: DYN, mutable: false }], + body: [{ kind: "return", value, loc }], + loc, + }; +} + +function moduleOf(...functions: IrFunction[]): IrModule { + const main: IrFunction = { + name: "%main", + params: [], + returnType: VOID, + locals: [], + body: [], + loc, + }; + return { irVersion: 16, sourceFile: "tiers.ts", functions: [main, ...functions], entry: "%main" }; +} + +const keyGet: IrExpr = { + kind: "dynKeyGet", + key: { kind: "strLit", value: "name", type: STRING, loc }, + value: dynParam, + type: DYN, + loc, +}; + +test("dynamic operations and boundary operations are told apart", () => { + expect(dynamicOperationOf(keyGet)).toBe("dynKeyGet"); + expect( + dynamicOperationOf({ + kind: "libCall", + fn: "dyn.add", + args: [dynParam, dynParam], + type: DYN, + loc, + }), + ).toBe("dyn.add"); + expect( + dynamicOperationOf({ + kind: "call", + callee: "%dyn.class.property.12", + args: [dynParam], + type: DYN, + loc, + }), + ).toBe("%dyn.class.property"); + // Boundary operations: conversions and kind tests. + expect(dynamicOperationOf({ kind: "dynCheck", value: dynParam, type: F64, loc })).toBeNull(); + expect( + dynamicOperationOf({ + kind: "dynFrom", + value: { kind: "numLit", value: 1, type: F64, loc }, + type: DYN, + loc, + }), + ).toBeNull(); + expect( + dynamicOperationOf({ kind: "dynTest", test: "string", value: dynParam, type: BOOL, loc }), + ).toBeNull(); + expect( + dynamicOperationOf({ kind: "libCall", fn: "dyn.typedRefIs", args: [dynParam], type: DYN, loc }), + ).toBeNull(); + // Moves and calls of ordinary functions. + expect(dynamicOperationOf(dynParam)).toBeNull(); + expect( + dynamicOperationOf({ kind: "call", callee: "helper", args: [dynParam], type: DYN, loc }), + ).toBeNull(); +}); + +test("lowering's classification puts exactly the functions with dynamic operations in the dynamic tier", () => { + const reader = fn("reader", keyGet); + const mover = fn("mover", dynParam); + const dynamic = assignLegacyTiers([reader, mover]); + expect(dynamic.map((d) => [d.name, d.operation])).toEqual([["reader", "dynKeyGet"]]); + expect(functionTier(reader)).toBe("dynamic"); + expect(functionTier(mover)).toBe("static"); + expect(firstDynamicOperation(mover)).toBeNull(); +}); + +test("the validator enforces I1: static functions perform no dynamic operation", () => { + const reader = fn("reader", keyGet); + const errors = validateModule(moduleOf(reader)); + expect(errors.map((e) => e.message)).toEqual([ + "in reader: static-tier function performs the dynamic operation dynKeyGet", + ]); + reader.tier = "dynamic"; + expect(validateModule(moduleOf(reader))).toEqual([]); + // A logical operator over dynamic values applies ToBoolean to them. + const either = fn("either", { + kind: "logical", + op: "||", + left: dynParam, + right: dynParam, + type: DYN, + loc, + }); + expect(validateModule(moduleOf(either)).map((e) => e.message)).toEqual([ + "in either: static-tier function performs the dynamic operation logical", + ]); +}); + +test("serialized IR carries no tiers in stage S2, and loading derives them again", () => { + const reader = fn("reader", keyGet); + const mod = moduleOf(reader); + const untiered = serializeModule(mod, true); + assignLegacyTiers(mod.functions); + expect(reader.tier).toBe("dynamic"); + expect(serializeModule(mod, true)).toBe(untiered); + expect(serializeModuleParts(mod).join("")).toBe(untiered); + expect(serializeModule(mod)).not.toContain('"tier"'); + for (const loaded of [ + deserializeModule(untiered), + deserializeModuleBytes(Buffer.from(untiered)), + ]) { + expect(loaded.functions.map(functionTier)).toEqual(["static", "dynamic"]); + expect(validateModule(loaded)).toEqual([]); + } +}); diff --git a/packages/compiler/src/ir/tiers.ts b/packages/compiler/src/ir/tiers.ts new file mode 100644 index 000000000..201bc2a58 --- /dev/null +++ b/packages/compiler/src/ir/tiers.ts @@ -0,0 +1,157 @@ +/* Execution tiers (two-tier design, stage S2: report-only). + * + * Every IR function belongs to one of two tiers: + * + * - static: today's compiler. Exact layouts, direct calls, unboxed values. + * A static function may hold dynamic (`dyn`) values, pass them to calls, + * return them and store them, and it may cross the boundary with the + * boundary operations below, but it never operates on a dynamic value. + * - dynamic: code that operates on dynamic values with JavaScript semantics + * (property reads and writes, generic operators, calls of dynamic values, + * dynamic object and array construction). + * + * Invariant I1 (validate.ts): a static function contains no dynamic + * operation. Only boundary operations and moves may mix the tiers. Calls + * of generated `%dyn.*` helpers count as dynamic operations: they are the + * dynamic dispatch the lowering placed out of line. Calls of other dynamic + * functions are boundaries (the design's "call to a dynamic function"). + * + * Boundary operations: + * - `dynFrom` (static to dynamic, the design's `toJs`) and `caughtToDyn`; + * - `dynCheck` (dynamic to static, the design's `fromJs`); + * - kind tests that only inspect a dynamic value's tag (`dynTest`, the + * `dyn.*Is` and `dyn.typeof` builtins): the tag test of a conversion; + * - calls of non-`dyn.*` builtins whose signatures take or return `dyn` + * (typed APIs implemented through dyn, leak L6 of the design). They stay + * boundaries until stage S1 gives them typed signatures. + * + * Classification of existing code. The `dyn` vocabulary is today's + * dynamic tier (`ScrDyn`): `any`, untyped JavaScript, unschematized + * `JSON.parse`, and the generated dynamic dispatch helpers already lower to + * it. Lowering therefore assigns tiers from function bodies: a function + * containing a dynamic operation is dynamic, every other function is static + * (`assignLegacyTiers`). Demotion (stage S6) will add tier decisions that + * do not follow from a body; until then the classification is exact, and + * the validator guards it against later passes and hand-written IR. + * + * Serialization. Stage S2 must not change any artifact, so serialized IR + * omits `tier` (serialize.ts) and loaders derive it again with the same + * classification. The next IR version serializes the field. */ +import type { IrExpr, IrFunction, IrModule, SrcLoc } from "./ir.js"; +import { everyStmtList, type IrVisitor } from "./traverse.js"; + +export type IrTier = "static" | "dynamic"; + +export function functionTier(fn: IrFunction): IrTier { + return fn.tier === "dynamic" ? "dynamic" : "static"; +} + +/** `dyn.*` builtins that only inspect a dynamic value's kind: the tag test + * half of a dynamic-to-static conversion. */ +const DYN_KIND_TESTS: ReadonlySet = new Set([ + "dyn.typedRefIs", + "dyn.typedRefKeyIs", + "dyn.classIs", + "dyn.nativeSetIs", + "dyn.nativeMapIs", + "dyn.nativeUrlIs", + "dyn.nativeDateIs", + "dyn.nativeRegexIs", + "dyn.dataViewIs", + "dyn.typeof", +]); + +const DYN_HELPER_PREFIX = "%dyn."; + +/** `%dyn.class.property.12` → `%dyn.class.property`. */ +function dynHelperFamily(name: string): string { + return name.replace(/(\.\d+)+$/, ""); +} + +/** The dynamic operation an expression performs, named for reports + * (`dynKeyGet`, `dyn.keySet`, ...), or null for static operations, boundary + * operations and moves. Only the node itself is inspected. */ +export function dynamicOperationOf(e: IrExpr): string | null { + switch (e.kind) { + case "dynCall": + case "dynInvoke": + case "dynObjLit": + case "dynArrLit": + case "dynKeyGet": + case "dynHasKey": + case "dynScalarEq": + case "dynDestrCheck": + case "dynIterN": + return e.kind; + case "logical": + // `a || b` over dyn operands applies ToBoolean to a dynamic value. + return e.type.kind === "dyn" ? "logical" : null; + case "libCall": + return e.fn.startsWith("dyn.") && !DYN_KIND_TESTS.has(e.fn) ? e.fn : null; + case "call": + // Generated `%dyn.*` helpers are dynamic operations lowered out of + // line (class dynamic dispatch, dynamic HOF adapters). + return e.callee.startsWith(DYN_HELPER_PREFIX) ? dynHelperFamily(e.callee) : null; + default: + return null; + } +} + +export interface DynamicOperationSite { + operation: string; + loc: SrcLoc; +} + +function collectDynamicOperations(fn: IrFunction, first: boolean): DynamicOperationSite[] { + const found: DynamicOperationSite[] = []; + const visitor: IrVisitor = { + expr: (e) => { + const operation = dynamicOperationOf(e); + if (operation === null) return true; + found.push({ operation, loc: e.loc }); + return !first; + }, + stmt: () => true, + }; + everyStmtList(fn.body, visitor); + return found; +} + +/** The first dynamic operation in a function body, in traversal order. */ +export function firstDynamicOperation(fn: IrFunction): DynamicOperationSite | null { + return collectDynamicOperations(fn, true)[0] ?? null; +} + +/** Every dynamic operation in a function body (coverage reports). */ +export function dynamicOperationSites(fn: IrFunction): DynamicOperationSite[] { + return collectDynamicOperations(fn, false); +} + +/** A function the classification placed in the dynamic tier, with the + * operation that put it there. */ +export interface DynamicFunction { + name: string; + loc: SrcLoc; + operation: string; + operationLoc: SrcLoc; +} + +/** Assign every function its tier from its body (see the header). Returns + * the dynamic functions in module order. */ +export function assignLegacyTiers(functions: readonly IrFunction[]): DynamicFunction[] { + const dynamic: DynamicFunction[] = []; + for (const fn of functions) { + const site = firstDynamicOperation(fn); + if (site === null) continue; + fn.tier = "dynamic"; + dynamic.push({ name: fn.name, loc: fn.loc, operation: site.operation, operationLoc: site.loc }); + } + return dynamic; +} + +/** Loaders call this on deserialized modules (stage S2 serializes no + * tiers). */ +export function deriveModuleTiers(mod: IrModule): IrModule { + assignLegacyTiers(mod.functions); + return mod; +} diff --git a/packages/compiler/src/ir/validate.test.ts b/packages/compiler/src/ir/validate.test.ts index b7caf8c41..8086cc176 100644 --- a/packages/compiler/src/ir/validate.test.ts +++ b/packages/compiler/src/ir/validate.test.ts @@ -20,6 +20,7 @@ import { type IrUnionDef, } from "./ir.js"; import { deserializeModule, serializeModule } from "./serialize.js"; +import { assignLegacyTiers } from "./tiers.js"; import { validateModule } from "./validate.js"; const loc = { file: "numeric-read.ts", start: 0, end: 0 }; @@ -399,7 +400,7 @@ test.each(["sortPrimitive", "toSortedPrimitive"] as const)( ); function expressionModule(expr: IrExpr, unions: IrUnionDef[]): IrModule { - return { + const mod: IrModule = { irVersion: 16, sourceFile: loc.file, entry: "main", @@ -415,6 +416,9 @@ function expressionModule(expr: IrExpr, unions: IrUnionDef[]): IrModule { }, ], }; + // Lowering assigns tiers from function bodies (ir/tiers.ts). + assignLegacyTiers(mod.functions); + return mod; } test("numeric byte tokens and DataView stores validate their storage before emission", () => { diff --git a/packages/compiler/src/ir/validate.ts b/packages/compiler/src/ir/validate.ts index d10160115..09ab7d951 100644 --- a/packages/compiler/src/ir/validate.ts +++ b/packages/compiler/src/ir/validate.ts @@ -59,6 +59,7 @@ import { import { DYN_CLASS_PROPERTIES } from "./ir.js"; import { unionWideningTags } from "./analysis.js"; import { alwaysReturns } from "./control-flow.js"; +import { dynamicOperationOf, functionTier } from "./tiers.js"; import { LIB_FN_SIGS } from "./builtin-signatures.js"; /** The promisified crypto calls' value kinds besides checked-dynamic. */ @@ -1019,6 +1020,14 @@ function validateFunction( const locals = new Map(fn.locals.map((l) => [l.id, l])); const err = (message: string, loc: SrcLoc) => errors.push({ message: `in ${fn.name}: ${message}`, loc }); + // Invariant I1 (ir/tiers.ts): a static function reaches dynamic values + // only through boundary operations and moves. + const staticTier = functionTier(fn) === "static"; + const checkTier = (e: IrExpr): void => { + const operation = dynamicOperationOf(e); + if (operation !== null) + err(`static-tier function performs the dynamic operation ${operation}`, e.loc); + }; const asyncCaches = [ ["asyncCacheGlobal", fn.asyncCacheGlobal], @@ -1263,6 +1272,8 @@ function validateFunction( continue; } if (e.kind === "logical") { + // checkExpr already checked the root's tier. + if (staticTier && e !== root) checkTier(e); if ( e.type.kind !== "f64" && e.type.kind !== "string" && @@ -1306,6 +1317,7 @@ function validateFunction( } function checkExpr(e: IrExpr): void { + if (staticTier) checkTier(e); switch (e.kind) { case "numLit": case "strLit": diff --git a/packages/compiler/src/library/prepare.ts b/packages/compiler/src/library/prepare.ts index 94f20198a..298bb43bb 100644 --- a/packages/compiler/src/library/prepare.ts +++ b/packages/compiler/src/library/prepare.ts @@ -553,6 +553,8 @@ export function prepareLibrary( try { lowered = fe.lower({ targetPlatform: buildPlatform, + // Library artifacts are static-tier only (two-tier design I2). + tierMode: "static-only", ...(cbImports.length > 0 ? { ffiImports: cbImports } : {}), ...(cbImports.length > 0 ? { libraryCallbacks: true } : {}), // The profile-mapped exports are called from OUTSIDE the graph: diff --git a/packages/compiler/surface-manifest.json b/packages/compiler/surface-manifest.json index f5313f12d..6dc871976 100644 --- a/packages/compiler/surface-manifest.json +++ b/packages/compiler/surface-manifest.json @@ -466,6 +466,13 @@ "status": "static", "note": "recognized module (bare and node:-prefixed specifiers)" }, + { + "id": "node-builtin.constants", + "kind": "node-builtin", + "name": "constants", + "status": "static", + "note": "recognized module (bare and node:-prefixed specifiers)" + }, { "id": "node-builtin.crypto", "kind": "node-builtin", @@ -2107,6 +2114,56 @@ "status": "static", "note": "shared version dictionary containing node and openssl; other components are absent unless defined by the program" }, + { + "id": "node-builtin.punycode", + "kind": "node-builtin", + "name": "punycode", + "status": "static", + "note": "recognized module (bare and node:-prefixed specifiers)" + }, + { + "id": "node-builtin.punycode.decode", + "kind": "node-builtin", + "name": "punycode.decode", + "status": "static" + }, + { + "id": "node-builtin.punycode.encode", + "kind": "node-builtin", + "name": "punycode.encode", + "status": "static" + }, + { + "id": "node-builtin.punycode.toASCII", + "kind": "node-builtin", + "name": "punycode.toASCII", + "status": "static" + }, + { + "id": "node-builtin.punycode.toUnicode", + "kind": "node-builtin", + "name": "punycode.toUnicode", + "status": "static" + }, + { + "id": "node-builtin.punycode.ucs2.decode", + "kind": "node-builtin", + "name": "punycode.ucs2.decode", + "status": "static" + }, + { + "id": "node-builtin.punycode.ucs2.encode", + "kind": "node-builtin", + "name": "punycode.ucs2.encode", + "status": "static" + }, + { + "id": "node-builtin.punycode.version", + "kind": "node-builtin", + "name": "punycode.version", + "status": "static", + "note": "constant value read" + }, { "id": "node-builtin.querystring", "kind": "node-builtin", diff --git a/packages/compiler/test/class-early-field-reads.test.ts b/packages/compiler/test/class-early-field-reads.test.ts deleted file mode 100644 index c602a2f42..000000000 --- a/packages/compiler/test/class-early-field-reads.test.ts +++ /dev/null @@ -1,93 +0,0 @@ -import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { expect, test } from "vitest"; -import { compile, deserializeModule, validateModule } from "../src/index.js"; -import type { IrModule } from "../src/ir/ir.js"; - -async function lower(source: string, file = "main.ts"): Promise { - const dir = await mkdtemp(join(tmpdir(), "scriptc-early-field-reads-")); - try { - const entry = join(dir, file), - output = join(dir, "main.ir.json"); - await writeFile(entry, source); - const result = await compile(entry, { outDir: dir, outPath: output, outputKind: "ir" }); - if (!result.ok) - throw new Error(result.diagnostics.map((item) => `${item.code}: ${item.message}`).join("\n")); - const module = deserializeModule(await readFile(output, "utf8")); - expect(validateModule(module)).toEqual([]); - return module; - } finally { - await rm(dir, { recursive: true, force: true }); - } -} - -/** Field name → IR kind for a class. */ -function fields(module: IrModule, name: string): string { - const cls = module.classes.find((candidate) => candidate.name === name); - expect(cls, name).toBeDefined(); - return cls!.fields - .filter((field) => !field.name.startsWith("%")) - .map((field) => `${field.name}:${field.type.kind}`) - .join(" "); -} - -test("fields assigned after construction can call an override admit undefined", async () => { - const module = await lower(` -abstract class Base { - constructor() { - this.describe(); - } - abstract describe(): void; -} -class Panel extends Base { - label: string; - width: number; - count = 0; - constructor() { - super(); - this.label = "panel"; - this.width = 2; - } - describe(): void { - console.log(this.label, this.width, this.count); - } -} -class Plain { - label: string; - width: number; - constructor() { - this.label = "plain"; - this.width = 1; - this.describe(); - } - describe(): void { - console.log(this.label, this.width); - } -} -new Panel(); -new Plain(); -`); - expect(fields(module, "Panel")).toBe("label:union width:union count:union"); - expect(fields(module, "Plain")).toBe("label:string width:f64"); -}); - -test("JavaScript fields created after the constructor calls a method admit undefined", async () => { - const module = await lower( - ` -class Widget { - constructor() { - this.ready = true; - this.render(); - this.title = "widget"; - } - render() { - console.log(this.ready, this.title); - } -} -new Widget(); -`, - "main.js", - ); - expect(fields(module, "Widget")).toBe("ready:bool title:union"); -}); diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5633-zlib-typed-options.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5633-zlib-typed-options.ts.json new file mode 100644 index 000000000..f4d9673d5 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5633-zlib-typed-options.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5633-zlib-typed-options.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5683-date-local-fields-tz.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5683-date-local-fields-tz.ts.json new file mode 100644 index 000000000..7ac133650 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5683-date-local-fields-tz.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5683-date-local-fields-tz.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5919-record-unions-shared-arms.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5919-record-unions-shared-arms.ts.json new file mode 100644 index 000000000..bfb23e115 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5919-record-unions-shared-arms.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5919-record-unions-shared-arms.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5930-record-union-pointers.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5930-record-union-pointers.ts.json new file mode 100644 index 000000000..e6e3632f5 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5930-record-union-pointers.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5930-record-union-pointers.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5931-record-union-unit-arms.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5931-record-union-unit-arms.ts.json new file mode 100644 index 000000000..53d531568 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5931-record-union-unit-arms.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5931-record-union-unit-arms.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5932-record-union-trees.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5932-record-union-trees.ts.json new file mode 100644 index 000000000..ee4281193 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5932-record-union-trees.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5932-record-union-trees.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5933-record-union-conversions.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5933-record-union-conversions.ts.json new file mode 100644 index 000000000..a1fcec317 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5933-record-union-conversions.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5933-record-union-conversions.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5934-record-union-set-members.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5934-record-union-set-members.ts.json new file mode 100644 index 000000000..98b0ac1f4 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5934-record-union-set-members.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5934-record-union-set-members.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5943-fields-read-before-assignment.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5943-fields-read-before-assignment.ts.json deleted file mode 100644 index d2dcdcc24..000000000 --- a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5943-fields-read-before-assignment.ts.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "order": [ - "/tests/corpus/5943-fields-read-before-assignment.ts" - ], - "diags": [] -} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5944-js-fields-read-before-assignment.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5944-js-fields-read-before-assignment.js.json deleted file mode 100644 index e9800b71d..000000000 --- a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5944-js-fields-read-before-assignment.js.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "order": [ - "/tests/corpus/5944-js-fields-read-before-assignment.js" - ], - "diags": [] -} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5976-builtin-class-aliases.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5976-builtin-class-aliases.ts.json new file mode 100644 index 000000000..f83d23fc7 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5976-builtin-class-aliases.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5976-builtin-class-aliases.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5977-builtin-class-aliases.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5977-builtin-class-aliases.cjs.json new file mode 100644 index 000000000..99d07e4cd --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5977-builtin-class-aliases.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5977-builtin-class-aliases.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5978-builtin-class-alias-modules/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5978-builtin-class-alias-modules/main.ts.json new file mode 100644 index 000000000..fd6bb94f5 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5978-builtin-class-alias-modules/main.ts.json @@ -0,0 +1,7 @@ +{ + "order": [ + "/tests/corpus/5978-builtin-class-alias-modules/classes.ts", + "/tests/corpus/5978-builtin-class-alias-modules/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5980-cycle-components.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5980-cycle-components.ts.json new file mode 100644 index 000000000..cdbf47751 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5980-cycle-components.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5980-cycle-components.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5993-array-length-loop-resizing.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5993-array-length-loop-resizing.ts.json new file mode 100644 index 000000000..e4cf027ce --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5993-array-length-loop-resizing.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5993-array-length-loop-resizing.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5994-array-length-sparse-loops.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5994-array-length-sparse-loops.ts.json new file mode 100644 index 000000000..6289e8a21 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5994-array-length-sparse-loops.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5994-array-length-sparse-loops.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5995-array-dense-loop-boundaries.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5995-array-dense-loop-boundaries.ts.json new file mode 100644 index 000000000..d0afcf505 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5995-array-dense-loop-boundaries.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5995-array-dense-loop-boundaries.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5996-rejections-between-macrotasks.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5996-rejections-between-macrotasks.cjs.json new file mode 100644 index 000000000..0e719fe78 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/5996-rejections-between-macrotasks.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/5996-rejections-between-macrotasks.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6100-record-get-or-create.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6100-record-get-or-create.ts.json new file mode 100644 index 000000000..b7c18a8a4 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6100-record-get-or-create.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6100-record-get-or-create.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6101-record-get-or-create-unchecked/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6101-record-get-or-create-unchecked/main.ts.json new file mode 100644 index 000000000..dcb5b192f --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6101-record-get-or-create-unchecked/main.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6101-record-get-or-create-unchecked/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6103-typeof-builtin-bindings.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6103-typeof-builtin-bindings.cjs.json new file mode 100644 index 000000000..008914d85 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6103-typeof-builtin-bindings.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6103-typeof-builtin-bindings.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6104-typeof-builtin-bindings.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6104-typeof-builtin-bindings.mjs.json new file mode 100644 index 000000000..b6a8a48e2 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6104-typeof-builtin-bindings.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6104-typeof-builtin-bindings.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6105-cjs-top-level-this/main.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6105-cjs-top-level-this/main.js.json new file mode 100644 index 000000000..ef768b4ce --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6105-cjs-top-level-this/main.js.json @@ -0,0 +1,7 @@ +{ + "order": [ + "/tests/corpus/6105-cjs-top-level-this/lib.js", + "/tests/corpus/6105-cjs-top-level-this/main.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6106-esm-top-level-this.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6106-esm-top-level-this.mjs.json new file mode 100644 index 000000000..ad427bd7f --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6106-esm-top-level-this.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6106-esm-top-level-this.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6110-cjs-speculative-wrapper-fences.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6110-cjs-speculative-wrapper-fences.cjs.json new file mode 100644 index 000000000..0a5ce66b0 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6110-cjs-speculative-wrapper-fences.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6110-cjs-speculative-wrapper-fences.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6120-type-only-import-graph/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6120-type-only-import-graph/main.ts.json new file mode 100644 index 000000000..979391c1c --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6120-type-only-import-graph/main.ts.json @@ -0,0 +1,8 @@ +{ + "order": [ + "/tests/corpus/6120-type-only-import-graph/mixed.ts", + "/tests/corpus/6120-type-only-import-graph/flags.ts", + "/tests/corpus/6120-type-only-import-graph/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6121-type-specifier-edges/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6121-type-specifier-edges/main.ts.json new file mode 100644 index 000000000..c0f3f653c --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6121-type-specifier-edges/main.ts.json @@ -0,0 +1,10 @@ +{ + "order": [ + "/tests/corpus/6121-type-specifier-edges/config.ts", + "/tests/corpus/6121-type-specifier-edges/config-barrel.ts", + "/tests/corpus/6121-type-specifier-edges/b.ts", + "/tests/corpus/6121-type-specifier-edges/a.ts", + "/tests/corpus/6121-type-specifier-edges/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6122-punycode-conversions.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6122-punycode-conversions.ts.json new file mode 100644 index 000000000..b15e4fe13 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6122-punycode-conversions.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6122-punycode-conversions.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6124-punycode-require-warnings.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6124-punycode-require-warnings.cjs.json new file mode 100644 index 000000000..dfd4aa912 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6124-punycode-require-warnings.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6124-punycode-require-warnings.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6125-constants-module.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6125-constants-module.cjs.json new file mode 100644 index 000000000..ef8c7f2bf --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6125-constants-module.cjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/6125-constants-module.cjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6126-punycode-dependency/main.cjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6126-punycode-dependency/main.cjs.json new file mode 100644 index 000000000..5de83361a --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/6126-punycode-dependency/main.cjs.json @@ -0,0 +1,18 @@ +{ + "order": [ + "/tests/corpus/6126-punycode-dependency/main.cjs" + ], + "diags": [ + { + "code": "SC1010", + "message": "the 'puny-dep' module is not supported yet", + "loc": { + "file": "/tests/corpus/6126-punycode-dependency/main.cjs", + "start": 347, + "end": 376 + }, + "milestone": "M4", + "hint": "relative imports (./file, ../dir/file), package.json-mediated project imports (#alias via the imports field, self-name references via exports), statically compiled npm packages, and the built-in fs, fs/promises, path, os, url, crypto, zlib, child_process, net, http, tls, https, http2, dgram, dns, util, util/types, string_decoder, querystring, readline, events, stream, stream/promises, stream/consumers, buffer, assert, assert/strict, worker_threads, cluster, tty, async_hooks, timers, timers/promises, diagnostics_channel, perf_hooks, module, and console modules (bare or node:-prefixed) and node:test (node:-prefixed only, like in Node) are supported" + } + ] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-builder-string-method-names/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-builder-string-method-names/main.ts.json new file mode 100644 index 000000000..d1f487f89 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-builder-string-method-names/main.ts.json @@ -0,0 +1,7 @@ +{ + "order": [ + "/tests/corpus/js-builder-string-method-names/builder.js", + "/tests/corpus/js-builder-string-method-names/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-callable-builder.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-callable-builder.js.json new file mode 100644 index 000000000..8f1025c47 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-callable-builder.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/js-callable-builder.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-class-constructor-returns/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-class-constructor-returns/main.ts.json new file mode 100644 index 000000000..e05f1760c --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-class-constructor-returns/main.ts.json @@ -0,0 +1,7 @@ +{ + "order": [ + "/tests/corpus/js-class-constructor-returns/factory.js", + "/tests/corpus/js-class-constructor-returns/main.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-function-prototype-value.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-function-prototype-value.js.json new file mode 100644 index 000000000..b0fa827e1 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-function-prototype-value.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/js-function-prototype-value.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-number-parse-dynamic.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-number-parse-dynamic.js.json new file mode 100644 index 000000000..608e24d40 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-number-parse-dynamic.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/js-number-parse-dynamic.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-process-destructure.mjs.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-process-destructure.mjs.json new file mode 100644 index 000000000..87d43324e --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-process-destructure.mjs.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/js-process-destructure.mjs" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-reshaped-object-literal.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-reshaped-object-literal.js.json new file mode 100644 index 000000000..939240b2c --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/js-reshaped-object-literal.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/js-reshaped-object-literal.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/scratch-string-slices.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/scratch-string-slices.ts.json new file mode 100644 index 000000000..5145c33d1 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/scratch-string-slices.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/scratch-string-slices.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/truthy-object-logical.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/truthy-object-logical.ts.json new file mode 100644 index 000000000..96661c630 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/truthy-object-logical.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/truthy-object-logical.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/builtin-class-alias-escape.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/builtin-class-alias-escape.ts.json new file mode 100644 index 000000000..f56d058f1 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/builtin-class-alias-escape.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/diagnostics/builtin-class-alias-escape.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query-brand.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query-brand.ts.json new file mode 100644 index 000000000..e17ea3bf5 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query-brand.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/diagnostics/local-class-type-query-brand.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query-js.js.json b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query-js.js.json new file mode 100644 index 000000000..6bf774cd8 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query-js.js.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/diagnostics/local-class-type-query-js.js" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query.ts.json new file mode 100644 index 000000000..3c89fba9f --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/local-class-type-query.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/diagnostics/local-class-type-query.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/speculative-function-refusal.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/speculative-function-refusal.ts.json new file mode 100644 index 000000000..e24349766 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/diagnostics/speculative-function-refusal.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/diagnostics/speculative-function-refusal.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/chalk5/main.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/chalk5/main.ts.json new file mode 100644 index 000000000..065c96347 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/fixtures/npm/cases/chalk5/main.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/fixtures/npm/cases/chalk5/main.ts" + ], + "diags": [] +} diff --git a/packages/runtime/src/scr_async.c b/packages/runtime/src/scr_async.c index 308c9198d..48f3a5329 100644 --- a/packages/runtime/src/scr_async.c +++ b/packages/runtime/src/scr_async.c @@ -4464,7 +4464,9 @@ static bool scr_loop_run_pass(ScrPromise *top_level, bool first_checkpoint) { } scr_firing_id = 0; if (scr_exc_pending()) return false; /* uncaught throw in a callback: main handles it */ - if (scr_ready_len > 0 || scr_nt_head != NULL) break; /* drain ticks and microtasks before more timers */ + /* Drain ticks and microtasks, and report the rejections the callback + * left, before more timers (Node's runNextTicks between timers). */ + if (scr_ready_len > 0 || scr_nt_head != NULL || scr_nunhandled > 0) break; } /* The check phase: immediates queued BEFORE this phase started run * now, FIFO; ones a callback queues wait for the next turn (the end @@ -4474,7 +4476,7 @@ static bool scr_loop_run_pass(ScrPromise *top_level, bool first_checkpoint) { * every macrotask; pending ticks or a woken fiber restart the turn * first (the timer loop's break above), so the phase only runs on a * drained checkpoint. */ - if (scr_ready_len == 0 && scr_nt_head == NULL && scr_pending_immediates > 0) { + if (scr_ready_len == 0 && scr_nt_head == NULL && scr_nunhandled == 0 && scr_pending_immediates > 0) { size_t end = scr_nimmediates; while (scr_immediates_head < end) { size_t i = scr_immediates_head++; @@ -4490,6 +4492,13 @@ static bool scr_loop_run_pass(ScrPromise *top_level, bool first_checkpoint) { scr_closure_release(cb); if (scr_exc_pending()) return false; /* uncaught throw: main handles it */ if (!scr_loop_run_checkpoint()) return false; + /* Node's runNextTicks between immediates also reports the + * rejections the callback left before the next one runs; a + * listener's jobs drain before the next report. */ + while (scr_nunhandled > 0 && (top_level == NULL || top_level->state != SCR_PROM_REJECTED)) { + if (scr_report_unhandled_rejections()) return true; + if (!scr_loop_run_checkpoint()) return false; + } if (top_level != NULL && top_level->state == SCR_PROM_REJECTED) break; } /* Fully drained: reset the cursor so the array reuses its slots. */ diff --git a/packages/runtime/src/scr_async.h b/packages/runtime/src/scr_async.h index 7f294ca29..efa98e942 100644 --- a/packages/runtime/src/scr_async.h +++ b/packages/runtime/src/scr_async.h @@ -212,6 +212,11 @@ void scr_process_on_warning(ScrDyn *fn); void scr_process_off_warning(ScrDyn *fn); void scr_process_emit_warning(ScrDyn *args); void scr_emit_warning(const char *name, const char *code, ScrStr *message); +/* node:punycode's first load (scr_async_dyn.c): later calls are module + * cache hits. The first reports DEP0040 when `report` (false for a require + * made from node_modules) — at once for a static ES import, which Node + * links before any module evaluates, otherwise on the next tick. */ +void scr_punycode_warn(bool immediate, bool report); /* scr_emit_warning once a 'warning' listener exists (scr_lib.c; set by the * listener registration): units that cannot link the dyn-async unit route * their warnings through it, and print the default report otherwise. */ diff --git a/packages/runtime/src/scr_async_dyn.c b/packages/runtime/src/scr_async_dyn.c index b7387c13e..95771d457 100644 --- a/packages/runtime/src/scr_async_dyn.c +++ b/packages/runtime/src/scr_async_dyn.c @@ -904,6 +904,27 @@ void scr_warning_emit_deferred(ScrDyn *w) { scr_warning_defer(w, scr_warning_tick); } +static SCR_TL bool scr_punycode_loaded = false; + +void scr_punycode_warn(bool immediate, bool report) { + if (scr_punycode_loaded) return; + scr_punycode_loaded = true; + if (!report) return; + static const char text[] = + "The `punycode` module is deprecated. Please use a userland alternative instead."; + ScrStr *message = scr_str_new(text, sizeof text - 1); + ScrStr *code = scr_str_new("DEP0040", 7); + ScrDyn *w = scr_warning_new(message, "DeprecationWarning", code); + if (immediate) { + if (!scr_process_deprecation_flag_on(0)) scr_warning_dispatch(w); + } else { + scr_warning_emit_deferred(w); + } + scr_dyn_release(w); + scr_str_release(code); + scr_str_release(message); +} + /* The console members' warnings. Node's emitWarning emits on * process.nextTick, so these queue in order and each rides one raw tick * on the shared tick queue; any left at exit are dropped with it. */ diff --git a/packages/runtime/src/scr_cycle.c b/packages/runtime/src/scr_cycle.c index 69485dd6c..e8e7c73fa 100644 --- a/packages/runtime/src/scr_cycle.c +++ b/packages/runtime/src/scr_cycle.c @@ -26,14 +26,19 @@ * count was already given up (which would over-decrement and free live * data). * - * The main graph phases keep a bounded depth-first walk and defer deeper - * traces to reusable worklists. Ordinary RC destruction and cross-generation - * releases are separate paths. White candidates are recorded during scan - * and filtered after outside roots settle, avoiding a separate gather walk. - * markGray also records the objects it grays and counts those still - * externally referenced (rc > 0 after the trial decrements). When none is, - * every gray object is garbage, which is what scan would find, so the pass - * whitens the recorded set and skips scan. + * The graph phases keep a bounded depth-first walk and defer deeper traces + * to reusable worklists. Ordinary RC destruction and cross-generation + * releases are separate paths. markGray records the objects it grays (up + * to SCR_CYC_GRAY_LIST_MAX) and counts those still externally referenced + * (rc > 0 after the trial decrements), per pass and per component: the + * objects first reached from one candidate. A component with none is + * garbage, which is what scan would find, so the pass dooms it at once. + * For the others, scan blackens from each recorded object that is still + * externally referenced, which restores exactly the survivors, and what + * stays gray is the garbage: no walk whitens it. Only a pass too large to + * record walks scan from the candidates, whitening as it goes, and then + * filters the white set after outside roots settle, avoiding a separate + * gather walk. * * ── generations ────────────────────────────────────────────────────── * A pass costs O(objects reachable from its candidates), and a candidate @@ -225,6 +230,9 @@ typedef struct { size_t trigger; size_t alloc_trigger; size_t gray_referenced; + size_t comp_referenced; + uint32_t gray_comp; + ScrVec live_ranges; size_t scheduled_mature_age; size_t xg_freed; size_t mature_dead; @@ -256,6 +264,9 @@ static _Thread_local ScrCycThread scr_cyc_thread = {.gen_limit = SCR_CYC_OLD}; #define scr_cyc_trigger (scr_cyc_thread.trigger) #define scr_cyc_alloc_trigger (scr_cyc_thread.alloc_trigger) #define scr_gray_referenced (scr_cyc_thread.gray_referenced) +#define scr_comp_referenced (scr_cyc_thread.comp_referenced) +#define scr_gray_comp (scr_cyc_thread.gray_comp) +#define scr_live_ranges (scr_cyc_thread.live_ranges) #define scr_cyc_scheduled_mature_age (scr_cyc_thread.scheduled_mature_age) #define scr_xg_freed (scr_cyc_thread.xg_freed) #define scr_cyc_mature_dead (scr_cyc_thread.mature_dead) @@ -347,7 +358,7 @@ void *scr_cyc_alloc(size_t size, ScrCycType *type) { void scr_cyc_free(void *obj) { scr_cyc_free_inline(obj); } -static void scr_cyc_grow(ScrVec *vec) { +static __attribute__((noinline, cold)) void scr_cyc_grow(ScrVec *vec) { /* Candidate positions are 31-bit (ScrCycHdr.buf). */ if (vec->cap > (size_t)(SCR_CYC_BUFFERED_BIT - 1) / 2) scr_cyc_oom(); vec->cap = vec->cap ? vec->cap * 2 : 64; @@ -368,6 +379,16 @@ static inline void scr_cyc_push_work(ScrVec *vec, void *obj) { if (vec->n > vec->used) vec->used = vec->n; } +/* The same push inline, for the gray list markGray fills once per object; + * the pass records the list's high-water mark itself (scr_cyc_note_used). */ +static inline void scr_cyc_push_gray(ScrVec *vec, void *obj) { + if (__builtin_expect(vec->n == vec->cap, 0)) scr_cyc_grow(vec); + vec->v[vec->n++] = obj; +} +static inline void scr_cyc_note_used(ScrVec *vec) { + if (vec->n > vec->used) vec->used = vec->n; +} + /* ── the candidate-root buffers (one per generation) ──────────────────── */ #ifndef SCR_WORKERS @@ -392,7 +413,6 @@ static SCR_TL ScrVec scr_promote; static SCR_TL ScrVec scr_pending; static SCR_TL ScrVec scr_restore_pending; #endif -#define SCR_CYC_WALK_DEPTH 64 /* The gathered white set (freed after the walk completes). */ #ifndef SCR_WORKERS @@ -547,9 +567,19 @@ static SCR_TL size_t scr_cyc_alloc_trigger = 0; #endif /* Gray objects whose count is above zero after the trial decrements so far * (see scr_mg_visit). markGray records the gray objects in scr_white while - * there are fewer than SCR_CYC_GRAY_LIST_MAX; a larger walk runs scan. */ + * there are fewer than SCR_CYC_GRAY_LIST_MAX; a larger walk runs scan. + * + * The same count is kept per component (see "components" at scr_cyc_pass): + * the objects one candidate's walk grays first carry that walk's number in + * `buf` (gray objects are never buffered during a pass), and + * scr_comp_referenced counts those still externally referenced. + * scr_live_ranges holds (start, end) ranges of scr_white whose component + * was externally referenced and is left to scan. */ #ifndef SCR_WORKERS static SCR_TL size_t scr_gray_referenced = 0; +static SCR_TL size_t scr_comp_referenced = 0; +static SCR_TL uint32_t scr_gray_comp = 0; +static SCR_TL ScrVec scr_live_ranges; #endif #define SCR_CYC_GRAY_LIST_MAX 65536 @@ -982,42 +1012,103 @@ void scr_cyc_on_release(void *obj) { ((child) == NULL || SCR_RC(child) == SIZE_MAX \ || scr_cyc_gen(scr_cyc_hdr(child)) > scr_gen_limit) -/* Trace callbacks forward an integer depth through their opaque context; - * it is never dereferenced. The ABI already carries it through every edge. */ -static void scr_mark_gray(void *obj, uintptr_t depth); +/* The walks are depth-first and recursive up to SCR_CYC_WALK_DEPTH, where + * an object is queued instead (scr_pending, or scr_restore_pending for the + * restoration, which can run inside scan) and traced from the queue. Trace + * callbacks forward that integer depth through their opaque context; it is + * never dereferenced. */ +#define SCR_CYC_WALK_DEPTH 64 + +/* A walk visits a dense cycle-capable array's children itself: the present + * dense elements, which are all scr_arr_trace_v visits when the array has + * no sparse entries, properties or metadata (no `ext`). Each walk gets its + * own copy, out of line, so the per-edge visitors stay small. */ +#define SCR_CYC_ARRAY_WALK(name, visit) \ + static __attribute__((noinline)) void name(const ScrArr *arr, void *ctx) { \ + size_t end = arr->len < arr->cap ? arr->len : arr->cap; \ + size_t filled = arr->filled < end ? arr->filled : end; \ + for (size_t i = 0; i < filled; i++) visit((void *)(uintptr_t)arr->data[i], ctx); \ + for (size_t i = filled; i < end; i++) \ + if (arr->present[i] == SCR_ARR_VALUE) visit((void *)(uintptr_t)arr->data[i], ctx); \ + } + +/* Visit every child of `obj` with VISIT, through ARRAY_WALK for a dense + * cycle-capable array and through the type's trace otherwise. */ +#define SCR_CYC_CHILDREN(obj, type, ARRAY_WALK, VISIT, depth) \ + do { \ + if ((type)->kind == SCR_CYC_KIND_ARRAY && ((const ScrArr *)(obj))->ext == NULL) \ + ARRAY_WALK((const ScrArr *)(obj), (void *)(depth)); \ + else \ + (type)->trace((obj), (VISIT), (void *)(depth)); \ + } while (0) + +static void scr_mg_visit(void *child, void *ctx); +static void scr_sb_visit(void *child, void *ctx); +static void scr_scan_visit(void *child, void *ctx); +SCR_CYC_ARRAY_WALK(scr_mg_array, scr_mg_visit) +SCR_CYC_ARRAY_WALK(scr_sb_array, scr_sb_visit) +SCR_CYC_ARRAY_WALK(scr_scan_array, scr_scan_visit) + +static inline __attribute__((always_inline)) void scr_mark_gray(void *obj, uintptr_t depth); static void scr_mg_visit(void *child, void *ctx) { if (child == NULL || SCR_RC(child) == SIZE_MAX) return; - if (scr_cyc_gen(scr_cyc_hdr(child)) > scr_gen_limit) { + ScrCycHdr *h = scr_cyc_hdr(child); + if (scr_cyc_gen(h) > scr_gen_limit) { scr_cyc_push_work(&scr_xgen_edges, scr_gray_holder); scr_cyc_push_work(&scr_xgen_edges, child); return; } size_t rc = --SCR_RC(child); - if (scr_cyc_color(scr_cyc_hdr(child)) == SCR_CYC_GRAY) { - if (rc == 0) scr_gray_referenced--; + if (scr_cyc_color(h) == SCR_CYC_GRAY) { + if (rc == 0) { + scr_gray_referenced--; + if (h->buf == scr_gray_comp) scr_comp_referenced--; + } return; } scr_mark_gray(child, (uintptr_t)ctx); } -static void scr_mark_gray(void *obj, uintptr_t depth) { +/* markGray reaches `obj` (its count already trial-decremented): gray it, + * number it with the walk's component, record it, count it while it is + * externally referenced, and walk its children. */ +static inline __attribute__((always_inline)) void scr_mark_gray(void *obj, uintptr_t depth) { ScrCycHdr *h = scr_cyc_hdr(obj); - if (scr_cyc_color(h) == SCR_CYC_GRAY) return; scr_cyc_set_color(h, SCR_CYC_GRAY); /* A full pass takes every buffer, so each buffered object it reaches - * leaves its buffer here, in the header write the walk makes anyway. */ - if (scr_gen_limit == SCR_CYC_OLD && !scr_cyc_take) scr_cyc_clear_buffered(h); - if (scr_white.n < SCR_CYC_GRAY_LIST_MAX) scr_cyc_push_work(&scr_white, obj); - if (SCR_RC(obj) > 0) scr_gray_referenced++; + * leaves its buffer here, in the header write that numbers it anyway. A + * limited pass can gray candidates it did not take; they keep their + * buffer position and are simply never counted as members. */ + if (!scr_cyc_is_buffered(h) || (scr_gen_limit == SCR_CYC_OLD && !scr_cyc_take)) + h->buf = scr_gray_comp; + if (scr_white.n < SCR_CYC_GRAY_LIST_MAX) scr_cyc_push_gray(&scr_white, obj); + if (SCR_RC(obj) > 0) { + scr_gray_referenced++; + scr_comp_referenced++; + } if (depth == SCR_CYC_WALK_DEPTH) { scr_cyc_push_work(&scr_pending, obj); return; } void *holder = scr_gray_holder; scr_gray_holder = obj; - scr_cyc_type(h)->trace(obj, scr_mg_visit, (void *)(depth + 1)); + const ScrCycType *type = scr_cyc_type(h); + SCR_CYC_CHILDREN(obj, type, scr_mg_array, scr_mg_visit, depth + 1); scr_gray_holder = holder; } +/* markGray from one (black) candidate, through the queue. */ +static void scr_mark_gray_from(void *root) { + scr_mark_gray(root, 0); + while (scr_pending.n) { + void *obj = scr_pending.v[--scr_pending.n]; + const ScrCycType *type = scr_cyc_type(scr_cyc_hdr(obj)); + scr_gray_holder = obj; + SCR_CYC_CHILDREN(obj, type, scr_mg_array, scr_mg_visit, 1); + } +} +/* scan's restoration: blacken an externally referenced object and + * everything it reaches that is not black yet, giving back the trial + * decrements of the edges it follows. */ static void scr_scan_black(void *obj, uintptr_t depth); static void scr_sb_visit(void *child, void *ctx) { if (SCR_CYC_SKIP(child)) return; @@ -1028,14 +1119,29 @@ static void scr_sb_visit(void *child, void *ctx) { static void scr_scan_black(void *obj, uintptr_t depth) { ScrCycHdr *h = scr_cyc_hdr(obj); scr_cyc_set_color(h, SCR_CYC_BLACK); + /* Drop markGray's component number: a survivor leaves the pass unbuffered + * unless it is a candidate that a limited pass did not take. */ + if (!scr_cyc_is_buffered(h)) scr_cyc_clear_buffered(h); if (scr_cyc_gen(h) < SCR_CYC_OLD) scr_cyc_push_work(&scr_promote, obj); if (depth == SCR_CYC_WALK_DEPTH) { scr_cyc_push_work(&scr_restore_pending, obj); return; } - scr_cyc_type(h)->trace(obj, scr_sb_visit, (void *)(depth + 1)); + const ScrCycType *type = scr_cyc_type(h); + SCR_CYC_CHILDREN(obj, type, scr_sb_array, scr_sb_visit, depth + 1); +} +static void scr_restore_from(void *root) { + scr_scan_black(root, 0); + while (scr_restore_pending.n) { + void *obj = scr_restore_pending.v[--scr_restore_pending.n]; + const ScrCycType *type = scr_cyc_type(scr_cyc_hdr(obj)); + SCR_CYC_CHILDREN(obj, type, scr_sb_array, scr_sb_visit, 1); + } } +/* scan, walking: restore a gray object that is externally referenced, else + * whiten it, record it once (a later outside root can still restore it; + * settle filters the list) and walk its children. */ static void scr_scan(void *obj, uintptr_t depth); static void scr_scan_visit(void *child, void *ctx) { if (SCR_CYC_SKIP(child)) return; @@ -1045,22 +1151,17 @@ static void scr_scan(void *obj, uintptr_t depth) { ScrCycHdr *h = scr_cyc_hdr(obj); if (scr_cyc_color(h) != SCR_CYC_GRAY) return; if (SCR_RC(obj) > 0) { - scr_scan_black(obj, 0); - while (scr_restore_pending.n) { - void *next = scr_restore_pending.v[--scr_restore_pending.n]; - scr_cyc_type(scr_cyc_hdr(next))->trace(next, scr_sb_visit, NULL); - } + scr_restore_from(obj); return; } scr_cyc_set_color(h, SCR_CYC_WHITE); - /* Record each whitened object once. A later outside root can restore it; - * filter the list only after every scan finishes. No gather trace needed. */ scr_cyc_push_work(&scr_white, obj); if (depth == SCR_CYC_WALK_DEPTH) { scr_cyc_push_work(&scr_pending, obj); return; } - scr_cyc_type(h)->trace(obj, scr_scan_visit, (void *)(depth + 1)); + const ScrCycType *type = scr_cyc_type(h); + SCR_CYC_CHILDREN(obj, type, scr_scan_array, scr_scan_visit, depth + 1); } /* ── cross-generation edges ───────────────────────────────────────────── */ @@ -1138,22 +1239,28 @@ static SCR_CYC_COMPACT void scr_xg_release(void *obj) { * spend their time in the trace and teardown calls they make, so this half is * built for size; the graph walk above it stays fully optimized. Returns how * many objects the pass freed. */ -static SCR_CYC_COMPACT size_t scr_cyc_settle(unsigned gen_limit) { +static SCR_CYC_COMPACT size_t scr_cyc_settle(unsigned gen_limit, bool doomed) { size_t dead = 0; scr_cyc_mature_dead = 0; - for (size_t i = 0; i < scr_white.n; i++) { + scr_cyc_note_used(&scr_white); + /* The list holds the dead set: doomed components, and what scan whitened + * (black once a later outside root restored it), or, when the pass found + * nothing externally referenced, every gray object. `doomed` says that + * every entry is a doomed component's already. */ + for (size_t i = 0; i < scr_white.n && !doomed; i++) { void *obj = scr_white.v[i]; ScrCycHdr *h = scr_cyc_hdr(obj); - if (scr_cyc_color(h) != SCR_CYC_WHITE) continue; - /* A limited pass can reach candidates it did not take: out of their - * buffers before they are freed. */ - if (scr_cyc_take && scr_cyc_is_buffered(h)) scr_cyc_on_dead(obj); - scr_cyc_set_color(h, SCR_CYC_DOOMED); - if (gen_limit < SCR_CYC_OLD) scr_cyc_freed(scr_cyc_type(h), gen_limit); - else scr_cyc_mature_dead += scr_cyc_gen(h) == SCR_CYC_MATURE; + unsigned color = scr_cyc_color(h); + if (color == SCR_CYC_BLACK) continue; + if (color != SCR_CYC_DOOMED) { + /* A limited pass can reach candidates it did not take: out of their + * buffers before they are freed. */ + if (scr_cyc_take && scr_cyc_is_buffered(h)) scr_cyc_on_dead(obj); + scr_cyc_set_color(h, SCR_CYC_DOOMED); + } scr_white.v[dead++] = obj; } - scr_white.n = dead; + if (!doomed) scr_white.n = dead; /* Pin the cross-generation targets of everything about to be freed, while * `gen` still holds the values the walk filtered on (promotion below is @@ -1217,13 +1324,19 @@ static SCR_CYC_COMPACT size_t scr_cyc_settle(unsigned gen_limit) { * over every page the dead structure used, and the structures built from * them, and every later walk over those, would miss the cache on most * objects. In address order the blocks come back in the order the bump - * pointers hand out fresh memory. Teardown order is not observable. */ + * pointers hand out fresh memory. Teardown order is not observable. The + * level statistics are counted here, where the teardown reads the type + * anyway; they only steer later passes. */ size_t freed = scr_white.n; if (scr_white.n >= SCR_CYC_ORDER_MIN) scr_mem_order_for_free(scr_white.v, scr_white.n, sizeof(ScrCycHdr)); for (size_t i = 0; i < scr_white.n; i++) { void *obj = scr_white.v[i]; - scr_cyc_type(scr_cyc_hdr(obj))->free_fn(obj); + ScrCycHdr *h = scr_cyc_hdr(obj); + const ScrCycType *type = scr_cyc_type(h); + if (gen_limit < SCR_CYC_OLD) scr_cyc_freed(type, gen_limit); + else scr_cyc_mature_dead += scr_cyc_gen(h) == SCR_CYC_MATURE; + type->free_fn(obj); } scr_white.n = 0; @@ -1299,6 +1412,7 @@ static size_t scr_cyc_pass_done(size_t freed) { scr_cyc_trim(&scr_restore_pending); scr_cyc_trim(&scr_xgen); scr_cyc_trim(&scr_xgen_edges); + scr_cyc_trim(&scr_live_ranges); return freed; } @@ -1369,28 +1483,114 @@ static size_t scr_cyc_pass(unsigned gen_limit) { scr_xgen_edges.n = 0; scr_white.n = 0; scr_gray_referenced = 0; - for (size_t i = 0; i < scr_cands.n; i++) scr_mark_gray(scr_cands.v[i], 0); - while (scr_pending.n) { - void *obj = scr_pending.v[--scr_pending.n]; - scr_gray_holder = obj; - scr_cyc_type(scr_cyc_hdr(obj))->trace(obj, scr_mg_visit, NULL); + scr_live_ranges.n = 0; + /* Components. Each candidate that no earlier walk reached starts a walk + * of its own, and the objects that walk grays first form its component. + * A component whose objects all end the walk at count zero is garbage: + * a count of zero means every reference to the object was trial-deleted, + * i.e. came from an object grayed so far. An earlier component cannot + * hold one (the walk that grayed it would have grayed the target too), + * so every reference comes from inside the component, and a later walk + * cannot add one: its objects would be referencing this component, and + * those references are still counted. scan would whiten exactly this + * set, so it is doomed at once and scan runs only over the components + * that are externally referenced, if any. A limited pass keeps the single + * whole-pass decision below (its untaken candidates cannot carry the + * component number). */ + bool split = scr_cyc_take == NULL; + uint32_t comp = 0; + for (size_t i = 0; i < scr_cands.n; i++) { + void *obj = scr_cands.v[i]; + if (scr_cyc_color(scr_cyc_hdr(obj)) != SCR_CYC_BLACK) continue; /* walked already */ + size_t start = scr_white.n; + scr_gray_comp = split ? ++comp : 0; + scr_comp_referenced = 0; + scr_mark_gray_from(obj); + if (!split) continue; + size_t end = scr_white.n; + if (scr_comp_referenced == 0 && end < SCR_CYC_GRAY_LIST_MAX) { + for (size_t j = start; j < end; j++) scr_cyc_set_color(scr_cyc_hdr(scr_white.v[j]), SCR_CYC_DOOMED); + } else if (scr_live_ranges.n && (uintptr_t)scr_live_ranges.v[scr_live_ranges.n - 1] == start) { + scr_live_ranges.v[scr_live_ranges.n - 1] = (void *)(uintptr_t)end; + } else { + scr_cyc_push_work(&scr_live_ranges, (void *)(uintptr_t)start); + scr_cyc_push_work(&scr_live_ranges, (void *)(uintptr_t)end); + } } - if (scr_gray_referenced == 0 && scr_white.n < SCR_CYC_GRAY_LIST_MAX) { - for (size_t i = 0; i < scr_white.n; i++) scr_cyc_set_color(scr_cyc_hdr(scr_white.v[i]), SCR_CYC_WHITE); - return scr_cyc_pass_done(scr_cyc_settle(gen_limit)); + scr_cyc_note_used(&scr_white); + /* Nothing externally referenced at all: every gray object is garbage, + * including components that looked referenced only because a later + * component held them. settle dooms the recorded set. */ + if (scr_gray_referenced == 0 && scr_white.n < SCR_CYC_GRAY_LIST_MAX) + return scr_cyc_pass_done(scr_cyc_settle(gen_limit, split && scr_live_ranges.n == 0)); + + /* scan, from the recorded set when markGray recorded every gray object: + * the survivors are exactly what the externally referenced gray objects + * reach, so blackening from each of them in turn leaves the garbage gray + * without walking it again, and settle dooms what is still gray. Only + * the referenced components' entries need looking at; the doomed ones + * are settled already. */ + if (scr_white.n < SCR_CYC_GRAY_LIST_MAX) { + /* One sweep over the list, keeping the doomed components' entries and, + * of the referenced components', only those still gray: a survivor + * leaves the list as soon as it is seen black, and an object kept here + * that a later restoration blackens is skipped by settle. */ + size_t kept = 0, from = 0; + size_t nranges = split ? scr_live_ranges.n : 2; + for (size_t k = 0; k < nranges; k += 2) { + size_t start = split ? (uintptr_t)scr_live_ranges.v[k] : 0; + size_t end = split ? (uintptr_t)scr_live_ranges.v[k + 1] : scr_white.n; + if (start > from) memmove(scr_white.v + kept, scr_white.v + from, (start - from) * sizeof *scr_white.v); + kept += start - from; + for (size_t j = start; j < end; j++) { + void *obj = scr_white.v[j]; + if (scr_cyc_color(scr_cyc_hdr(obj)) != SCR_CYC_GRAY) continue; + if (SCR_RC(obj) > 0) scr_restore_from(obj); + else scr_white.v[kept++] = obj; + } + from = end; + } + if (scr_white.n > from) memmove(scr_white.v + kept, scr_white.v + from, (scr_white.n - from) * sizeof *scr_white.v); + scr_white.n = kept + (scr_white.n - from); + return scr_cyc_pass_done(scr_cyc_settle(gen_limit, false)); } - scr_white.n = 0; - for (size_t i = 0; i < scr_cands.n; i++) scr_scan(scr_cands.v[i], 0); + /* Otherwise scan walks from the candidates. Keep only the doomed + * components' entries; scan appends what it whitens. */ + if (!split) { + scr_white.n = 0; + } else if (scr_live_ranges.n) { + size_t kept = 0, from = 0; + for (size_t k = 0; k < scr_live_ranges.n; k += 2) { + size_t start = (uintptr_t)scr_live_ranges.v[k], end = (uintptr_t)scr_live_ranges.v[k + 1]; + memmove(scr_white.v + kept, scr_white.v + from, (start - from) * sizeof *scr_white.v); + kept += start - from; + from = end; + } + memmove(scr_white.v + kept, scr_white.v + from, (scr_white.n - from) * sizeof *scr_white.v); + scr_white.n = kept + (scr_white.n - from); + } + /* Externally referenced candidates first: what they restore is never + * whitened only to be restored again. */ + for (size_t i = 0; i < scr_cands.n; i++) { + void *obj = scr_cands.v[i]; + if (scr_cyc_color(scr_cyc_hdr(obj)) == SCR_CYC_GRAY && SCR_RC(obj) > 0) scr_restore_from(obj); + } + for (size_t i = 0; i < scr_cands.n; i++) { + void *obj = scr_cands.v[i]; + if (scr_cyc_color(scr_cyc_hdr(obj)) == SCR_CYC_GRAY) scr_scan(obj, 0); + } while (scr_pending.n) { void *obj = scr_pending.v[--scr_pending.n]; - /* A later outside root may have restored this deferred white node and - * its descendants already. Never scan a black node's edges again. */ - if (scr_cyc_color(scr_cyc_hdr(obj)) == SCR_CYC_WHITE) - scr_cyc_type(scr_cyc_hdr(obj))->trace(obj, scr_scan_visit, NULL); + /* A later outside root may have restored this deferred white object and + * its descendants already. Never scan a black object's edges again. */ + ScrCycHdr *h = scr_cyc_hdr(obj); + if (scr_cyc_color(h) != SCR_CYC_WHITE) continue; + const ScrCycType *type = scr_cyc_type(h); + SCR_CYC_CHILDREN(obj, type, scr_scan_array, scr_scan_visit, 1); } - return scr_cyc_pass_done(scr_cyc_settle(gen_limit)); + return scr_cyc_pass_done(scr_cyc_settle(gen_limit, false)); } #ifdef SCR_CYC_TESTING @@ -1425,7 +1625,7 @@ void scr_collect_cycles(void) { void scr_cyc_context_cleanup(void) { scr_collect_cycles(); ScrVec *vectors[] = {&scr_roots[SCR_CYC_NURSERY], &scr_roots[SCR_CYC_MATURE], - &scr_roots[SCR_CYC_OLD], &scr_cands, &scr_promote, &scr_pending, &scr_restore_pending, &scr_white, &scr_xgen, &scr_xgen_edges, + &scr_roots[SCR_CYC_OLD], &scr_cands, &scr_promote, &scr_pending, &scr_restore_pending, &scr_white, &scr_xgen, &scr_xgen_edges, &scr_live_ranges, &scr_pub_holes[SCR_CYC_NURSERY], &scr_pub_holes[SCR_CYC_MATURE], &scr_pub_holes[SCR_CYC_OLD]}; for (size_t i = 0; i < sizeof vectors / sizeof vectors[0]; i++) { free(vectors[i]->v); diff --git a/packages/runtime/src/scr_lib.c b/packages/runtime/src/scr_lib.c index 9307d9b01..fd51e9c78 100644 --- a/packages/runtime/src/scr_lib.c +++ b/packages/runtime/src/scr_lib.c @@ -824,6 +824,26 @@ static void scr_node_debug_capture(const ScrStr *name) { if (name->len == 10 && memcmp(name->data, "NODE_DEBUG", 10) == 0) (void)scr_node_debug_initial(); } +/* Counts writes and deletes of process.env.TZ. The local-time getters keep + * their last conversion for the generation it was made in. */ +#ifdef SCR_WORKERS +static atomic_uint scr_date_tz_generation; +#else +static unsigned scr_date_tz_generation; +#endif + +/* After a write or delete of the process's TZ, local time follows the new + * value, as in Node: tzset(3) rereads it for later localtime calls. */ +static void scr_env_tz_changed(const ScrStr *name) { + /* Compared by character: a string literal would join the merged + * read-only strings of every program that links this file. */ + if (name->len != 2 || name->data[0] != 'T' || name->data[1] != 'Z') return; +#if !defined(_WIN32) && !defined(__wasi__) + tzset(); +#endif + scr_date_tz_generation++; +} + /* process.env.NAME = v — setenv(3): later scr_env_get reads and spawned * children (posix_spawn inherits environ) observe the write, like Node. * Both args borrowed (NUL-terminated ScrStr data); setenv copies. */ @@ -842,6 +862,7 @@ void scr_env_set(const ScrStr *name, const ScrStr *value) { #else setenv(k, v, 1); #endif + scr_env_tz_changed(name); #ifdef SCR_WORKERS done: #endif @@ -861,6 +882,7 @@ void scr_env_unset(const ScrStr *name) { #else unsetenv(name->data); #endif + scr_env_tz_changed(name); } static ScrStr *scr_fs_read_file_core(ScrStr *path, bool env_file, size_t *ascii); @@ -10475,9 +10497,14 @@ typedef struct ScrDateParts { * endpoint. The checked wrapper is the public UTC-getter gate; the inner * walk accepts those bounded local-time values too. */ static void scr_date_utc_parts_unchecked(double t, ScrDateParts *out) { - double dayd = floor(t / 86400000.0); - long long msday = (long long)(t - dayd * 86400000.0); - long long z = (long long)dayd + 719468; + /* t is an integral number of milliseconds within about 2^53. */ + long long tms = (long long)t; + long long day = tms / 86400000, msday = tms % 86400000; + if (msday < 0) { + msday += 86400000; + day--; + } + long long z = day + 719468; long long era = (z >= 0 ? z : z - 146096) / 146097; unsigned long long doe = (unsigned long long)(z - era * 146097); unsigned long long yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365; @@ -10487,7 +10514,7 @@ static void scr_date_utc_parts_unchecked(double t, ScrDateParts *out) { unsigned long long d = doy - (153 * mp + 2) / 5 + 1; unsigned long long m = mp < 10 ? mp + 3 : mp - 9; if (m <= 2) y++; - long long wday = ((long long)dayd + 4) % 7; + long long wday = (day + 4) % 7; if (wday < 0) wday += 7; out->year = y; out->month = (int)m - 1; @@ -10516,10 +10543,39 @@ static bool scr_date_localtime(double secd, struct tm *out) { #endif } +/* The thread's last local conversion: reading several local fields of one + * Date (getFullYear, getMonth, getDate, getHours, ...) converts it once. + * A change of process.env.TZ starts a new generation. */ +typedef struct { + bool valid; + unsigned generation; + double ms; + ScrDateParts parts; +} ScrDateLocalCache; +static SCR_TL ScrDateLocalCache scr_date_local_cache; + +static bool scr_date_local_parts_uncached(double clipped, ScrDateParts *out); + static bool scr_date_local_parts(double ms, ScrDateParts *out) { if (!isfinite(ms) || fabs(ms) > 8640000000000000.0) return false; double clipped = trunc(ms); - double secd = floor(clipped / 1000.0); + ScrDateLocalCache *c = &scr_date_local_cache; + unsigned generation = scr_date_tz_generation; + if (c->valid && c->ms == clipped && c->generation == generation) { + *out = c->parts; + return true; + } + if (!scr_date_local_parts_uncached(clipped, out)) return false; + c->valid = true; + c->generation = generation; + c->ms = clipped; + c->parts = *out; + return true; +} + +static bool scr_date_local_parts_uncached(double clipped, ScrDateParts *out) { + long long cms = (long long)clipped; + double secd = (double)(cms / 1000 - (cms % 1000 < 0)); struct tm tmv; double basis_secd = secd; if (!scr_date_localtime(basis_secd, &tmv)) { diff --git a/packages/runtime/src/scr_map.c b/packages/runtime/src/scr_map.c index 8e34533ad..14181968c 100644 --- a/packages/runtime/src/scr_map.c +++ b/packages/runtime/src/scr_map.c @@ -77,7 +77,8 @@ static inline uint64_t scr_map_hash_str(const ScrStr *k) { uint32_t h = k->hash; if (__builtin_expect(h != 0, 1)) return scr_map_hash_widen(h); h = scr_map_hash_fold(k->data, k->len); - if (k->rc != SIZE_MAX) ((ScrStr *)k)->hash = h; + /* A scratch slice is a writable stack copy (scr_str_is_scratch). */ + if (k->rc != SIZE_MAX || scr_str_is_scratch(k)) ((ScrStr *)k)->hash = h; return scr_map_hash_widen(h); #else return scr_map_hash_bytes(k->data, k->len); @@ -931,7 +932,11 @@ static void scr_map_append(ScrMap *m, uint64_t hash, uint64_t key, uint64_t val, m->filter |= scr_map_filter_bits(hash); m->nlive++; if (m->key_kind == SCR_MAP_KEY_STR) { - scr_str_retain((ScrStr *)scr_map_slot_to_ptr(key)); /* key is borrowed */ + ScrStr *k = scr_map_slot_to_ptr(key); /* key is borrowed */ + if (__builtin_expect(scr_str_is_scratch(k), 0)) + m->entries[idx].key = scr_map_slot_from_ptr(scr_str_scratch_keep(k)); + else + scr_str_retain(k); } else if (m->key_kind == SCR_MAP_KEY_DYN) { const ScrDyn *d = (ScrDyn *)scr_map_slot_to_ptr(key); if (d->kind == SCR_DYN_NUM && d->v.num == 0) { diff --git a/packages/runtime/src/scr_os.c b/packages/runtime/src/scr_os.c index 76f69588e..2e70d0d85 100644 --- a/packages/runtime/src/scr_os.c +++ b/packages/runtime/src/scr_os.c @@ -27,6 +27,8 @@ #include #include #endif +#include +#include #ifdef __linux__ #include #endif @@ -1332,3 +1334,195 @@ ScrDyn *scr_os_constants(void) { scr_atexit(scr_os_constants_clear); return scr_dyn_retain(os); } + +/* ── node:constants (deprecated, DEP0008) ────────────────────────────── + * Node's lib/constants.js: Object.assign({}, os.constants.dlopen, errno, + * priority, signals, fs.constants, crypto.constants), frozen, with an + * ordinary prototype. The fs group follows node_constants.cc's + * DefineFsConstants order and guards (libuv's UV_* values are fixed on + * POSIX); crypto_table (borrowed) is a compiler-emitted function that + * answers the baked crypto.constants table, called only on first use. One + * object per thread. */ +#define SCR_FS_CONSTANT(o, name) scr_os_constant((o), #name, (double)(name)) + +static void scr_fs_constants(ScrDyn *o) { + scr_os_constant(o, "UV_FS_SYMLINK_DIR", 1); + scr_os_constant(o, "UV_FS_SYMLINK_JUNCTION", 2); +#ifdef O_RDONLY + SCR_FS_CONSTANT(o, O_RDONLY); +#endif +#ifdef O_WRONLY + SCR_FS_CONSTANT(o, O_WRONLY); +#endif +#ifdef O_RDWR + SCR_FS_CONSTANT(o, O_RDWR); +#endif + scr_os_constant(o, "UV_DIRENT_UNKNOWN", 0); + scr_os_constant(o, "UV_DIRENT_FILE", 1); + scr_os_constant(o, "UV_DIRENT_DIR", 2); + scr_os_constant(o, "UV_DIRENT_LINK", 3); + scr_os_constant(o, "UV_DIRENT_FIFO", 4); + scr_os_constant(o, "UV_DIRENT_SOCKET", 5); + scr_os_constant(o, "UV_DIRENT_CHAR", 6); + scr_os_constant(o, "UV_DIRENT_BLOCK", 7); +#ifdef S_IFMT + SCR_FS_CONSTANT(o, S_IFMT); +#endif +#ifdef S_IFREG + SCR_FS_CONSTANT(o, S_IFREG); +#endif +#ifdef S_IFDIR + SCR_FS_CONSTANT(o, S_IFDIR); +#endif +#ifdef S_IFCHR + SCR_FS_CONSTANT(o, S_IFCHR); +#endif +#ifdef S_IFBLK + SCR_FS_CONSTANT(o, S_IFBLK); +#endif +#ifdef S_IFIFO + SCR_FS_CONSTANT(o, S_IFIFO); +#endif +#ifdef S_IFLNK + SCR_FS_CONSTANT(o, S_IFLNK); +#endif +#ifdef S_IFSOCK + SCR_FS_CONSTANT(o, S_IFSOCK); +#endif +#ifdef O_CREAT + SCR_FS_CONSTANT(o, O_CREAT); +#endif +#ifdef O_EXCL + SCR_FS_CONSTANT(o, O_EXCL); +#endif + /* libuv's Windows-only open flags are 0 everywhere else. */ + scr_os_constant(o, "UV_FS_O_FILEMAP", 0); + scr_os_constant(o, "UV_FS_O_TEMPORARY", 0); + scr_os_constant(o, "UV_FS_O_SHORT_LIVED", 0); + scr_os_constant(o, "UV_FS_O_SEQUENTIAL", 0); + scr_os_constant(o, "UV_FS_O_RANDOM", 0); +#ifdef O_NOCTTY + SCR_FS_CONSTANT(o, O_NOCTTY); +#endif +#ifdef O_TRUNC + SCR_FS_CONSTANT(o, O_TRUNC); +#endif +#ifdef O_APPEND + SCR_FS_CONSTANT(o, O_APPEND); +#endif +#ifdef O_DIRECTORY + SCR_FS_CONSTANT(o, O_DIRECTORY); +#endif +#ifdef O_NOATIME + SCR_FS_CONSTANT(o, O_NOATIME); +#endif +#ifdef O_NOFOLLOW + SCR_FS_CONSTANT(o, O_NOFOLLOW); +#endif +#ifdef O_SYNC + SCR_FS_CONSTANT(o, O_SYNC); +#endif +#ifdef O_DSYNC + SCR_FS_CONSTANT(o, O_DSYNC); +#endif +#ifdef O_SYMLINK + SCR_FS_CONSTANT(o, O_SYMLINK); +#endif +#ifdef O_DIRECT + SCR_FS_CONSTANT(o, O_DIRECT); +#endif +#ifdef O_NONBLOCK + SCR_FS_CONSTANT(o, O_NONBLOCK); +#endif +#ifdef S_IRWXU + SCR_FS_CONSTANT(o, S_IRWXU); +#endif +#ifdef S_IRUSR + SCR_FS_CONSTANT(o, S_IRUSR); +#endif +#ifdef S_IWUSR + SCR_FS_CONSTANT(o, S_IWUSR); +#endif +#ifdef S_IXUSR + SCR_FS_CONSTANT(o, S_IXUSR); +#endif +#ifdef S_IRWXG + SCR_FS_CONSTANT(o, S_IRWXG); +#endif +#ifdef S_IRGRP + SCR_FS_CONSTANT(o, S_IRGRP); +#endif +#ifdef S_IWGRP + SCR_FS_CONSTANT(o, S_IWGRP); +#endif +#ifdef S_IXGRP + SCR_FS_CONSTANT(o, S_IXGRP); +#endif +#ifdef S_IRWXO + SCR_FS_CONSTANT(o, S_IRWXO); +#endif +#ifdef S_IROTH + SCR_FS_CONSTANT(o, S_IROTH); +#endif +#ifdef S_IWOTH + SCR_FS_CONSTANT(o, S_IWOTH); +#endif +#ifdef S_IXOTH + SCR_FS_CONSTANT(o, S_IXOTH); +#endif +#ifdef F_OK + SCR_FS_CONSTANT(o, F_OK); +#endif +#ifdef R_OK + SCR_FS_CONSTANT(o, R_OK); +#endif +#ifdef W_OK + SCR_FS_CONSTANT(o, W_OK); +#endif +#ifdef X_OK + SCR_FS_CONSTANT(o, X_OK); +#endif + scr_os_constant(o, "UV_FS_COPYFILE_EXCL", 1); + scr_os_constant(o, "COPYFILE_EXCL", 1); + scr_os_constant(o, "UV_FS_COPYFILE_FICLONE", 2); + scr_os_constant(o, "COPYFILE_FICLONE", 2); + scr_os_constant(o, "UV_FS_COPYFILE_FICLONE_FORCE", 4); + scr_os_constant(o, "COPYFILE_FICLONE_FORCE", 4); +} +#undef SCR_FS_CONSTANT + +static SCR_TL ScrDyn *scr_node_constants_value; +static void scr_node_constants_clear(void) { + ScrDyn *value = scr_node_constants_value; + scr_node_constants_value = NULL; + scr_dyn_release(value); +} + +ScrDyn *scr_node_constants(const ScrDyn *crypto_table) { + if (scr_node_constants_value) return scr_dyn_retain(scr_node_constants_value); + ScrDyn *crypto = scr_dyn_call(crypto_table, NULL, 0, "constants"); + if (!crypto) return NULL; + ScrDyn *o = scr_dyn_new_obj(); + scr_os_dlopen_constants(o); + scr_os_errno_constants(o); + scr_os_constant(o, "PRIORITY_LOW", SCR_OS_PRIORITY_LOW); + scr_os_constant(o, "PRIORITY_BELOW_NORMAL", SCR_OS_PRIORITY_BELOW_NORMAL); + scr_os_constant(o, "PRIORITY_NORMAL", SCR_OS_PRIORITY_NORMAL); + scr_os_constant(o, "PRIORITY_ABOVE_NORMAL", SCR_OS_PRIORITY_ABOVE_NORMAL); + scr_os_constant(o, "PRIORITY_HIGH", SCR_OS_PRIORITY_HIGH); + scr_os_constant(o, "PRIORITY_HIGHEST", SCR_OS_PRIORITY_HIGHEST); + scr_os_signal_constants(o); + scr_fs_constants(o); + if (crypto && crypto->kind == SCR_DYN_OBJ) { + for (size_t i = 0; i < crypto->v.obj.len; i++) { + const ScrDynEntry *entry = &crypto->v.obj.entries[i]; + if (entry->accessor || !entry->value) continue; + scr_dyn_obj_set(o, entry->key, entry->key_len, scr_dyn_retain(entry->value)); + } + } + scr_dyn_release(crypto); + scr_dyn_release(scr_dyn_freeze(o)); + scr_node_constants_value = o; + scr_atexit(scr_node_constants_clear); + return scr_dyn_retain(o); +} diff --git a/packages/runtime/src/scr_os.h b/packages/runtime/src/scr_os.h index 3c5009fc5..219726660 100644 --- a/packages/runtime/src/scr_os.h +++ b/packages/runtime/src/scr_os.h @@ -38,6 +38,10 @@ ScrStr *scr_os_cpus_model(double i); double scr_os_cpus_field(double i, double field); /* os.constants: the per-thread null-prototype constants object (+1). */ ScrDyn *scr_os_constants(void); +/* The deprecated node:constants module object (scr_os.c): the os groups, + * fs.constants and the crypto table the borrowed function answers, merged + * and frozen (+1). */ +ScrDyn *scr_node_constants(const ScrDyn *crypto_table); double scr_os_available_parallelism(void); /* uv_available_parallelism: CPUs this process may use, >= 1 */ /* os.userInfo()'s field trio (pw_name / pw_shell / pw_dir — the passwd * home, not the $HOME cascade). +1 fresh; abort on lookup failure. */ diff --git a/packages/runtime/src/scr_rc.h b/packages/runtime/src/scr_rc.h index cdc03c966..d6e1f8e88 100644 --- a/packages/runtime/src/scr_rc.h +++ b/packages/runtime/src/scr_rc.h @@ -92,6 +92,9 @@ typedef struct ScrCycType { * SCR_CYC_KIND_EMITTED (llvm/alloc.ts FIRST_EMITTED_CYC_KIND). */ enum { SCR_CYC_KIND_NONE = 0, + /* Also how the collector recognizes a cycle-capable ScrArr, whose dense + * elements it visits inline (scr_cycle.c SCR_CYC_CHILDREN): reserved for + * scr_array.c's descriptor. */ SCR_CYC_KIND_ARRAY, SCR_CYC_KIND_PROMISE, SCR_CYC_KIND_BOX, diff --git a/packages/runtime/src/scr_string.c b/packages/runtime/src/scr_string.c index 6f3529777..43cf21b0c 100644 --- a/packages/runtime/src/scr_string.c +++ b/packages/runtime/src/scr_string.c @@ -1999,20 +1999,34 @@ ScrStr *scr_str_slice(ScrStr *s, double start, double end) { return scr_str_slice_units(s, e, from, to); } +/* One slice() or substring() boundary in [0, length]: ToIntegerOrInfinity, + * then slice's end-relative negative or substring's clamp. Conversion to an + * integer truncates toward zero exactly like ToIntegerOrInfinity, so no + * libm call is needed; NaN fails both comparisons and lands on 0, and a + * negative fraction above -1 is -0, which slice treats as 0. */ +static inline size_t scr_slice_unit(double v, size_t length, bool substring) { + if (v >= 0) return v >= (double)length ? length : (size_t)v; + if (substring || !(v > -(double)length)) return 0; + long long unit = (long long)v; /* in (-length, 0] */ + return unit == 0 ? 0 : (size_t)((long long)length + unit); +} + +/* slice()/substring() boundaries in UTF-16 units over the receiver's length. */ +static void scr_str_slice_bounds(ScrStr *s, ScrSidx *e, double start, double end, bool substring, + size_t *from_out, size_t *to_out) { + size_t length = scr_sidx_len(s, e); + size_t from = scr_slice_unit(start, length, substring); + size_t to = scr_slice_unit(end, length, substring); + if (substring && from > to) { size_t swap = from; from = to; to = swap; } + *from_out = from; + *to_out = to; +} + void scr_str_slice_range(ScrStr *s, double start, double end, bool substring, ScrStringSlice *out) { ScrSidx *e = scr_sidx_for(s); - size_t length = scr_sidx_len(s, e); - double a = scr_to_integer_or_infinity(start), b = scr_to_integer_or_infinity(end); size_t from, to; - if (substring) { - from = a <= 0 ? 0 : a >= (double)length ? length : (size_t)a; - to = b <= 0 ? 0 : b >= (double)length ? length : (size_t)b; - if (from > to) { size_t swap = from; from = to; to = swap; } - } else { - from = scr_slice_boundary(a, length); - to = scr_slice_boundary(b, length); - } + scr_str_slice_bounds(s, e, start, end, substring, &from, &to); out->start = from; out->length = to > from ? to - from : 0; out->split = 0; @@ -2024,6 +2038,56 @@ void scr_str_slice_range(ScrStr *s, double start, double end, bool substring, } } +/* Fill a scratch buffer with `units` ASCII bytes (scr_str_is_scratch). */ +static inline ScrStr *scr_str_scratch_fill(ScrStr *scratch, const char *data, size_t units) { + scratch->rc = SIZE_MAX; + scratch->len = units; + scratch->cap = 0; /* below len: the scratch marker */ + scratch->ascii = 1; +#if SCR_STR_HASH_CACHE + scratch->hash = 0; +#endif + scr_copy_bytes(scratch->data, data, units); + scratch->data[units] = '\0'; + return scratch; +} + +ScrStr *scr_str_slice_scratch(ScrStr *s, double start, double end, bool substring, + ScrStr *scratch, size_t bytes) { + size_t capacity = bytes - sizeof(ScrStr) - 1; + if (__builtin_expect(s->ascii, 1)) { + /* Units are bytes: no index entry, and the bounds are byte offsets. */ + size_t length = s->len; + size_t from = scr_slice_unit(start, length, substring); + size_t to = scr_slice_unit(end, length, substring); + if (substring && from > to) { size_t swap = from; from = to; to = swap; } + size_t units = to > from ? to - from : 0; + /* Shorter results are interned statics. */ + if (units >= 2 && units <= capacity) return scr_str_scratch_fill(scratch, s->data + from, units); + return scr_str_slice_units(s, NULL, from, to); + } + ScrSidx *e = scr_sidx_for(s); + size_t from, to; + scr_str_slice_bounds(s, e, start, end, substring, &from, &to); + size_t units = to > from ? to - from : 0; + if (units >= 2 && units <= capacity) { + /* Equal unit and byte counts between two character boundaries prove + * the span is ASCII: every other character takes more bytes than + * UTF-16 units. Non-ASCII results keep the owned copy and its index. */ + bool first, last; + size_t from_b = scr_u16_to_byte_c(s, e, from, &first); + size_t to_b = scr_u16_to_byte_c(s, e, to, &last); + if (!first && !last && to_b - from_b == units) + return scr_str_scratch_fill(scratch, s->data + from_b, units); + } + return scr_str_slice_units(s, e, from, to); +} + +ScrStr *scr_str_scratch_keep(ScrStr *s) { + if (!scr_str_is_scratch(s)) return scr_str_retain(s); + return scr_str_from_span_ascii(s->data, s->len); +} + double scr_str_slice_char_code_at(ScrStr *s, const ScrStringSlice *range, double index) { double unit = scr_to_integer_or_infinity(index); if (!(unit >= 0) || unit >= (double)range->length) return NAN; diff --git a/packages/runtime/src/scr_string.h b/packages/runtime/src/scr_string.h index e4bf09995..425dfaabf 100644 --- a/packages/runtime/src/scr_string.h +++ b/packages/runtime/src/scr_string.h @@ -184,6 +184,14 @@ static inline ScrStr *scr_str_retain(ScrStr *s) { void scr_str_release(ScrStr *s); /* NULL-tolerant (uninitialized locals) */ +/* A transient stack copy of a slice (scr_str_slice_scratch): immortal for + * retain and release, and valid only while its borrowing consumer runs. No + * other string has a capacity below its length. Containers that keep a + * string they were lent store scr_str_scratch_keep's copy instead. */ +static inline bool scr_str_is_scratch(const ScrStr *s) { + return s->rc == SIZE_MAX && s->cap < s->len; +} + /* ── lone surrogates: canonical WTF-8 storage ───────────────────────── * String storage is UTF-8 extended to lone UTF-16 surrogates: an unpaired * surrogate is its generalized 3-byte encoding ED A0..BF 80..BF, while a diff --git a/packages/runtime/src/scr_string_methods.h b/packages/runtime/src/scr_string_methods.h index 537747530..62354142e 100644 --- a/packages/runtime/src/scr_string_methods.h +++ b/packages/runtime/src/scr_string_methods.h @@ -83,6 +83,16 @@ typedef struct ScrStringSlice { void scr_str_slice_range(ScrStr *s, double start, double end, bool substring, ScrStringSlice *out); double scr_str_slice_char_code_at(ScrStr *s, const ScrStringSlice *range, double index); +/* A slice that only borrowing consumers read (string-slices.ts) is written + * into the caller's `bytes`-sized stack buffer as an immortal ASCII string, + * whose header marks it as a scratch copy (scr_str_is_scratch). Results + * that do not fit, are not ASCII, or are interned statics are returned as + * the ordinary +1 slice. Releasing either result is correct. */ +ScrStr *scr_str_slice_scratch(ScrStr *s, double start, double end, bool substring, + ScrStr *scratch, size_t bytes); +/* The owning reference a container stores for `s`: a heap copy of a scratch + * string, else `s` retained. */ +ScrStr *scr_str_scratch_keep(ScrStr *s); /* repeat(count): count < 0 or Infinity is a JS RangeError; scriptc has no * exceptions, so it prints "scriptc: RangeError: Invalid count value" to diff --git a/packages/runtime/src/scr_test.c b/packages/runtime/src/scr_test.c index cb2c81971..cc52513a6 100644 --- a/packages/runtime/src/scr_test.c +++ b/packages/runtime/src/scr_test.c @@ -80,6 +80,7 @@ enum { typedef struct { ScrClosure *cb; int flags; + ScrStr *at; /* "file:line:col" of the hook call, owned; NULL for t.* hooks */ } ScrTestHook; typedef struct { @@ -123,6 +124,7 @@ struct ScrTestCtx { ScrPromise *slot; /* owned; the parent waiting for its next subtest to finish */ struct ScrMockTimers *mock_timers; /* t.mock.timers, made on first use; freed at exit */ bool passed; /* t.passed: set when the body passes, as Node's pass() does */ + bool listed; /* a suite listed in the failing section (a failing root before) */ }; static SCR_TL ScrTestCtx *scr_test_root = NULL; @@ -136,6 +138,18 @@ static SCR_TL long scr_test_ct_tests = 0, scr_test_ct_suites = 0, scr_test_ct_pa scr_test_ct_todo = 0; static SCR_TL double scr_test_t0 = 0; static SCR_TL bool scr_test_suite_hook_failed = false; +/* The promises of async module-level before hooks, which the runner waits + * for before the first top-level test. */ +static SCR_TL ScrPromise **scr_test_root_pending = NULL; +static SCR_TL size_t scr_test_root_npending = 0, scr_test_root_pending_cap = 0; + +/* A module-level before hook threw or rejected: the first failure is kept + * for the top-level tests and suites, later ones are dropped. */ +static void scr_test_root_before_failed(ScrTestCtx *root) { + if (!scr_exc_pending()) return; + if (!root->before_err) root->before_err = scr_exc_take(); + else scr_exc_clear(); +} /* Above 0 while subtests run concurrently: result lines are held, and the * outermost concurrent parent prints its subtrees in declaration order, * as Node's reporter does, once they all finish. */ @@ -164,7 +178,10 @@ ScrTestCtx *scr_testctx_retain(ScrTestCtx *t) { } static void scr_test_hooks_free(ScrTestHooks *h) { - for (size_t i = 0; i < h->n; i++) scr_closure_release(h->v[i].cb); + for (size_t i = 0; i < h->n; i++) { + scr_closure_release(h->v[i].cb); + scr_str_release(h->v[i].at); + } free(h->v); h->v = NULL; h->n = h->cap = 0; @@ -277,7 +294,9 @@ static void scr_test_indent(int depth) { /* "(1.234567ms)" — durations print microsecond-rounded (Node's hrtime * granularity), not with double noise. */ +/* NaN prints nothing: a test cancelled before it started has no duration. */ static void scr_test_print_dur(double ms) { + if (isnan(ms)) return; char num[32]; fputs(" (", stdout); size_t n = scr_f64_to_str(round(ms * 1e6) / 1e6, num); @@ -409,10 +428,14 @@ static void scr_test_mocks_restore(ScrTestHooks *mocks, bool forget) { /* `ctx` is the context a hook that takes one receives: the suite or test * for its own before/after hooks, the test for the beforeEach/afterEach * chain. */ -static void scr_test_run_hooks_of(ScrTestCtx *owner, int which, ScrTestCtx *ctx) { - for (size_t i = 0; i < owner->hooks[which].n && !scr_exc_pending(); i++) { - scr_test_call(owner->hooks[which].v[i].cb, owner->hooks[which].v[i].flags, ctx, NULL); +static ScrTestHook *scr_test_run_hooks_of(ScrTestCtx *owner, int which, ScrTestCtx *ctx) { + if (scr_exc_pending()) return NULL; /* an earlier hook in the chain threw */ + for (size_t i = 0; i < owner->hooks[which].n; i++) { + ScrTestHook *h = &owner->hooks[which].v[i]; + scr_test_call(h->cb, h->flags, ctx, NULL); + if (scr_exc_pending()) return h; /* the hook that threw */ } + return NULL; } /* Returns whether the chain has any `which` hooks. */ @@ -559,6 +582,36 @@ static void scr_test_finish_concurrent(ScrTestCtx *t) { for (size_t i = 0; i < t->nchildren; i++) scr_test_report_tree(t->children[i]); } +/* A failing root before hook (Node): each top-level test fails with its + * error without running its body or beforeEach hooks, though its + * afterEach hooks run; a top-level suite fails with it, and the tests and + * suites in it are cancelled before they start. */ +static void scr_test_cancel_unrun(ScrTestCtx *c) { + for (size_t i = 0; i < c->nchildren; i++) scr_test_cancel_unrun(c->children[i]); + static const char msg[] = "test did not finish before its parent and was cancelled"; + scr_throw_str(scr_str_new(msg, sizeof msg - 1)); + c->err = scr_exc_take(); + c->listed = true; + c->dur_ms = NAN; + c->state = SCR_TS_CANCELLED; + scr_test_report_result(c); +} + +static void scr_test_fail_unrun(ScrTestCtx *c, ScrCaught *err) { + double start = scr_now_ms(); + c->err = scr_caught_retain(err); + c->listed = true; + if (c->is_suite) { + for (size_t i = 0; i < c->nchildren; i++) scr_test_cancel_unrun(c->children[i]); + } else { + scr_test_run_each_hooks(c, SCR_HOOK_AFTER_EACH, false); + if (scr_exc_pending()) scr_exc_clear(); + } + c->dur_ms = scr_now_ms() - start; + c->state = SCR_TS_FAIL; + scr_test_report_result(c); +} + static void scr_test_run_children(ScrTestCtx *t) { /* `only` at this level: when a child is marked, or contains a marked * test or suite, only those children run. The others are dropped entirely @@ -572,6 +625,10 @@ static void scr_test_run_children(ScrTestCtx *t) { /* Subtests t.test already ran INLINE at their call site, or started on * their own fiber. */ if (c->state != SCR_TS_PENDING || c->done) continue; + if (t == scr_test_root && t->before_err) { + scr_test_fail_unrun(c, t->before_err); + continue; + } if (t->concurrency > 1) scr_test_start(c); else scr_test_run_node(c); } @@ -813,7 +870,7 @@ static void scr_test_print_failures(ScrTestCtx *t, bool *any) { for (size_t i = 0; i < t->nchildren; i++) { ScrTestCtx *c = t->children[i]; scr_test_print_failures(c, any); - if (!c->is_suite && + if ((!c->is_suite || c->listed) && (c->state == SCR_TS_FAIL || c->state == SCR_TS_TODO_FAIL || c->state == SCR_TS_CANCELLED) && c->err) { @@ -1404,7 +1461,41 @@ static void scr_test_runner_entry(ScrFiber *self, void *argpack) { * the first test only after the file's synchronous body completes. */ scr_await_hop(); ScrTestCtx *root = scr_test_root; + /* The root's before hooks ran at their before() calls; the first test + * waits for the async ones. */ + for (size_t i = 0; i < scr_test_root_npending; i++) { + scr_await_void(scr_test_root_pending[i]); + scr_test_root_before_failed(root); + scr_promise_release(scr_test_root_pending[i]); + } + free(scr_test_root_pending); + scr_test_root_pending = NULL; + scr_test_root_npending = scr_test_root_pending_cap = 0; scr_test_run_children(root); + /* Its after hooks: a throw reports the root as "✖ " with the hook's + * location, after the top-level results, without counting it or changing + * the exit code, as Node does. */ + double after_start = scr_now_ms(); + ScrTestHook *after_failed = scr_test_run_hooks_of(root, SCR_HOOK_AFTER, root); + if (scr_exc_pending()) { + root->err = scr_exc_take(); + root->dur_ms = scr_now_ms() - after_start; + root->state = SCR_TS_FAIL; + if (after_failed && after_failed->at) { + root->at = scr_str_retain(after_failed->at); + /* The name is the file: the location without its ":line:col". */ + size_t end = root->at->len; + for (int colons = 0; end > 0 && colons < 2; end--) { + if (root->at->data[end - 1] == ':') colons++; + } + scr_str_release(root->name); + root->name = scr_str_new(root->at->data, end); + } + fputs("\xE2\x9C\x96 ", stdout); + scr_test_print_str(root->name); + scr_test_print_dur(root->dur_ms); + fputc('\n', stdout); + } scr_test_run_done = true; scr_snap_write_all(); scr_test_count(root); @@ -1418,6 +1509,10 @@ static void scr_test_runner_entry(ScrFiber *self, void *argpack) { scr_test_info_line("duration_ms", round((scr_now_ms() - scr_test_t0) * 1e6) / 1e6); bool any = false; scr_test_print_failures(root, &any); + if (root->err) { + if (!any) fputs("\n\xE2\x9C\x96 failing tests:\n", stdout); + scr_test_print_failure(root); + } /* The tree is done reporting: release it (test bodies may still hold * their TestContext refs — the boxes stay alive through those). */ scr_test_reg_parent = NULL; @@ -1534,7 +1629,41 @@ void scr_test_suite(ScrStr *name /*borrowed*/, double mode, ScrStr *msg /*borrow scr_closure_release(cb); } -void scr_test_hook(double which, ScrClosure *cb /*moves*/, double flags) { +/* A module-level before hook runs at its before() call, as Node runs the + * hooks of its already-started root at once: the synchronous part runs + * now, on the caller's stack, and the runner waits for an async hook's + * promise before the first top-level test. The first failure fails the + * top-level tests and suites (scr_test_fail_unrun). */ +static void scr_test_root_before_now(ScrTestCtx *root, ScrClosure *cb, int flags) { + bool takes_ctx = (flags & SCR_TEST_F_CTX) != 0; + if (flags & SCR_TEST_F_ASYNC) { + ScrPromise *p = + takes_ctx + ? ((ScrPromise * (*)(ScrClosure *, ScrTestCtx *)) cb->fn)(cb, scr_testctx_retain(root)) + : ((ScrPromise * (*)(ScrClosure *)) cb->fn)(cb); + if (p && !scr_exc_pending()) { + if (scr_test_root_npending == scr_test_root_pending_cap) { + scr_test_root_pending_cap = scr_test_root_pending_cap ? scr_test_root_pending_cap * 2 : 2; + scr_test_root_pending = realloc(scr_test_root_pending, + scr_test_root_pending_cap * sizeof *scr_test_root_pending); + if (!scr_test_root_pending) scr_test_oom(); + } + /* The runner observes it: a rejection is the hook's failure, not an + * unhandled rejection. */ + scr_promise_mark_handled(p); + scr_test_root_pending[scr_test_root_npending++] = p; + return; + } + scr_promise_release(p); + } else if (takes_ctx) { + ((void (*)(ScrClosure *, ScrTestCtx *))cb->fn)(cb, scr_testctx_retain(root)); + } else { + ((void (*)(ScrClosure *))cb->fn)(cb); + } + scr_test_root_before_failed(root); +} + +void scr_test_hook(double which, ScrClosure *cb /*moves*/, double flags, ScrStr *at) { scr_test_ensure_root(); ScrTestCtx *owner = scr_test_parent_get(); ScrTestHooks *h = &owner->hooks[(int)which]; @@ -1545,7 +1674,10 @@ void scr_test_hook(double which, ScrClosure *cb /*moves*/, double flags) { } h->v[h->n].cb = cb; h->v[h->n].flags = (int)flags; + h->v[h->n].at = at && at->len > 0 ? scr_str_retain(at) : NULL; h->n++; + if ((int)which == SCR_HOOK_BEFORE && owner == scr_test_root) + scr_test_root_before_now(owner, cb, (int)flags); } /* t.test(...): registers under `t` and — because subtests run inline on @@ -1620,6 +1752,7 @@ void scr_test_ctx_hook(ScrTestCtx *t, double which, ScrClosure *cb /*moves*/, do } h->v[h->n].cb = cb; h->v[h->n].flags = (int)flags; + h->v[h->n].at = NULL; h->n++; } @@ -1631,6 +1764,7 @@ static void scr_test_mocks_add(ScrTestHooks *h, ScrClosure *restore /*moves*/) { } h->v[h->n].cb = restore; h->v[h->n].flags = 0; + h->v[h->n].at = NULL; h->n++; } @@ -2172,6 +2306,7 @@ void scr_test_ctx_diagnostic(ScrTestCtx *t, ScrStr *msg /*borrowed*/) { } ScrStr *scr_test_ctx_name(ScrTestCtx *t) { + if (!t->name && !t->parent) return scr_str_new("", 6); /* Node's root test */ return t->name ? scr_str_retain(t->name) : scr_str_new("", 0); } diff --git a/packages/runtime/src/scr_test.h b/packages/runtime/src/scr_test.h index 26f8928f3..64a280bf7 100644 --- a/packages/runtime/src/scr_test.h +++ b/packages/runtime/src/scr_test.h @@ -33,8 +33,10 @@ void scr_test_register(ScrStr *name, double mode, ScrStr *msg, ScrClosure *cb, d void scr_test_suite(ScrStr *name, double mode, ScrStr *msg, ScrClosure *cb, ScrStr *at, double flags, double concurrency); /* before/after/beforeEach/afterEach on the CURRENT registration parent - * (`which`: 0 before, 1 after, 2 beforeEach, 3 afterEach). */ -void scr_test_hook(double which, ScrClosure *cb, double flags); + * (`which`: 0 before, 1 after, 2 beforeEach, 3 afterEach); `at` is the + * hook call's "file:line:col". The root's before and after hooks run + * around the top-level tests. */ +void scr_test_hook(double which, ScrClosure *cb, double flags, ScrStr *at); /* t.test(...): registers under `t`, runs the subtest INLINE on the * runner fiber, and returns the settled promise the `await` consumes (+1). */ ScrPromise *scr_test_sub(ScrTestCtx *t, ScrStr *name, double mode, ScrStr *msg, ScrClosure *cb, double flags, ScrStr *at, diff --git a/packages/runtime/src/scr_url.c b/packages/runtime/src/scr_url.c index eeadba98f..67de82471 100644 --- a/packages/runtime/src/scr_url.c +++ b/packages/runtime/src/scr_url.c @@ -1989,3 +1989,342 @@ ScrBytes *scr_url_checked_to_path_buffer(const ScrDyn *value, const ScrDyn *opti scr_url_release(url); return result; } + +/* ── node:punycode (deprecated, DEP0040) ─────────────────────────────── + * A port of the punycode.js 2.1.0 that Node 24 bundles as lib/punycode.js. + * It works on UTF-16 code units: ucs2.decode pairs surrogates and keeps + * lone ones, which is exactly a code-point walk over WTF-8 storage. + * Errors throw the module's RangeErrors ("Overflow: ...", "Illegal input + * >= 0x80 ...", "Invalid input"); decoded code points past U+10FFFF throw + * String.fromCodePoint's "Invalid code point N". All inputs are borrowed; + * results are +1, or NULL with the error pending. The DEP0040 report is + * scr_punycode_warn (scr_async_dyn.c). */ + +enum { + PC_MAX_INT = 2147483647, + PC_BASE = 36, + PC_TMIN = 1, + PC_TMAX = 26, + PC_SKEW = 38, + PC_DAMP = 700, + PC_INITIAL_BIAS = 72, + PC_INITIAL_N = 128 +}; + +static void pc_error(const char *message) { + scr_throw_error_msg(SCR_ERR_RANGE, message, strlen(message)); +} +#define PC_OVERFLOW "Overflow: input needs wider integers to process" +#define PC_NOT_BASIC "Illegal input >= 0x80 (not a basic code point)" +#define PC_INVALID "Invalid input" + +typedef struct { + uint32_t *data; + size_t len, cap; +} PcPoints; + +static void pc_push(PcPoints *p, uint32_t cp) { + if (p->len == p->cap) { + size_t cap = p->cap ? p->cap * 2 : 16; + uint32_t *grown = realloc(p->data, cap * sizeof *grown); + if (!grown) scr_trap("scriptc: out of memory\n"); + p->data = grown; + p->cap = cap; + } + p->data[p->len++] = cp; +} + +/* ucs2decode over WTF-8 storage: a pair is one 4-byte sequence, a lone + * surrogate one 3-byte sequence carrying its own value. */ +static void pc_code_points(const char *s, size_t n, PcPoints *out) { + const unsigned char *b = (const unsigned char *)s; + for (size_t i = 0; i < n;) { + uint32_t c = b[i]; + if (c < 0x80) { + i += 1; + } else if (c < 0xE0 && i + 1 < n) { + c = ((c & 0x1F) << 6) | (b[i + 1] & 0x3F); + i += 2; + } else if (c < 0xF0 && i + 2 < n) { + c = ((c & 0x0F) << 12) | ((uint32_t)(b[i + 1] & 0x3F) << 6) | (b[i + 2] & 0x3F); + i += 3; + } else if (i + 3 < n) { + c = ((c & 0x07) << 18) | ((uint32_t)(b[i + 1] & 0x3F) << 12) | + ((uint32_t)(b[i + 2] & 0x3F) << 6) | (b[i + 3] & 0x3F); + i += 4; + } else { + i += 1; + } + pc_push(out, c); + } +} + +static int64_t pc_adapt(int64_t delta, int64_t points, bool first) { + int64_t k = 0; + delta = first ? delta / PC_DAMP : delta >> 1; + delta += delta / points; + for (; delta > ((PC_BASE - PC_TMIN) * PC_TMAX) >> 1; k += PC_BASE) delta /= PC_BASE - PC_TMIN; + return k + (PC_BASE - PC_TMIN + 1) * delta / (delta + PC_SKEW); +} + +static int64_t pc_basic_to_digit(unsigned char c) { + if (c >= 0x30 && c < 0x3A) return 26 + (c - 0x30); + if (c >= 0x41 && c < 0x5B) return c - 0x41; + if (c >= 0x61 && c < 0x7B) return c - 0x61; + return PC_BASE; +} + +static char pc_digit_to_basic(int64_t digit) { + return (char)(digit + 22 + 75 * (digit < 26)); +} + +/* String.fromCodePoint(...points). */ +static ScrStr *pc_from_points(const PcPoints *p) { + for (size_t i = 0; i < p->len; i++) { + if (p->data[i] > 0x10FFFF) { + char message[48]; + int size = snprintf(message, sizeof message, "Invalid code point %u", (unsigned)p->data[i]); + scr_throw_error_msg(SCR_ERR_RANGE, message, (size_t)size); + return NULL; + } + } + char *out = malloc(p->len * 4 + 1); + if (!out) scr_trap("scriptc: out of memory\n"); + size_t offset = 0; + for (size_t i = 0; i < p->len; i++) { + uint32_t point = p->data[i]; + if (point >= 0xD800 && point <= 0xDBFF && i + 1 < p->len && p->data[i + 1] >= 0xDC00 && + p->data[i + 1] <= 0xDFFF) { + point = 0x10000 + ((point - 0xD800) << 10) + (p->data[i + 1] - 0xDC00); + i++; + } + offset += scr_wtf8_put(out + offset, point); + } + ScrStr *result = scr_str_new(out, offset); + free(out); + return result; +} + +/* punycode.decode over the bytes of one label. Any non-ASCII byte fails + * where its first code unit would: before the last delimiter it is not + * basic, after it it is no digit — so byte positions stand in for code + * unit positions until the first error. */ +static ScrStr *pc_decode(const char *s, size_t n) { + PcPoints output = {0}; + size_t basic = 0; + for (size_t k = n; k > 0; k--) { + if (s[k - 1] == '-') { + basic = k - 1; + break; + } + } + ScrStr *result = NULL; + for (size_t j = 0; j < basic; j++) { + if ((unsigned char)s[j] >= 0x80) { + pc_error(PC_NOT_BASIC); + goto done; + } + pc_push(&output, (unsigned char)s[j]); + } + int64_t i = 0, cp = PC_INITIAL_N, bias = PC_INITIAL_BIAS; + for (size_t index = basic > 0 ? basic + 1 : 0; index < n;) { + int64_t old = i; + for (int64_t w = 1, k = PC_BASE;; k += PC_BASE) { + if (index >= n) { + pc_error(PC_INVALID); + goto done; + } + int64_t digit = pc_basic_to_digit((unsigned char)s[index++]); + if (digit >= PC_BASE) { + pc_error(PC_INVALID); + goto done; + } + if (digit > (PC_MAX_INT - i) / w) { + pc_error(PC_OVERFLOW); + goto done; + } + i += digit * w; + int64_t t = k <= bias ? PC_TMIN : (k >= bias + PC_TMAX ? PC_TMAX : k - bias); + if (digit < t) break; + if (w > PC_MAX_INT / (PC_BASE - t)) { + pc_error(PC_OVERFLOW); + goto done; + } + w *= PC_BASE - t; + } + int64_t out = (int64_t)output.len + 1; + bias = pc_adapt(i - old, out, old == 0); + if (i / out > PC_MAX_INT - cp) { + pc_error(PC_OVERFLOW); + goto done; + } + cp += i / out; + i %= out; + pc_push(&output, 0); + memmove(output.data + i + 1, output.data + i, (output.len - 1 - (size_t)i) * sizeof *output.data); + output.data[i++] = (uint32_t)cp; + } + result = pc_from_points(&output); +done: + free(output.data); + return result; +} + +/* punycode.encode of one label's code points, appended to buf. */ +static bool pc_encode(const PcPoints *input, ScrJsonBuf *buf) { + int64_t cp = PC_INITIAL_N, delta = 0, bias = PC_INITIAL_BIAS; + size_t basic_length = 0; + for (size_t k = 0; k < input->len; k++) { + if (input->data[k] < 0x80) { + scr_jb_putc(buf, (char)input->data[k]); + basic_length++; + } + } + size_t handled = basic_length; + if (basic_length) scr_jb_putc(buf, '-'); + while (handled < input->len) { + int64_t m = PC_MAX_INT; + for (size_t k = 0; k < input->len; k++) + if (input->data[k] >= cp && input->data[k] < m) m = input->data[k]; + int64_t handled_plus_one = (int64_t)handled + 1; + if (m - cp > (PC_MAX_INT - delta) / handled_plus_one) { + pc_error(PC_OVERFLOW); + return false; + } + delta += (m - cp) * handled_plus_one; + cp = m; + for (size_t k = 0; k < input->len; k++) { + int64_t current = input->data[k]; + if (current < cp && ++delta > PC_MAX_INT) { + pc_error(PC_OVERFLOW); + return false; + } + if (current == cp) { + int64_t q = delta; + for (int64_t j = PC_BASE;; j += PC_BASE) { + int64_t t = j <= bias ? PC_TMIN : (j >= bias + PC_TMAX ? PC_TMAX : j - bias); + if (q < t) break; + scr_jb_putc(buf, pc_digit_to_basic(t + (q - t) % (PC_BASE - t))); + q = (q - t) / (PC_BASE - t); + } + scr_jb_putc(buf, pc_digit_to_basic(q)); + bias = pc_adapt(delta, handled_plus_one, handled == basic_length); + delta = 0; + handled++; + } + } + delta++; + cp++; + } + return true; +} + +ScrStr *scr_punycode_decode(const ScrStr *input) { return pc_decode(input->data, input->len); } + +ScrStr *scr_punycode_encode(const ScrStr *input) { + PcPoints points = {0}; + pc_code_points(input->data, input->len, &points); + ScrJsonBuf buf; + scr_jb_init(&buf); + bool ok = pc_encode(&points, &buf); + free(points.data); + ScrStr *result = scr_jb_finish(&buf); + if (ok) return result; + scr_str_release(result); + return NULL; +} + +ScrArr *scr_punycode_ucs2_decode(const ScrStr *input) { + PcPoints points = {0}; + pc_code_points(input->data, input->len, &points); + ScrArr *out = scr_arr_new(SCR_ELEM_F64, points.len); + for (size_t i = 0; i < points.len; i++) scr_arr_push_f64(out, (double)points.data[i]); + free(points.data); + return out; +} + +/* mapDomain: only the part after the first '@' maps (and only up to a + * second one — punycode.js keeps parts[0] and parts[1]); the RFC 3490 + * separators U+002E, U+3002, U+FF0E and U+FF61 all split labels, and the + * labels rejoin with '.'. */ +static ScrStr *pc_map_domain(const ScrStr *input, bool to_ascii) { + const char *s = input->data; + size_t n = input->len; + ScrJsonBuf buf; + scr_jb_init(&buf); + const char *at = memchr(s, '@', n); + if (at) { + scr_jb_write(&buf, s, (size_t)(at - s) + 1); + const char *rest = at + 1; + const char *second = memchr(rest, '@', n - (size_t)(rest - s)); + n = (second ? (size_t)(second - rest) : n - (size_t)(rest - s)); + s = rest; + } + size_t start = 0; + bool ok = true; + for (size_t i = 0; ok && i <= n;) { + size_t sep = 0; + if (i == n) sep = 1; + else if (s[i] == '.') sep = 1; + else if (i + 2 < n && (unsigned char)s[i] == 0xE3 && (unsigned char)s[i + 1] == 0x80 && + (unsigned char)s[i + 2] == 0x82) + sep = 3; + else if (i + 2 < n && (unsigned char)s[i] == 0xEF && + (((unsigned char)s[i + 1] == 0xBC && (unsigned char)s[i + 2] == 0x8E) || + ((unsigned char)s[i + 1] == 0xBD && (unsigned char)s[i + 2] == 0xA1))) + sep = 3; + if (!sep) { + i++; + continue; + } + const char *label = s + start; + size_t len = i - start; + bool ascii = true; + for (size_t k = 0; k < len; k++) ascii = ascii && (unsigned char)label[k] < 0x80; + if (to_ascii && !ascii) { + PcPoints points = {0}; + pc_code_points(label, len, &points); + scr_jb_puts(&buf, "xn--"); + ok = pc_encode(&points, &buf); + free(points.data); + } else if (!to_ascii && len >= 4 && memcmp(label, "xn--", 4) == 0) { + /* string.slice(4).toLowerCase(): ASCII letters fold; the one other + * code point whose lowercase is ASCII is KELVIN SIGN (U+212A → k). + * Any other non-ASCII stays non-ASCII and fails in decode either + * way, at the same position. */ + ScrJsonBuf lower; + scr_jb_init(&lower); + for (size_t k = 4; k < len; k++) { + unsigned char c = (unsigned char)label[k]; + if (c == 0xE2 && k + 2 < len && (unsigned char)label[k + 1] == 0x84 && + (unsigned char)label[k + 2] == 0xAA) { + scr_jb_putc(&lower, 'k'); + k += 2; + } else { + scr_jb_putc(&lower, (char)(c >= 'A' && c <= 'Z' ? c + 32 : c)); + } + } + ScrStr *folded = scr_jb_finish(&lower); + ScrStr *decoded = pc_decode(folded->data, folded->len); + scr_str_release(folded); + if (decoded) { + scr_jb_put_str(&buf, decoded); + scr_str_release(decoded); + } else { + ok = false; + } + } else { + scr_jb_write(&buf, label, len); + } + if (ok && i < n) scr_jb_putc(&buf, '.'); + i += sep; + start = i; + } + ScrStr *result = scr_jb_finish(&buf); + if (ok) return result; + scr_str_release(result); + return NULL; +} + +ScrStr *scr_punycode_to_ascii(const ScrStr *input) { return pc_map_domain(input, true); } +ScrStr *scr_punycode_to_unicode(const ScrStr *input) { return pc_map_domain(input, false); } diff --git a/packages/runtime/src/scr_url.h b/packages/runtime/src/scr_url.h index 134893054..05e5702bb 100644 --- a/packages/runtime/src/scr_url.h +++ b/packages/runtime/src/scr_url.h @@ -53,6 +53,15 @@ ScrDyn *scr_url_legacy_parse(const ScrDyn *url, const ScrDyn *parse_query, const ScrStr *scr_url_legacy_format(const ScrDyn *object, const ScrDyn *options, bool warn); ScrStr *scr_url_legacy_resolve(const ScrDyn *from, const ScrDyn *to, bool warn); ScrStr *scr_url_domain_to_ascii(const ScrDyn *domain, bool missing); +/* node:punycode (scr_url.c): punycode.js 2.1.0's encode/decode, toASCII/ + * toUnicode (mapDomain over RFC 3490 separators) and ucs2.decode (UTF-16 + * code points as an f64 array). Borrowed input; +1, or NULL with the + * module's RangeError pending. */ +ScrStr *scr_punycode_decode(const ScrStr *input); +ScrStr *scr_punycode_encode(const ScrStr *input); +ScrStr *scr_punycode_to_ascii(const ScrStr *input); +ScrStr *scr_punycode_to_unicode(const ScrStr *input); +ScrArr *scr_punycode_ucs2_decode(const ScrStr *input); ScrStr *scr_url_domain_to_unicode(const ScrDyn *domain, bool missing); ScrStr *scr_url_to_path(ScrUrl *u); /* +1, or throws */ ScrStr *scr_url_str_to_path(ScrStr *s); /* +1, or throws */ diff --git a/packages/runtime/src/scr_zlib.c b/packages/runtime/src/scr_zlib.c index b61cdc1cb..ff09e7ea1 100644 --- a/packages/runtime/src/scr_zlib.c +++ b/packages/runtime/src/scr_zlib.c @@ -420,7 +420,22 @@ void scr_zlib_options_drop(ScrZlibOptions *o) { o->dict = NULL; } +static bool scr_zlib_options_parse_object(int mode, const ScrDyn *opts, ScrZlibOptions *out); + +/* A typed options object (a record that crossed into a checked-dynamic + * slot by reference) is read through its materialized view, as a plain + * object is. */ bool scr_zlib_options_parse(int mode, const ScrDyn *opts, ScrZlibOptions *out) { + if (opts == NULL || opts->kind != SCR_DYN_TYPED_REF) return scr_zlib_options_parse_object(mode, opts, out); + ScrDyn *view = scr_dyn_typed_ref_materialize(opts); + bool ok = false; + if (scr_exc_pending()) memset(out, 0, sizeof *out); + else ok = scr_zlib_options_parse_object(mode, view, out); + scr_dyn_release(view); + return ok; +} + +static bool scr_zlib_options_parse_object(int mode, const ScrDyn *opts, ScrZlibOptions *out) { memset(out, 0, sizeof *out); out->window_bits = 15; out->level = Z_DEFAULT_COMPRESSION; diff --git a/packages/runtime/src/scr_zlib_stream.c b/packages/runtime/src/scr_zlib_stream.c index 14d8747e8..4ed717d53 100644 --- a/packages/runtime/src/scr_zlib_stream.c +++ b/packages/runtime/src/scr_zlib_stream.c @@ -431,6 +431,13 @@ static const char *scr_zs_class_name(int mode) { } ScrStream *scr_zlib_stream_new(double mode_d, ScrDyn *opts) { + /* A typed options object is read through its materialized view. */ + if (opts != NULL && opts->kind == SCR_DYN_TYPED_REF) { + ScrDyn *view = scr_dyn_typed_ref_materialize(opts); + ScrStream *stream = scr_exc_pending() ? NULL : scr_zlib_stream_new(mode_d, view); + scr_dyn_release(view); + return stream; + } int mode = (int)mode_d; const ScrDyn *o = opts != NULL && scr_dyn_truthy(opts) ? opts : NULL; ScrZlibOptions zo; diff --git a/packages/runtime/test/test_map.c b/packages/runtime/test/test_map.c index 6eee2d6bb..8899974a8 100644 --- a/packages/runtime/test/test_map.c +++ b/packages/runtime/test/test_map.c @@ -284,6 +284,36 @@ static void test_span_keys(void) { scr_map_release(numbers); } +/* A scratch slice probes like any key; an insertion stores a heap copy, so + * the stored key survives the buffer's next contents. */ +static void test_scratch_keys(void) { + ScrMap *m = scr_map_new(SCR_MAP_KEY_STR, SCR_MAP_VAL_F64, NULL); + ScrStr *line = S("alpha beta alpha"); + _Alignas(16) unsigned char buffer[96]; + ScrStr *alpha = scr_str_slice_scratch(line, 0, 5, false, (ScrStr *)buffer, sizeof buffer); + check(alpha == (ScrStr *)buffer && scr_str_is_scratch(alpha), "short ASCII slice uses the buffer"); + scr_map_set_str_f64(m, alpha, 1); + double out = 0; + check(scr_map_get_str_f64(m, alpha, &out) && out == 1, "scratch key probes its entry"); + ScrStr *beta = scr_str_slice_scratch(line, 6, 10, false, (ScrStr *)buffer, sizeof buffer); + check(!scr_map_has_str(m, beta), "refilled buffer probes its new text"); + scr_map_set_str_f64(m, beta, 2); + ScrStr *again = scr_str_slice_scratch(line, 11, 16, false, (ScrStr *)buffer, sizeof buffer); + scr_map_set_str_f64(m, again, 3); + check(scr_map_size(m) == 2, "scratch overwrite keeps one entry"); + ScrStr *first = scr_map_iter_key_str(m, 0), *second = scr_map_iter_key_str(m, 1); + ScrStr *a = S("alpha"), *b = S("beta"); + check(!scr_str_is_scratch(first) && scr_str_eq(first, a), "stored key is an owned copy"); + check(!scr_str_is_scratch(second) && scr_str_eq(second, b), "second stored key is a copy"); + check(scr_map_get_str_f64(m, a, &out) && out == 3, "overwrite through scratch key"); + scr_str_release(a); + scr_str_release(b); + scr_str_release(first); + scr_str_release(second); + scr_str_release(line); + scr_map_release(m); +} + static void test_same_value_zero(void) { ScrMap *m = scr_map_new(SCR_MAP_KEY_F64, SCR_MAP_VAL_BOOL, NULL); scr_map_set_f64_bool(m, 0.0 / 0.0, true); /* NaN key */ @@ -793,6 +823,7 @@ int main(void) { test_string_entries(); test_same_value_zero(); test_span_keys(); + test_scratch_keys(); test_string_hash_cache(); #ifdef SCR_RC_AUDIT test_rc_accounting(); diff --git a/packages/runtime/test/test_string.c b/packages/runtime/test/test_string.c index 55c49af7e..e49d8eb8b 100644 --- a/packages/runtime/test/test_string.c +++ b/packages/runtime/test/test_string.c @@ -205,6 +205,38 @@ static void surrogate_asserts(void) { } } } + /* Scratch slices hold the materialized text. Only ASCII results of at + * least two units use the caller's buffer, marked as scratch. */ + { + ScrStr *sources[] = {two, s, scr_str_new("GET /api/users 200", 18)}; + const double marks[] = {-INFINITY, -7.5, -3, -0.5, -0.0, 0, 0.5, 1, + 2, 3.9, 5, 9, 40, INFINITY, NAN}; + const size_t nmarks = sizeof marks / sizeof marks[0]; + _Alignas(16) unsigned char buffer[96]; + for (size_t k = 0; k < 3; k++) { + for (size_t a = 0; a < nmarks; a++) { + for (size_t b = 0; b < nmarks; b++) { + for (unsigned substring = 0; substring < 2; substring++) { + ScrStr *src = sources[k]; + ScrStr *scratch = scr_str_slice_scratch(src, marks[a], marks[b], substring != 0, + (ScrStr *)buffer, sizeof buffer); + ScrStr *copy = substring ? scr_str_substring(src, marks[a], marks[b]) + : scr_str_slice(src, marks[a], marks[b]); + bool used = scratch == (ScrStr *)buffer; + bool expected = copy->len >= 2 && scr_bytes_all_ascii(copy->data, copy->len); + total++; + if (scratch->len != copy->len || memcmp(scratch->data, copy->data, copy->len) != 0 || + used != expected || used != scr_str_is_scratch(scratch) || + scr_str_utf16_len(scratch) != scr_str_utf16_len(copy)) + failed++; + scr_str_release(scratch); + scr_str_release(copy); + } + } + } + } + scr_str_release(sources[2]); + } scr_str_release(two); scr_str_release(s); } diff --git a/scripts/zero-cost-compare.mjs b/scripts/zero-cost-compare.mjs new file mode 100644 index 000000000..a89fe4650 --- /dev/null +++ b/scripts/zero-cost-compare.mjs @@ -0,0 +1,490 @@ +/* Comparison logic for scripts/zero-cost-gate.mjs, kept free of process + * orchestration so tests/harness/zero-cost-gate.test.ts can pin it. + * + * Artifacts are compared as bytes first. Only when they differ are they split + * into named units (IR functions, LLVM functions, executable symbols) to name + * the first unit that differs. The self-hosted compiler's IR and LLVM exceed + * V8's maximum string length, so splitting works on Buffers and hashes each + * unit instead of decoding the artifact. */ +import { createHash } from "node:crypto"; + +const hash = (bytes) => createHash("sha1").update(bytes).digest("hex"); + +/** Named units of an artifact: `order` lists unit keys in artifact order, + * `hashes` maps each key to a content hash, and `module` holds the + * module-level remainder (IR header, LLVM globals and declarations) as a list + * of comparable pieces with their labels. Repeated names get `#n` suffixes. */ +function unitsBuilder() { + const order = []; + const hashes = new Map(); + const seen = new Map(); + return { + order, + hashes, + module: [], + add(name, bytes) { + const count = seen.get(name) ?? 0; + seen.set(name, count + 1); + const key = count === 0 ? name : `${name}#${count + 1}`; + order.push(key); + hashes.set(key, hash(bytes)); + }, + }; +} + +/* ── IR (compact or readable JSON) ─────────────────────────────────────── */ + +function skipString(bytes, index) { + for (index++; index < bytes.length; index++) { + if (bytes[index] === 0x5c) index++; + else if (bytes[index] === 0x22) return index + 1; + } + return bytes.length; +} + +/** Byte ranges of the root object's `"functions"` array elements, or null. */ +export function irFunctionSpans(bytes) { + const key = Buffer.from('"functions"'); + let depth = 0; + for (let index = 0; index < bytes.length;) { + const byte = bytes[index]; + if (byte === 0x22) { + const end = skipString(bytes, index); + if (depth === 1 && end - index === key.length && bytes.subarray(index, end).equals(key)) { + let next = end; + while (next < bytes.length && bytes[next] !== 0x3a) next++; + next++; + while (next < bytes.length && bytes[next] !== 0x5b && bytes[next] <= 0x20) next++; + if (bytes[next] === 0x5b) return elementSpans(bytes, next); + } + index = end; + continue; + } + if (byte === 0x7b || byte === 0x5b) depth++; + else if (byte === 0x7d || byte === 0x5d) depth--; + index++; + } + return null; +} + +function elementSpans(bytes, open) { + const spans = []; + let depth = 0; + let start = -1; + for (let index = open + 1; index < bytes.length;) { + const byte = bytes[index]; + if (byte === 0x22) { + if (start < 0) start = index; + index = skipString(bytes, index); + continue; + } + if (byte === 0x7b || byte === 0x5b) { + if (start < 0) start = index; + depth++; + } else if (byte === 0x7d || byte === 0x5d) { + if (depth === 0) { + if (start >= 0) spans.push([start, trimEnd(bytes, index)]); + return { open, close: index, spans }; + } + depth--; + } else if (byte === 0x2c && depth === 0) { + spans.push([start, trimEnd(bytes, index)]); + start = -1; + } else if (start < 0 && byte > 0x20) start = index; + index++; + } + return null; +} + +function trimEnd(bytes, end) { + while (end > 0 && bytes[end - 1] <= 0x20) end--; + return end; +} + +/** The `name` of one serialized IR function. Functions serialize `name` + * first; anything else is parsed (a single function is small). */ +function irFunctionName(bytes, [start, end]) { + const head = bytes.toString("utf8", start, Math.min(end, start + 4096)); + const match = /^\{\s*"name"\s*:\s*("(?:[^"\\]|\\.)*")/.exec(head); + if (match) return JSON.parse(match[1]); + const fn = JSON.parse(bytes.toString("utf8", start, end)); + return typeof fn?.name === "string" ? fn.name : ""; +} + +/** IR units: one per function, plus the module header split by top-level + * key (and, for arrays, by element) so header differences are named too. */ +export function irUnits(bytes) { + const units = unitsBuilder(); + const located = irFunctionSpans(bytes); + let header; + if (located === null) header = JSON.parse(bytes.toString("utf8")); + else { + for (const span of located.spans) + units.add(irFunctionName(bytes, span), bytes.subarray(span[0], span[1])); + header = JSON.parse( + bytes.toString("utf8", 0, located.open + 1) + bytes.toString("utf8", located.close), + ); + } + for (const [key, value] of Object.entries(header)) { + if (key === "functions" && located !== null) continue; + if (Array.isArray(value)) + value.forEach((item, index) => { + const name = item && typeof item === "object" && "name" in item ? `:${item.name}` : ""; + units.module.push({ label: `${key}[${index}${name}]`, text: JSON.stringify(item) }); + }); + else units.module.push({ label: key, text: JSON.stringify(value) }); + } + return units; +} + +/* ── LLVM text ─────────────────────────────────────────────────────────── */ + +const DEFINE_NAME = /@("(?:[^"\\]|\\.)*"|[-\w$.]+)\s*\(/; + +/** LLVM units: one per `define … { … }` body; every other line is + * module-level text compared line by line. */ +export function llvmUnits(bytes) { + const units = unitsBuilder(); + let fnStart = -1; + let fnName = ""; + let lineNumber = 0; + for (let start = 0; start < bytes.length;) { + let end = bytes.indexOf(10, start); + if (end < 0) end = bytes.length; + lineNumber++; + if (fnStart >= 0) { + if (end - start === 1 && bytes[start] === 0x7d) { + units.add(fnName, bytes.subarray(fnStart, end)); + fnStart = -1; + } + } else if (bytes[start] === 0x64 && bytes.toString("latin1", start, start + 7) === "define ") { + const line = bytes.toString("utf8", start, end); + fnName = DEFINE_NAME.exec(line)?.[1] ?? ``; + fnStart = start; + if (line.endsWith("}")) { + units.add(fnName, bytes.subarray(fnStart, end)); + fnStart = -1; + } + } else if (end > start) { + units.module.push({ label: `line ${lineNumber}`, text: bytes.toString("utf8", start, end) }); + } + start = end + 1; + } + return units; +} + +/* ── unit comparison ───────────────────────────────────────────────────── */ + +const clip = (text, width = 160) => (text.length > width ? `${text.slice(0, width)}…` : text); + +/** Compare two unit sets. `first` names the first difference in baseline + * order: a changed or removed function, then an added one, then the first + * differing module-level piece. */ +export function compareUnits(baseline, candidate) { + const changed = []; + const removed = []; + for (const key of baseline.order) { + const other = candidate.hashes.get(key); + if (other === undefined) removed.push(key); + else if (other !== baseline.hashes.get(key)) changed.push(key); + } + const added = candidate.order.filter((key) => !baseline.hashes.has(key)); + let module = null; + const count = Math.max(baseline.module.length, candidate.module.length); + for (let index = 0; index < count && module === null; index++) { + const a = baseline.module[index]; + const b = candidate.module[index]; + if (a?.text !== b?.text) + module = { + label: (a ?? b).label, + baseline: a === undefined ? null : clip(a.text), + candidate: b === undefined ? null : clip(b.text), + }; + } + const firstFunction = baseline.order.find( + (key) => !candidate.hashes.has(key) || candidate.hashes.get(key) !== baseline.hashes.get(key), + ); + let first = null; + if (firstFunction !== undefined) + first = `${changed.includes(firstFunction) ? "changed" : "removed"} ${firstFunction}`; + else if (added.length > 0) first = `added ${added[0]}`; + else if (module !== null) first = `module ${module.label}`; + else if (baseline.order.join("\n") !== candidate.order.join("\n")) first = "function order"; + return { + identical: first === null, + first, + changed, + added, + removed, + module, + functions: { baseline: baseline.order.length, candidate: candidate.order.length }, + }; +} + +/** Byte-compare an artifact and, when it differs, name what changed. */ +export function compareArtifact(kind, baseline, candidate) { + if (baseline.equals(candidate)) return { identical: true }; + const split = kind === "ir" ? irUnits : llvmUnits; + const result = compareUnits(split(baseline), split(candidate)); + // Equal units with unequal bytes means formatting outside the units. + return { ...result, identical: false, first: result.first ?? "bytes outside named units" }; +} + +/* ── executables ───────────────────────────────────────────────────────── */ + +/** The output of `nm` for `binary`, from its spawnSync result. Throws when + * nm could not start, was killed by a signal, or exited with a non-zero + * status, because the symbol checks would otherwise read an empty list. */ +export function nmOutput(binary, result) { + if (result.error) throw new Error(`nm ${binary}: ${result.error.message}`); + if (result.signal) throw new Error(`nm ${binary}: killed by ${result.signal}`); + if (result.status !== 0) { + const detail = typeof result.stderr === "string" ? result.stderr.trim() : ""; + throw new Error( + `nm ${binary}: exited with status ${result.status}${detail ? `: ${detail}` : ""}`, + ); + } + return result.stdout; +} + +/** Defined symbols from `nm -S` output (GNU/LLVM nm, address size type + * name) or `nm` output without sizes (Apple, address type name). */ +export function parseNm(text) { + const symbols = []; + for (const line of text.split("\n")) { + const fields = line.trim().split(/\s+/); + if (fields.length === 4 && /^[0-9a-f]+$/i.test(fields[1])) + symbols.push({ + address: BigInt(`0x${fields[0]}`), + size: BigInt(`0x${fields[1]}`), + type: fields[2], + name: fields[3], + }); + else if (fields.length === 3 && /^[0-9a-f]+$/i.test(fields[0])) + symbols.push({ + address: BigInt(`0x${fields[0]}`), + size: null, + type: fields[1], + name: fields[2], + }); + } + return symbols; +} + +/** The virtual address of file offset `offset` in an ELF64 little-endian + * executable, from its PT_LOAD program headers, or null. */ +export function elfOffsetToAddress(bytes, offset) { + if (bytes.length < 64 || bytes.readUInt32BE(0) !== 0x7f454c46 || bytes[4] !== 2) return null; + const phoff = Number(bytes.readBigUInt64LE(0x20)); + const phentsize = bytes.readUInt16LE(0x36); + const phnum = bytes.readUInt16LE(0x38); + for (let index = 0; index < phnum; index++) { + const at = phoff + index * phentsize; + if (bytes.readUInt32LE(at) !== 1) continue; + const fileOffset = bytes.readBigUInt64LE(at + 8); + const address = bytes.readBigUInt64LE(at + 16); + const fileSize = bytes.readBigUInt64LE(at + 32); + const target = BigInt(offset); + if (target >= fileOffset && target < fileOffset + fileSize) + return address + (target - fileOffset); + } + return null; +} + +/** Name what differs between two executables: symbols added, removed or + * resized, then the symbol holding the first differing byte. */ +export function compareExecutables(baseline, candidate, baselineNm, candidateNm) { + if (baseline.equals(candidate)) return { identical: true }; + const before = new Map(parseNm(baselineNm).map((s) => [s.name, s])); + const after = new Map(parseNm(candidateNm).map((s) => [s.name, s])); + const removed = [...before.keys()].filter((name) => !after.has(name)); + const added = [...after.keys()].filter((name) => !before.has(name)); + const resized = [...before.values()] + .filter((s) => after.has(s.name) && s.size !== null && after.get(s.name).size !== s.size) + .map((s) => ({ + name: s.name, + baseline: Number(s.size), + candidate: Number(after.get(s.name).size), + })); + let offset = 0; + const limit = Math.min(baseline.length, candidate.length); + while (offset < limit && baseline[offset] === candidate[offset]) offset++; + const address = elfOffsetToAddress(baseline, offset); + let holder = null; + if (address !== null) + for (const s of before.values()) + if (s.size !== null && address >= s.address && address < s.address + s.size) holder = s.name; + const first = + resized[0] !== undefined + ? `resized ${resized[0].name}` + : removed[0] !== undefined + ? `removed ${removed[0]}` + : added[0] !== undefined + ? `added ${added[0]}` + : holder !== null + ? `bytes in ${holder}` + : `byte offset ${offset}`; + return { identical: false, first, offset, holder, added, removed, resized }; +} + +/* ── dynamic-tier symbols (two-tier design, invariant I3) ────────────────── */ + +/** Runtime symbol prefixes that belong to the dynamic tier: code a fully + * typed program has no reason to reach. A typed program's release binary + * must define none of them. Extend this list as dynamic-tier units land + * (S3/S4 add their scr_js_* families) and as legacy ScrDyn families move. + * The ScrDyn prefixes are the object-model entry points the two-tier audit + * (results/design-two-tier/DESIGN.md §3.2-3.3) lists as dynamic-only. */ +export const DYNAMIC_TIER_PREFIXES = [ + // Dynamic-tier runtime units (scr_js_value, scr_js_object, ...), from S3. + "scr_js_", + // ScrDyn property bags: reads, writes, lookup and key indexing. + "scr_dyn_obj_", + "scr_dyn_find_", + "scr_dyn_index_", + "scr_dyn_key_", + "sc_dyn_key_", + // Descriptors, own-property reflection, prototypes and proxies. + "scr_dyn_property_", + "scr_dyn_define_", + "scr_dyn_descriptor_", + "scr_dyn_own_", + "scr_dyn_get_own_", + "scr_dyn_prototype", + "scr_dyn_proxy_", + // Dynamic calls and the dynamic receiver read. + "scr_dyn_call", + "scr_dyn_apply", + "scr_dyn_invoke", + "scr_dyn_construct", + "scr_dyn_function_apply", + "scr_dyn_fn_get", + "scr_dyn_this_get", + // Dynamic operators and coercions. + "scr_dyn_add_", + "scr_dyn_arithmetic", + "scr_dyn_to_numeric", + "scr_dyn_number_coerce", + "scr_dyn_string_coerce", +]; + +/** Symbols under those prefixes that typed binaries reach today through a + * leak the two-tier design schedules for removal (S1). Remove an entry when + * its leak closes. */ +export const DYNAMIC_TIER_EXEMPT = new Map([ + // §3.3 L10: scr_dyn_release's dispose switch, linked whenever any dyn + // value can be released. + ["scr_dyn_obj_drop_keys", "L10"], +]); + +/** The dynamic-tier symbols among `names`, sorted. */ +export function dynamicTierSymbols(names) { + return [...new Set(names)] + .filter( + (name) => + DYNAMIC_TIER_PREFIXES.some((prefix) => name.startsWith(prefix)) && + !DYNAMIC_TIER_EXEMPT.has(name), + ) + .sort(); +} + +/* ── cachegrind ────────────────────────────────────────────────────────── */ + +/** Background threads whose instruction counts depend on scheduling: the + * allocator's prefault thread. */ +export const CACHEGRIND_EXCLUDED = new Set(["scr_sa_pf_main", "scr_sa_prefault"]); + +/** Per-function instruction counts (Ir) from a cachegrind output file, + * restricted to `keep` (the executable's own defined symbols). Dynamic + * linker and libc/pthread functions are outside `keep`; their counts vary + * run to run with lazy binding and thread wakeups. */ +export function cachegrindCounts(text, keep) { + const counts = new Map(); + let events = null; + let current = null; + for (const line of text.split("\n")) { + if (line.startsWith("events:")) events = line.slice(7).trim().split(/\s+/); + else if (line.startsWith("fn=")) { + const name = line.slice(3).trim(); + current = keep.has(name) && !CACHEGRIND_EXCLUDED.has(name) ? name : null; + } else if (line.startsWith("fl=") || line.startsWith("fi=") || line.startsWith("fe=")) continue; + else if (current !== null && /^\d/.test(line)) { + const fields = line.trim().split(/\s+/); + const column = events === null ? 0 : Math.max(0, events.indexOf("Ir")); + const ir = Number(fields[column + 1] ?? 0); + counts.set(current, (counts.get(current) ?? 0) + ir); + } + } + return counts; +} + +/** Per-function instruction-count deltas, largest absolute first. */ +export function compareCounts(baseline, candidate) { + const names = new Set([...baseline.keys(), ...candidate.keys()]); + const deltas = []; + let totalBaseline = 0; + let totalCandidate = 0; + for (const name of names) { + const a = baseline.get(name) ?? 0; + const b = candidate.get(name) ?? 0; + totalBaseline += a; + totalCandidate += b; + if (a !== b) deltas.push({ name, baseline: a, candidate: b, delta: b - a }); + } + deltas.sort((x, y) => Math.abs(y.delta) - Math.abs(x.delta) || (x.name < y.name ? -1 : 1)); + return { + identical: deltas.length === 0, + total: { baseline: totalBaseline, candidate: totalCandidate }, + deltas, + }; +} + +/* ── verdicts ──────────────────────────────────────────────────────────── */ + +/** One program's verdict from its build results and artifact comparisons. + * `allowed` marks programs that match the change's pattern signature. */ +export function programVerdict({ baseline, candidate, artifacts, allowed }) { + if (!baseline.ok || !candidate.ok) { + if (!baseline.ok && !candidate.ok && baseline.error === candidate.error) + return { verdict: "both-fail", fails: false }; + return { verdict: "build-error", fails: true }; + } + const differing = Object.entries(artifacts).filter(([, r]) => !r.identical); + if (differing.length === 0) return { verdict: "identical", fails: false }; + return allowed ? { verdict: "allowed", fails: false } : { verdict: "differs", fails: true }; +} + +export function median(values) { + const sorted = [...values].sort((a, b) => a - b); + const middle = sorted.length >> 1; + return sorted.length % 2 ? sorted[middle] : (sorted[middle - 1] + sorted[middle]) / 2; +} + +export function geomean(values) { + return values.length === 0 + ? 1 + : Math.exp(values.reduce((sum, value) => sum + Math.log(value), 0) / values.length); +} + +/** Compile-time ratio (candidate / baseline) for a set of programs. Each + * row holds paired samples: one baseline and one candidate build that ran + * side by side, so both saw the same machine load. The estimate is the + * geometric mean of each program's median pair ratio, with a 95% bootstrap + * interval over the pairs. Builds on a shared VM vary by 10-20% run to run, + * so the gate judges the interval, not the point estimate. */ +export function timeRatio(rows, resamples = 2000) { + let state = 0x5eed; + const random = () => { + state = (Math.imul(state, 1664525) + 1013904223) >>> 0; + return state / 4294967296; + }; + const pick = (xs) => + Array.from({ length: xs.length }, () => xs[Math.floor(random() * xs.length)]); + const estimate = (draw) => geomean(rows.map((r) => median(draw(r.ratios)))); + const ratios = Array.from({ length: resamples }, () => estimate(pick)).sort((a, b) => a - b); + return { + ratio: estimate((xs) => xs), + low: ratios[Math.floor(resamples * 0.025)], + high: ratios[Math.floor(resamples * 0.975)], + }; +} diff --git a/scripts/zero-cost-gate.mjs b/scripts/zero-cost-gate.mjs new file mode 100644 index 000000000..f4272ac6a --- /dev/null +++ b/scripts/zero-cost-gate.mjs @@ -0,0 +1,652 @@ +/* Zero-cost gate: prove that a compiler change leaves good code untouched. + * + * Compiles a fixed good-code set with a baseline and a candidate checkout and + * compares IR, LLVM and release executables byte for byte, plus compile time + * and binary size, and optionally per-function cachegrind instruction counts. + * Both checkouts compile the same program files (from --programs, default the + * baseline), so program paths embedded in artifacts match; checkout paths and + * timestamps do not reach the artifacts. Builds bypass the native cache. + * + * The good-code set: the runtime benchmark workloads (and benchmarks/builds + * applications), the self-hosted compiler (IR and LLVM), and the TypeScript + * corpus programs. A program whose sources match --allow (the change's + * pattern signature) may differ; any other difference fails the gate. */ +import assert from "node:assert/strict"; +import { spawn, spawnSync } from "node:child_process"; +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + statSync, + writeFileSync, +} from "node:fs"; +import { availableParallelism } from "node:os"; +import { dirname, join, relative, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { parseArgs } from "node:util"; +import { + cachegrindCounts, + compareArtifact, + compareCounts, + compareExecutables, + dynamicTierSymbols, + median, + nmOutput, + parseNm, + programVerdict, + timeRatio, +} from "./zero-cost-compare.mjs"; + +const repoRoot = fileURLToPath(new URL("..", import.meta.url)); +const { values } = parseArgs({ + options: { + baseline: { type: "string" }, + candidate: { type: "string", default: repoRoot }, + programs: { type: "string", default: "baseline" }, + set: { type: "string", default: "bench,self,corpus" }, + quick: { type: "boolean", default: false }, + corpus: { type: "string", default: "ts" }, + shard: { type: "string" }, + only: { type: "string" }, + allow: { type: "string" }, + "timing-runs": { type: "string" }, + "self-timing-runs": { type: "string", default: "0" }, + "max-time-growth": { type: "string", default: "2" }, + "max-size-growth": { type: "string", default: "0" }, + cachegrind: { type: "boolean", default: false }, + jobs: { type: "string", default: String(Math.max(1, availableParallelism())) }, + json: { type: "string" }, + keep: { type: "boolean", default: false }, + help: { type: "boolean", default: false }, + }, +}); +if (values.help || values.baseline === undefined) { + console.log(`Usage: node scripts/zero-cost-gate.mjs --baseline= [--candidate=] + [--programs=baseline|candidate|] [--set=bench,self,corpus] [--quick] + [--corpus=ts|all] [--shard=i/n] [--only=] [--allow=] + [--timing-runs=N] [--self-timing-runs=N] [--max-time-growth=2] + [--max-size-growth=0] + [--cachegrind] [--jobs=N] [--json=] [--keep] + +Each checkout must be built (packages/cli/dist/bootstrap.js and this host's +runtime pack). Both compile the same program files, taken from --programs +(default: the baseline checkout). + +--set picks the good-code set: bench (benchmarks/runtime workloads and the +benchmarks/builds applications), self (packages/compiler/src/native/cli.ts, +LLVM only), corpus (tests/corpus TypeScript programs; --corpus=all adds +JavaScript entries). --quick is --set=bench --timing-runs=0, for CI. +--shard=i/n keeps every n-th corpus program; --only filters program ids. + +--allow= is the change's pattern signature: programs whose source +files match it may differ (reported as "allowed"). Any other difference in +IR, LLVM or the release executable fails the gate. + +--timing-runs=N (default 5) times N paired --emit=llvm builds per bench +program, baseline and candidate side by side. The self-hosted compiler's +comparison build is its first pair; --self-timing-runs adds more. The gate +fails when the 95% bootstrap interval of the bench geomean ratio, or of the +self-hosted ratio, lies entirely above --max-time-growth percent; an interval +that straddles it is reported as inconclusive. +--max-size-growth bounds executable growth (percent) of allowed programs. + +Dynamic-tier symbols (two-tier invariant I3): no executable may gain +symbols under DYNAMIC_TIER_PREFIXES (scripts/zero-cost-compare.mjs) unless +it matches --allow, and fully typed bench workloads (all but +DYNAMIC_WORKLOADS in this script) may define none. + +--cachegrind (Linux, valgrind on PATH) runs differing executables under +cachegrind (bench workloads with their arguments, corpus programs without) +and compares instruction counts per function of the executable's own +symbols, excluding the allocator's prefault thread. + +Exit status: 0 when the gate passes, 1 when it fails.`); + process.exit(values.help ? 0 : 2); +} + +const trees = { baseline: resolve(values.baseline), candidate: resolve(values.candidate) }; +for (const [label, root] of Object.entries(trees)) + assert.ok( + existsSync(join(root, "packages/cli/dist/bootstrap.js")), + `${label} checkout ${root} has no built CLI (packages/cli/dist/bootstrap.js)`, + ); +const programsRoot = + values.programs === "baseline" || values.programs === "candidate" + ? trees[values.programs] + : resolve(values.programs); +const sets = new Set((values.quick ? "bench" : values.set).split(",")); +for (const set of sets) assert.ok(["bench", "self", "corpus"].includes(set), `unknown set ${set}`); +const timingRuns = Number(values["timing-runs"] ?? (values.quick ? 0 : 5)); +const selfTimingRuns = Number(values["self-timing-runs"]); +const maxTimeGrowth = Number(values["max-time-growth"]) / 100; +const maxSizeGrowth = Number(values["max-size-growth"]) / 100; +const jobs = Number(values.jobs); +assert.ok(Number.isInteger(timingRuns) && timingRuns >= 0, "--timing-runs must be >= 0"); +assert.ok(Number.isInteger(jobs) && jobs >= 1, "--jobs must be >= 1"); +assert.ok(["ts", "all"].includes(values.corpus), "--corpus must be ts or all"); +const only = values.only ? new RegExp(values.only) : null; +const allow = values.allow ? new RegExp(values.allow, "m") : null; +if (values.cachegrind) + assert.equal( + spawnSync("valgrind", ["--version"]).status, + 0, + "--cachegrind needs valgrind on PATH", + ); + +const work = mkdtempSync("/tmp/scriptc-zero-cost-"); + +/* ── the good-code set ─────────────────────────────────────────────────── */ + +/** Every file a program reads: the entry and its relative imports. */ +function programInputs(entry) { + const files = new Set(); + const pending = [entry]; + while (pending.length > 0) { + const file = pending.pop(); + if (files.has(file)) continue; + files.add(file); + const text = readFileSync(file, "utf8"); + for (const match of text.matchAll( + /(?:from|import|require)\s*\(?\s*["'](\.{1,2}\/[^"']+)["']/g, + )) { + const base = resolve(dirname(file), match[1]); + const stem = base.replace(/\.(m|c)?js$/, ""); + const found = [ + base, + `${stem}.ts`, + `${stem}.mts`, + `${stem}.cts`, + `${base}.ts`, + join(base, "index.ts"), + ].find((candidate) => existsSync(candidate) && statSync(candidate).isFile()); + if (found !== undefined) pending.push(found); + } + } + return [...files]; +} + +function goodCodeSet() { + const programs = []; + if (sets.has("bench")) { + const suite = join(programsRoot, "benchmarks/runtime"); + const manifest = JSON.parse(readFileSync(join(suite, "workloads.json"), "utf8")); + const seen = new Set(); + for (const w of manifest.workloads) { + const entry = join(suite, w.entry); + if (seen.has(entry)) continue; + seen.add(entry); + programs.push({ id: `bench/${w.name}`, entry, kind: "exe", workload: w }); + } + } + if (sets.has("self")) + programs.push({ + id: "self/cli", + entry: join(programsRoot, "packages/compiler/src/native/cli.ts"), + kind: "source", + }); + if (sets.has("corpus")) { + const corpus = join(programsRoot, "tests/corpus"); + const exts = values.corpus === "all" ? ["ts", "js", "mjs", "cjs"] : ["ts"]; + const entries = []; + for (const name of readdirSync(corpus).sort()) { + const path = join(corpus, name); + if (statSync(path).isDirectory()) { + const main = exts.map((ext) => join(path, `main.${ext}`)).find((p) => existsSync(p)); + if (main !== undefined) entries.push(main); + } else if (exts.some((ext) => name.endsWith(`.${ext}`))) entries.push(path); + } + let selected = entries; + if (values.shard) { + const [index, count] = values.shard.split("/").map(Number); + assert.ok(index >= 1 && index <= count, "--shard must be i/n with 1 <= i <= n"); + selected = entries.filter((_, k) => k % count === index - 1); + } + for (const entry of selected) { + const head = readFileSync(entry, "utf8").split("\n", 2); + const speed = head.some((line) => /^\/\/ @optimization:\s*speed\s*$/.test(line)); + programs.push({ + id: `corpus/${relative(corpus, entry)}`, + entry, + kind: "exe", + optimization: speed ? "speed" : "release", + }); + } + } + return programs.filter((p) => only === null || only.test(p.id)); +} + +/* ── builds ────────────────────────────────────────────────────────────── */ + +function exec(command, args, options = {}) { + return new Promise((resolvePromise) => { + const started = process.hrtime.bigint(); + const child = spawn(command, args, { stdio: ["ignore", "pipe", "pipe"], ...options }); + let stdout = ""; + let stderr = ""; + child.stdout.setEncoding("utf8").on("data", (c) => (stdout += c)); + child.stderr.setEncoding("utf8").on("data", (c) => (stderr += c)); + child.on("close", (status, signal) => + resolvePromise({ + status, + signal, + stdout, + stderr, + ms: Number(process.hrtime.bigint() - started) / 1e6, + }), + ); + }); +} + +function buildEnv() { + const env = { ...process.env, SCRIPTC_NO_CACHE: "1" }; + delete env.SCRIPTC_TIMING; + delete env.SCRIPTC_CACHE_DIR; + return env; +} + +function compile(tree, args) { + return exec( + process.execPath, + ["--max-old-space-size=8192", join(tree, "packages/cli/dist/bootstrap.js"), "build", ...args], + { cwd: programsRoot, env: buildEnv() }, + ); +} + +/** Build one program with one checkout. Executable programs build once, + * keeping the LLVM and IR beside the release executable. The self-hosted + * compiler emits LLVM only: each of its frontend runs takes minutes, and its + * LLVM names the first differing function as well as its IR would. */ +async function buildProgram(program, label) { + const dir = join(work, label, program.id.replace(/[^\w.-]+/g, "_")); + mkdirSync(dir, { recursive: true }); + const tree = trees[label]; + const failure = (result) => ({ + ok: false, + error: (result.stderr || result.stdout || `exit ${result.status ?? result.signal}`) + .replaceAll(dir, "") + .trim() + .slice(0, 4000), + }); + if (program.kind === "source") { + const llvm = join(dir, "program.ll"); + const result = await compile(tree, [program.entry, "--emit=llvm", "-o", llvm]); + if (result.status !== 0) return failure(result); + return { ok: true, dir, llvm, ms: result.ms }; + } + const exe = join(dir, "program"); + const result = await compile(tree, [ + program.entry, + `--optimization=${program.optimization ?? "release"}`, + "--keep-llvm", + "--emit-ir", + "-o", + exe, + ]); + if (result.status !== 0) return failure(result); + const files = readdirSync(dir); + const pick = (suffix) => { + const name = files.find((f) => f.endsWith(suffix)); + return name === undefined ? null : join(dir, name); + }; + return { ok: true, dir, exe, ir: pick(".ir.json"), llvm: pick(".ll"), ms: result.ms }; +} + +const nmCache = new Map(); +function nm(binary) { + if (nmCache.has(binary)) return nmCache.get(binary); + const result = spawnSync("nm", process.platform === "darwin" ? [binary] : ["-S", binary], { + encoding: "utf8", + maxBuffer: 256 * 1024 * 1024, + }); + const text = nmOutput(binary, result); + nmCache.set(binary, text); + return text; +} + +/** Names of the functions an executable defines. */ +const definedFunctions = (binary) => + parseNm(nm(binary)) + .filter((s) => /^[tTwW]$/.test(s.type)) + .map((s) => s.name); + +/* Bench workloads whose release binaries link dynamic-tier symbols on + * b14377864 (DYNAMIC_TIER_PREFIXES in zero-cost-compare.mjs): JSON.parse + * trees, worker messages, regex exec metadata (two-tier §3.3 L5), and a + * shared scr_dyn_call/scr_dyn_define_property core not yet traced to its + * typed root. Every other bench workload is fully typed and must link none. + * Remove a workload here when a leak fix makes its binary clean. */ +const DYNAMIC_WORKLOADS = new Set([ + "bench/json-records", + "bench/cli-config", + "bench/http-api", + "bench/validate-errors", + "bench/regex-logs", + "bench/atomics-contended", + "bench/intern-table", + "bench/typed-array-scan-workers", + "bench/log-summary", + "bench/inventory-report", +]); + +/** The I3 symbol check. Every executable may not gain dynamic-tier symbols + * over the baseline (unless the program matches --allow); a fully typed + * bench workload may contain none at all. */ +function dynamicTierCheck(program, baseline, candidate, artifacts, allowed) { + if (!baseline.exe || !artifacts.exe) return null; + const typed = program.id.startsWith("bench/") && !DYNAMIC_WORKLOADS.has(program.id); + if (artifacts.exe.identical && !typed) return null; + const after = dynamicTierSymbols(definedFunctions(candidate.exe)); + const before = artifacts.exe.identical + ? after + : dynamicTierSymbols(definedFunctions(baseline.exe)); + const added = after.filter((name) => !before.includes(name)); + const failures = []; + if (typed && after.length > 0) + failures.push( + `fully typed program links dynamic-tier symbols: ${after.slice(0, 5).join(", ")}`, + ); + if (added.length > 0 && !allowed) + failures.push(`gained dynamic-tier symbols: ${added.slice(0, 5).join(", ")}`); + return { typed, count: after.length, added, failures }; +} + +function compareBuilds(baseline, candidate) { + const artifacts = {}; + for (const kind of ["ir", "llvm"]) { + if (!(kind in baseline)) continue; + if (baseline[kind] === null || candidate[kind] === null) { + artifacts[kind] = { identical: false, first: `missing ${kind} artifact` }; + continue; + } + artifacts[kind] = compareArtifact( + kind, + readFileSync(baseline[kind]), + readFileSync(candidate[kind]), + ); + } + if (baseline.exe) { + const a = readFileSync(baseline.exe); + const b = readFileSync(candidate.exe); + artifacts.exe = a.equals(b) + ? { identical: true } + : compareExecutables(a, b, nm(baseline.exe), nm(candidate.exe)); + artifacts.exe.bytes = { baseline: a.length, candidate: b.length }; + } + return artifacts; +} + +/** Run tasks with at most `jobs` at a time; heavy tasks run alone. */ +async function pool(tasks) { + const results = new Array(tasks.length); + let next = 0; + let running = 0; + let heavy = false; + await new Promise((done) => { + const pump = () => { + if (next === tasks.length && running === 0) return done(); + while (next < tasks.length && running < jobs && !heavy) { + const task = tasks[next]; + if (task.heavy && running > 0) break; + const index = next++; + running++; + if (task.heavy) heavy = true; + task.run().then((value) => { + results[index] = value; + running--; + if (task.heavy) heavy = false; + pump(); + }); + } + }; + pump(); + }); + return results; +} + +/* ── cachegrind ────────────────────────────────────────────────────────── */ + +async function instructionCounts(binary, args, out) { + const result = await exec("valgrind", [ + "--tool=cachegrind", + "--cache-sim=no", + `--cachegrind-out-file=${out}`, + binary, + ...args, + ]); + // Corpus programs may exit non-zero on purpose (`// @exit: 1`). + if (result.signal !== null || !existsSync(out)) return null; + return cachegrindCounts(readFileSync(out, "utf8"), new Set(definedFunctions(binary))); +} + +/* ── main ──────────────────────────────────────────────────────────────── */ + +function revision(root) { + const r = spawnSync("git", ["-C", root, "rev-parse", "--short", "HEAD"], { encoding: "utf8" }); + if (r.status !== 0) return null; + const dirty = spawnSync("git", ["-C", root, "status", "--porcelain", "--untracked-files=no"], { + encoding: "utf8", + }); + return r.stdout.trim() + (dirty.stdout.trim() ? "+dirty" : ""); +} + +const fmtMs = (ms) => `${(ms / 1000).toFixed(2)}s`; +const pct = (ratio) => `${ratio >= 1 ? "+" : ""}${((ratio - 1) * 100).toFixed(2)}%`; + +const programs = goodCodeSet(); +assert.ok(programs.length > 0, "the selected good-code set is empty"); +const report = { + schema: 1, + date: new Date().toISOString(), + host: { platform: process.platform, arch: process.arch, node: process.version }, + trees: Object.fromEntries( + Object.entries(trees).map(([label, root]) => [label, { root, revision: revision(root) }]), + ), + programsRoot, + allow: values.allow ?? null, + programs: [], + timing: null, + failures: [], +}; +console.log( + `zero-cost gate: ${programs.length} programs (${[...sets].join(", ")}), ` + + `baseline ${trees.baseline}, candidate ${trees.candidate}, programs from ${programsRoot}`, +); + +let failed = false; +const started = Date.now(); +await pool( + programs.map((program) => ({ + heavy: program.kind === "source", + run: async () => { + const [baseline, candidate] = await Promise.all([ + buildProgram(program, "baseline"), + buildProgram(program, "candidate"), + ]); + const allowed = + allow !== null && + programInputs(program.entry).some((f) => allow.test(readFileSync(f, "utf8"))); + const artifacts = baseline.ok && candidate.ok ? compareBuilds(baseline, candidate) : {}; + const { verdict, fails } = programVerdict({ baseline, candidate, artifacts, allowed }); + const entry = { + id: program.id, + verdict, + allowed, + buildMs: { baseline: baseline.ms ?? null, candidate: candidate.ms ?? null }, + artifacts, + }; + if (!baseline.ok) entry.baselineError = baseline.error; + if (!candidate.ok) entry.candidateError = candidate.error; + const size = artifacts.exe?.bytes; + if (verdict === "allowed" && size && size.candidate > size.baseline * (1 + maxSizeGrowth)) { + entry.sizeFailure = true; + report.failures.push( + `${program.id}: executable grew ${size.baseline} -> ${size.candidate} bytes`, + ); + } + if (fails) report.failures.push(`${program.id}: ${verdict}`); + const symbols = + baseline.ok && candidate.ok + ? dynamicTierCheck(program, baseline, candidate, artifacts, allowed) + : null; + if (symbols !== null) entry.dynamicTier = symbols; + for (const failure of symbols?.failures ?? []) + report.failures.push(`${program.id}: ${failure}`); + if (fails || entry.sizeFailure || symbols?.failures.length) failed = true; + if (values.cachegrind && artifacts.exe && !artifacts.exe.identical) { + // Bench workloads run with their benchmark arguments; corpus + // programs take none. + const w = program.workload ?? { args: [] }; + if (w.kind === "server" || w.args.includes("{input}")) + entry.cachegrind = { skipped: "needs a server client or generated input" }; + else { + const counts = await Promise.all([ + instructionCounts(baseline.exe, w.args, join(baseline.dir, "cg.out")), + instructionCounts(candidate.exe, w.args, join(candidate.dir, "cg.out")), + ]); + entry.cachegrind = counts.includes(null) + ? { skipped: "valgrind run failed" } + : (({ deltas, ...rest }) => ({ ...rest, deltas: deltas.slice(0, 20) }))( + compareCounts(counts[0], counts[1]), + ); + } + } + printProgram(entry); + report.programs.push(entry); + nmCache.delete(baseline.exe); + nmCache.delete(candidate.exe); + if (!values.keep) { + if (baseline.dir) rmSync(baseline.dir, { recursive: true, force: true }); + if (candidate.dir) rmSync(candidate.dir, { recursive: true, force: true }); + } + }, + })), +); + +function printProgram(entry) { + const label = { + identical: "identical", + allowed: "allowed ", + differs: "DIFFERS ", + "build-error": "ERROR ", + "both-fail": "both-fail", + }[entry.verdict]; + const notes = []; + for (const [kind, result] of Object.entries(entry.artifacts)) + if (!result.identical) { + const counts = result.changed + ? ` (${result.changed.length} changed, ${result.added.length} added, ${result.removed.length} removed)` + : ""; + notes.push(`${kind}: ${result.first}${counts}`); + } + const size = entry.artifacts.exe?.bytes; + if (size && size.baseline !== size.candidate) + notes.push( + `size ${size.baseline} -> ${size.candidate} B (${pct(size.candidate / size.baseline)})`, + ); + if (entry.cachegrind?.total) { + const { total, deltas } = entry.cachegrind; + notes.push( + `Ir ${total.baseline} -> ${total.candidate} (${pct(total.candidate / Math.max(1, total.baseline))})` + + (deltas[0] + ? `, largest ${deltas[0].name} ${deltas[0].delta > 0 ? "+" : ""}${deltas[0].delta}` + : ""), + ); + } else if (entry.cachegrind?.skipped) + notes.push(`cachegrind skipped: ${entry.cachegrind.skipped}`); + for (const failure of entry.dynamicTier?.failures ?? []) notes.push(`SYMBOLS: ${failure}`); + if (entry.baselineError) notes.push(`baseline: ${entry.baselineError.split("\n")[0]}`); + if (entry.candidateError) notes.push(`candidate: ${entry.candidateError.split("\n")[0]}`); + console.log(`${label} ${entry.id}${notes.length ? ` ${notes.join("; ")}` : ""}`); +} + +/* Compile time: paired --emit=llvm builds, the baseline and the candidate + * side by side, one program at a time. Identical LLVM means everything after + * emission (LLVM optimization, linking) costs the same, so the timed part is + * the compiler's own work. The self-hosted compiler takes minutes per build; + * its main comparison build is its first pair. */ +if (timingRuns > 0) { + const rows = []; + const pair = async (program) => { + const [baseline, candidate] = await Promise.all( + ["baseline", "candidate"].map((label) => + compile(trees[label], [ + program.entry, + "--emit=llvm", + "-o", + join(work, `timing-${label}.ll`), + ]), + ), + ); + return baseline.status === 0 && candidate.status === 0 ? [baseline.ms, candidate.ms] : null; + }; + for (const program of programs) { + if (program.id.startsWith("corpus/")) continue; + const row = { id: program.id, baselineMs: [], candidateMs: [], ratios: [] }; + const add = (sample) => { + if (sample === null) return; + row.baselineMs.push(Math.round(sample[0])); + row.candidateMs.push(Math.round(sample[1])); + row.ratios.push(sample[1] / sample[0]); + }; + const runs = program.kind === "source" ? selfTimingRuns : timingRuns; + if (program.kind === "source") { + const built = report.programs.find((p) => p.id === program.id)?.buildMs; + if (built?.baseline && built?.candidate) add([built.baseline, built.candidate]); + } + for (let run = 0; run < runs; run++) add(await pair(program)); + if (row.ratios.length === 0) continue; + rows.push(row); + console.log( + `time ${program.id} ${fmtMs(median(row.baselineMs))} -> ${fmtMs(median(row.candidateMs))} ` + + `(${pct(median(row.ratios))}, ${row.ratios.length} pairs)`, + ); + } + report.timing = { runs: timingRuns, maxGrowth: maxTimeGrowth, rows, groups: {} }; + const groups = { + bench: rows.filter((r) => r.id.startsWith("bench/")), + self: rows.filter((r) => r.id === "self/cli"), + }; + for (const [name, group] of Object.entries(groups)) { + if (group.length === 0) continue; + const result = timeRatio(group); + report.timing.groups[name] = result; + const verdict = + result.low > 1 + maxTimeGrowth + ? "SLOWER" + : result.high > 1 + maxTimeGrowth + ? "inconclusive (rerun with more runs)" + : "within tolerance"; + console.log( + `time ${name} ${group.length > 1 ? "geomean " : ""}${pct(result.ratio)} ` + + `[95% CI ${pct(result.low)}, ${pct(result.high)}]: ${verdict}`, + ); + if (result.low > 1 + maxTimeGrowth) { + failed = true; + report.failures.push( + `${name} compile time grew ${pct(result.ratio)} (CI low ${pct(result.low)})`, + ); + } + } +} + +const counts = {}; +for (const p of report.programs) counts[p.verdict] = (counts[p.verdict] ?? 0) + 1; +report.counts = counts; +report.verdict = failed ? "fail" : "pass"; +report.elapsedMs = Date.now() - started; +if (values.json) { + mkdirSync(dirname(resolve(values.json)), { recursive: true }); + writeFileSync(values.json, JSON.stringify(report, null, 2) + "\n"); +} +if (!values.keep) rmSync(work, { recursive: true, force: true }); +else console.log(`artifacts kept in ${work}`); +console.log( + `zero-cost gate: ${failed ? "FAIL" : "PASS"} — ${Object.entries(counts) + .map(([k, v]) => `${v} ${k}`) + .join(", ")} in ${fmtMs(report.elapsedMs)}`, +); +for (const failure of report.failures) console.log(` ${failure}`); +process.exit(failed ? 1 : 0); diff --git a/tests/corpus/5633-zlib-typed-options.ts b/tests/corpus/5633-zlib-typed-options.ts new file mode 100644 index 000000000..da657ca14 --- /dev/null +++ b/tests/corpus/5633-zlib-typed-options.ts @@ -0,0 +1,58 @@ +// zlib options held in a typed object (a cast object literal that crosses +// into the options slot by reference) are read like a plain options +// object: levels, windowBits and strategies apply, and out-of-range values +// throw Node's ERR_OUT_OF_RANGE from the sync, callback, promisified and +// stream forms. +import * as zlib from "node:zlib"; +import { promisify } from "node:util"; + +const data = Buffer.from("hello hello hello hello hello"); +const stored = { level: 0 } as zlib.ZlibOptions; +const fast = { level: 1 } as zlib.ZlibOptions; +const huffman = { strategy: zlib.constants.Z_HUFFMAN_ONLY } as zlib.ZlibOptions; +const smallWindow = { windowBits: 9, memLevel: 1 } as zlib.ZlibOptions; +const badLevel = { level: 99 } as zlib.ZlibOptions; +const badWindow = { windowBits: 3 } as zlib.ZlibOptions; + +function attempt(label: string, run: () => void): void { + try { + run(); + console.log(label, "ok"); + } catch (e) { + const err = e as NodeJS.ErrnoException; + console.log(label, err.name, err.code, err.message); + } +} + +console.log("sync stored", zlib.deflateSync(data, stored).toString("hex")); +console.log("sync fast", zlib.deflateSync(data, fast).toString("hex")); +console.log("sync huffman", zlib.deflateSync(data, huffman).toString("hex")); +console.log("raw small window", zlib.deflateRawSync(data, smallWindow).toString("hex")); +console.log("gzip stored", zlib.gzipSync(data, stored).subarray(10).toString("hex")); +attempt("sync bad level", () => zlib.deflateSync(data, badLevel)); +attempt("sync bad window", () => zlib.inflateSync(zlib.deflateSync(data), badWindow)); +attempt("callback bad level", () => zlib.gzip(data, badLevel, () => console.log("not called"))); +attempt("stream bad level", () => zlib.createDeflate(badLevel)); + +const deflateAsync = promisify(zlib.deflate); + +async function main(): Promise { + const viaCallback = await new Promise((resolve, reject) => + zlib.deflate(data, stored, (err, out) => (err ? reject(err) : resolve(out))), + ); + console.log("callback stored", viaCallback.toString("hex")); + console.log("promise stored", (await deflateAsync(data, stored)).toString("hex")); + try { + await deflateAsync(data, badLevel); + console.log("promise bad level fulfilled"); + } catch (e) { + console.log("promise bad level", (e as NodeJS.ErrnoException).code); + } + const stream = zlib.createDeflate(fast); + const parts: Buffer[] = []; + stream.on("data", (chunk: Buffer) => parts.push(chunk)); + stream.on("end", () => console.log("stream fast", Buffer.concat(parts).toString("hex"))); + stream.end(data); +} + +main().catch((e) => console.log("unexpected", String(e))); diff --git a/tests/corpus/5683-date-local-fields-tz.ts b/tests/corpus/5683-date-local-fields-tz.ts new file mode 100644 index 000000000..dc75011ed --- /dev/null +++ b/tests/corpus/5683-date-local-fields-tz.ts @@ -0,0 +1,95 @@ +// Local-time getters read repeatedly on one Date and alternately on two, +// on Dates parsed from strings, and across process.env.TZ writes and +// deletes: every read answers the fields of the zone in effect at that +// moment, as in Node. + +function fields(d: Date): string { + return [ + d.getFullYear(), + d.getMonth(), + d.getDate(), + d.getDay(), + d.getHours(), + d.getMinutes(), + d.getSeconds(), + d.getMilliseconds(), + d.getTimezoneOffset(), + ].join(","); +} + +const a = new Date(Date.UTC(2024, 2, 10, 6, 59, 59, 999)); +const b = new Date(Date.UTC(2024, 2, 10, 7, 0, 0, 0)); +const c = new Date(Date.UTC(2024, 10, 3, 5, 30, 0, 250)); + +function report(label: string): void { + console.log(label, fields(a), fields(a)); + console.log(label, fields(b), fields(c), fields(b)); + let mixed = 0; + for (let i = 0; i < 6; i++) mixed += a.getHours() * 100 + b.getHours() + c.getDate(); + console.log(label, "alternating", mixed); +} + +report("start"); +process.env.TZ = "America/New_York"; +report("new york"); +process.env.TZ = "Asia/Kolkata"; +report("kolkata"); +process.env.TZ = "Australia/Lord_Howe"; +report("lord howe"); + +// Dates parsed from strings with an explicit offset, read in local time. +const d = new Date("2024-06-01T23:45:00+05:30"); +console.log("offset string", fields(d), d.getTime()); +const e = new Date("2024-07-11T00:15:30.001Z"); +console.log("offset string", fields(e), e.getTime()); +console.log("first again", fields(d)); + +delete process.env.TZ; +report("deleted"); +process.env.TZ = "UTC"; +report("utc"); + +// Many instants read field by field. +process.env.TZ = "Europe/Berlin"; +let sum = 0; +for (let i = 0; i < 2000; i++) { + const t = new Date(1700000000000 + i * 3_599_999); + sum += t.getHours() + t.getDate() * 3 + t.getMonth() * 7 + t.getTimezoneOffset(); +} +console.log("berlin sum", sum); + +const invalid = new Date(Number.NaN); +console.log("invalid", fields(invalid), fields(invalid)); + +// Instants around day and second boundaries, before the epoch and at the +// ends of the Date range, in UTC and local fields; then 2000-2030 in a zone +// with DST. +function utcFields(d: Date): string { + return [ + d.getUTCFullYear(), + d.getUTCMonth(), + d.getUTCDate(), + d.getUTCDay(), + d.getUTCHours(), + d.getUTCMinutes(), + d.getUTCSeconds(), + d.getUTCMilliseconds(), + ].join(","); +} +const edges = [ + 0, -0, -0.5, 0.5, -1, 1, -999, -1000, -1001, 999.9, -86_400_000, -86_400_001, 86_399_999, + -62_198_755_200_000, -62_198_755_200_001, 8_640_000_000_000_000, -8_640_000_000_000_000, + 8_639_999_999_999_999, -8_639_999_999_999_999, 951_782_400_000, 4_107_542_399_999, +]; +process.env.TZ = "UTC"; +for (const t of edges) { + const x = new Date(t); + console.log(t, utcFields(x), fields(x)); +} +process.env.TZ = "America/Sao_Paulo"; +let walk = 0; +for (let t = 946_684_800_000; t < 1_893_456_000_000; t += 77_777_777) { + const x = new Date(t); + walk = (walk * 31 + x.getUTCDate() + x.getUTCHours() * 7 + x.getDate() + x.getMinutes() + x.getDay()) % 1_000_003; +} +console.log("walk", walk); diff --git a/tests/corpus/5919-record-unions-shared-arms.ts b/tests/corpus/5919-record-unions-shared-arms.ts new file mode 100644 index 000000000..89f34dd83 --- /dev/null +++ b/tests/corpus/5919-record-unions-shared-arms.ts @@ -0,0 +1,60 @@ +// Record unions that share arms: Third = D | A, First = A | B and +// Second = B | C. Only C is cyclic. Every arm of a union whose values the +// cycle collector can visit carries a collector header, and the shared +// arms pass that requirement from Second back through First to Third. +type A = { kind: "a"; v: number }; +type B = { kind: "b"; w: number }; +type C = { kind: "c"; kids: C[] }; +type D = { kind: "d"; label: string }; +type Third = D | A; +type First = A | B; +type Second = B | C; + +function third(u: Third): string { + return u.kind === "d" ? u.label : `a${u.v}`; +} + +function first(u: First): number { + return u.kind === "a" ? u.v : u.w; +} + +function second(u: Second): number { + return u.kind === "b" ? u.w : u.kids.length + u.kids.reduce((n, k) => n + second(k), 0); +} + +const zs: Third[] = [ + { kind: "d", label: "dee" }, + { kind: "a", v: 7 }, +]; +console.log(zs.map(third).join(",")); + +const xs: First[] = [ + { kind: "a", v: 1 }, + { kind: "b", w: 2 }, +]; +console.log(xs.map(first).join(",")); + +const leaf: C = { kind: "c", kids: [] }; +const ys: Second[] = [ + { kind: "b", w: 3 }, + { kind: "c", kids: [leaf, { kind: "c", kids: [leaf] }] }, +]; +console.log(ys.map(second).join(",")); + +// One record held through two unions. +const sharedA: A = { kind: "a", v: 5 }; +const sharedB: B = { kind: "b", w: 6 }; +const viaThird: Third = sharedA; +const viaFirst: First[] = [sharedA, sharedB]; +const viaSecond: Second = sharedB; +console.log(third(viaThird), viaFirst.map(first).join(","), second(viaSecond)); + +// A cyclic C reachable only through Second, dropped and rebuilt in a loop. +let total = 0; +for (let i = 0; i < 200; i++) { + const node: C = { kind: "c", kids: [] }; + node.kids.push(node); + const held: Second[] = [node, { kind: "b", w: i }]; + total += held.length + node.kids.length; +} +console.log("total", total); diff --git a/tests/corpus/5930-record-union-pointers.ts b/tests/corpus/5930-record-union-pointers.ts new file mode 100644 index 000000000..01882d730 --- /dev/null +++ b/tests/corpus/5930-record-union-pointers.ts @@ -0,0 +1,79 @@ +// Unions whose arms are all object literals (discriminated unions) hold the +// record itself: construction, narrowing, storage in arrays, maps, fields +// and closures, identity, and printing behave exactly as in Node. +type Shape = + | { kind: "circle"; r: number } + | { kind: "rect"; w: number; h: number } + | { kind: "tri"; b: number; h: number; label: string }; + +function area(s: Shape): number { + switch (s.kind) { + case "circle": + return Math.round(3.14159 * s.r * s.r); + case "rect": + return s.w * s.h; + case "tri": + return 0.5 * s.b * s.h; + } +} + +function make(i: number): Shape { + if (i % 3 === 0) return { kind: "circle", r: i }; + if (i % 3 === 1) return { kind: "rect", w: i, h: i + 1 }; + return { kind: "tri", b: i, h: 2, label: "t" + i }; +} + +const shapes: Shape[] = []; +for (let i = 0; i < 9; i++) shapes.push(make(i)); +let total = 0; +for (const s of shapes) total += area(s); +console.log("total", total); +console.log(shapes.map((s) => s.kind).join(",")); +console.log(shapes.filter((s) => s.kind === "rect").length); + +// Identity: the same record compares equal, a structurally equal one does not. +const first = shapes[0]!; +const again: Shape = shapes[0]!; +const copy: Shape = { kind: "circle", r: 0 }; +console.log(first === again, first === copy, shapes.indexOf(again), shapes.includes(copy)); + +// Mutation through one alias is visible through the union value. +const rect = { kind: "rect" as const, w: 2, h: 3 }; +const asShape: Shape = rect; +rect.w = 10; +console.log(area(asShape), asShape === rect); + +// Fields, maps and closures holding union values. +interface Holder { + current: Shape; + history: Shape[]; +} +const holder: Holder = { current: make(1), history: [] }; +for (let i = 0; i < 4; i++) { + holder.history.push(holder.current); + holder.current = make(i + 5); +} +console.log(holder.current.kind, holder.history.map(area).join(" ")); + +const byName = new Map(); +for (const s of shapes) byName.set(s.kind + area(s), s); +console.log(byName.size, byName.get("rect2")?.kind, byName.get("missing") === undefined); + +const pick = (k: Shape["kind"]): (() => Shape | undefined) => { + const found = shapes.find((s) => s.kind === k); + return () => found; +}; +const getTri = pick("tri"); +const tri = getTri(); +console.log(tri !== undefined && tri.kind === "tri" ? tri.label : "none"); + +// Printing and serialization see the plain objects. +console.log(shapes.slice(0, 3)); +console.log(JSON.stringify(shapes.slice(0, 3))); +const second = shapes[2]!; +console.log(`${String(shapes[1]!.kind)} ${"label" in second} ${"label" in shapes[1]!}`); + +// Spread copies and sorting. +const grown = shapes.map((s): Shape => (s.kind === "circle" ? { ...s, r: s.r + 1 } : s)); +grown.sort((a, b) => area(a) - area(b)); +console.log(grown.map((s) => `${s.kind}:${area(s)}`).join(" ")); diff --git a/tests/corpus/5931-record-union-unit-arms.ts b/tests/corpus/5931-record-union-unit-arms.ts new file mode 100644 index 000000000..1ade8a0a3 --- /dev/null +++ b/tests/corpus/5931-record-union-unit-arms.ts @@ -0,0 +1,67 @@ +// Record unions with undefined and null arms: NULL and the null sentinel +// encode the unit arms, optional properties keep their absent state, and +// narrowing, nullish operators and equality agree with Node. +type Ok = { ok: true; value: number }; +type Err = { ok: false; error: string }; +type Res = Ok | Err; + +function parse(text: string): Res { + const n = Number(text); + return Number.isNaN(n) ? { ok: false, error: `bad ${text}` } : { ok: true, value: n }; +} + +function lookup(table: Res[], i: number): Res | undefined { + return i < table.length ? table[i] : undefined; +} + +function maybe(i: number): Res | null | undefined { + if (i % 3 === 0) return null; + if (i % 3 === 1) return undefined; + return parse(String(i)); +} + +const table = ["1", "x", "3"].map(parse); +for (let i = 0; i < 4; i++) { + const r = lookup(table, i); + if (r === undefined) console.log(i, "missing"); + else if (r.ok) console.log(i, "value", r.value); + else console.log(i, "error", r.error); +} + +for (let i = 0; i < 6; i++) { + const m = maybe(i); + const tag = m === null ? "null" : m === undefined ? "undefined" : m.ok ? "ok" : "err"; + console.log(i, tag, m == null, m ?? "default", typeof m); +} + +// Optional properties of a record-union type. +interface Slot { + name: string; + result?: Res; + last?: Res | null; +} +const slots: Slot[] = [ + { name: "a" }, + { name: "b", result: parse("7") }, + { name: "c", result: parse("q"), last: null }, +]; +for (const s of slots) { + console.log(s.name, "result" in s, s.result?.ok, s.last === null, s.last === undefined); +} +console.log(JSON.stringify(slots)); +slots[0]!.result = table[1]; +slots[1]!.result = undefined; +console.log(slots.map((s) => (s.result === undefined ? "-" : s.result.ok ? "ok" : "err")).join("")); + +// Equality across arms and unit encodings. +const a: Res | undefined = table[0]; +const b: Res | undefined = undefined; +const c: Res | null = null; +console.log(a === table[0], a === b, b === undefined, c === null, (c as Res | null) === table[2]); + +// Unions of records reaching unknown and back. +const values: unknown[] = [table[0], table[1], undefined, null]; +for (const v of values) { + if (v === null || v === undefined) console.log("unit", v); + else console.log("object", JSON.stringify(v)); +} diff --git a/tests/corpus/5932-record-union-trees.ts b/tests/corpus/5932-record-union-trees.ts new file mode 100644 index 000000000..060cadc4b --- /dev/null +++ b/tests/corpus/5932-record-union-trees.ts @@ -0,0 +1,77 @@ +// A recursive discriminated union (an expression tree) whose leaf arm holds +// no references: every arm shares the cycle collector's header, so trees +// can be built, rewritten, shared and dropped in cycles. +type Expr = + | { kind: "num"; value: number } + | { kind: "neg"; arg: Expr } + | { kind: "add"; left: Expr; right: Expr } + | { kind: "ref"; target: { expr: Expr | null } }; + +function evaluate(e: Expr): number { + switch (e.kind) { + case "num": + return e.value; + case "neg": + return -evaluate(e.arg); + case "add": + return evaluate(e.left) + evaluate(e.right); + case "ref": + return e.target.expr === null ? 0 : evaluate(e.target.expr); + } +} + +function build(depth: number, seed: number): Expr { + if (depth === 0) return { kind: "num", value: seed % 10 }; + if (seed % 4 === 0) return { kind: "neg", arg: build(depth - 1, seed + 1) }; + return { kind: "add", left: build(depth - 1, seed * 3 + 1), right: build(depth - 1, seed + 7) }; +} + +function simplify(e: Expr): Expr { + switch (e.kind) { + case "neg": { + const inner = simplify(e.arg); + return inner.kind === "neg" ? inner.arg : inner.kind === "num" ? { kind: "num", value: -inner.value } : { kind: "neg", arg: inner }; + } + case "add": { + const l = simplify(e.left); + const r = simplify(e.right); + if (l.kind === "num" && r.kind === "num") return { kind: "num", value: l.value + r.value }; + return l === e.left && r === e.right ? e : { kind: "add", left: l, right: r }; + } + default: + return e; + } +} + +function show(e: Expr): string { + switch (e.kind) { + case "num": + return String(e.value); + case "neg": + return `-(${show(e.arg)})`; + case "add": + return `(${show(e.left)} + ${show(e.right)})`; + case "ref": + return "ref"; + } +} + +let checksum = 0; +for (let round = 0; round < 50; round++) { + const tree = build(6, round); + const simple = simplify(tree); + checksum += evaluate(tree) * 3 + evaluate(simple); + // A cycle through a record arm: the tree refers back to itself. + const target: { expr: Expr | null } = { expr: null }; + const loop: Expr = { kind: "add", left: tree, right: { kind: "ref", target } }; + target.expr = { kind: "num", value: round }; + checksum += evaluate(loop); + target.expr = loop.left; +} +console.log(checksum); +console.log(show(simplify(build(3, 5)))); +console.log(show(build(2, 4)), evaluate(build(4, 2))); + +const shared: Expr = { kind: "num", value: 4 }; +const twice: Expr = { kind: "add", left: shared, right: shared }; +console.log(evaluate(twice), twice.kind === "add" && twice.left === twice.right); diff --git a/tests/corpus/5933-record-union-conversions.ts b/tests/corpus/5933-record-union-conversions.ts new file mode 100644 index 000000000..80b411a9b --- /dev/null +++ b/tests/corpus/5933-record-union-conversions.ts @@ -0,0 +1,78 @@ +// Record unions meet other representations: widening into unions with +// string and number arms, narrowing back, `in` tests on undiscriminated +// arms, generic helpers, class fields and promises. +type Point = { x: number; y: number }; +type Named = { name: string }; +type Item = Point | Named; +type Wide = Item | string | number; + +function describe(i: Item): string { + return "x" in i ? `point(${i.x},${i.y})` : `named(${i.name})`; +} + +function widen(i: Item): Wide { + return i; +} + +function narrow(w: Wide): Item | undefined { + if (typeof w === "string" || typeof w === "number") return undefined; + return w; +} + +const items: Item[] = [{ x: 1, y: 2 }, { name: "a" }, { x: 3, y: 4 }]; +const wide: Wide[] = items.map(widen); +wide.push("text", 42); +for (const w of wide) { + const back = narrow(w); + console.log(typeof w, back === undefined ? "-" : describe(back), back !== undefined && items.includes(back)); +} + +// Generic helpers instantiate over the union. +function firstWhere(xs: T[], pred: (x: T) => boolean): T | undefined { + for (const x of xs) if (pred(x)) return x; + return undefined; +} +const named = firstWhere(items, (i) => "name" in i); +console.log(named === items[1], named !== undefined ? describe(named) : "none"); +const pair: [Item, Item] = [items[2]!, items[0]!]; +const [p0, p1] = pair; +console.log(describe(p0), describe(p1)); + +// Class fields and methods. +class Cursor { + private at: Item; + readonly seen: Item[] = []; + constructor(start: Item) { + this.at = start; + } + move(next: Item): Item { + const prev = this.at; + this.seen.push(prev); + this.at = next; + return prev; + } + get current(): Item { + return this.at; + } +} +const cursor = new Cursor(items[0]!); +for (const i of items) cursor.move(i); +console.log(describe(cursor.current), cursor.seen.map(describe).join(" ")); + +// Promises carry union payloads. +async function load(i: number): Promise { + await null; + return items[i % items.length]!; +} +async function main(): Promise { + const loaded: Item[] = []; + for (const i of [0, 1, 5]) loaded.push(await load(i)); + console.log(loaded.map(describe).join(" "), loaded[0] === items[0]); + const slow = new Promise((resolve) => setTimeout(() => resolve({ x: 9, y: 9 }), 5)); + const raced = await Promise.race([load(1), slow]); + console.log(describe(raced)); + const pending: Promise[] = [load(2), load(0)]; + const all = await Promise.all(pending); + console.log(all.map(describe).join(" "), all[1] === items[0]); +} +main().then(() => console.log("done")); diff --git a/tests/corpus/5934-record-union-set-members.ts b/tests/corpus/5934-record-union-set-members.ts new file mode 100644 index 000000000..46f838870 --- /dev/null +++ b/tests/corpus/5934-record-union-set-members.ts @@ -0,0 +1,25 @@ +// A record union used as Set elements and Map keys keeps the runtime's +// tagged representation; other unions over the same records convert to and +// from it without changing identity semantics. +type Point = { x: number; y: number }; +type Named = { name: string }; +type Item = Point | Named; + +const items: Item[] = [{ x: 1, y: 2 }, { name: "a" }, { x: 3, y: 4 }]; +const marked = new Set(); +marked.add(items[0]!); +marked.add(items[0]!); +marked.add({ x: 1, y: 2 }); +console.log(marked.size, marked.has(items[0]!), marked.has(items[1]!)); + +const counts = new Map(); +for (const i of [...items, items[0]!, items[2]!]) counts.set(i, (counts.get(i) ?? 0) + 1); +console.log([...counts.values()].join(",")); + +function maybeItem(i: number): Item | undefined { + return i < items.length ? items[i] : undefined; +} +for (let i = 0; i < 4; i++) { + const m = maybeItem(i); + console.log(i, m === undefined ? "none" : "x" in m ? m.x : m.name, m !== undefined && marked.has(m)); +} diff --git a/tests/corpus/5943-fields-read-before-assignment.ts b/tests/corpus/5943-fields-read-before-assignment.ts deleted file mode 100644 index 922b3c34d..000000000 --- a/tests/corpus/5943-fields-read-before-assignment.ts +++ /dev/null @@ -1,181 +0,0 @@ -// @transform-types -// Fields that a base constructor (or the constructor itself) can read -// through an override or a method call before they are assigned hold -// undefined then, as in Node, for every field kind. After construction -// the fields read and update normally. -class Point { - x: number; - constructor(x: number) { - this.x = x; - } -} -function show(label: string, value: number): string { - return `${label}=${value}`; -} -abstract class Widget { - log: string[] = []; - constructor() { - this.describe("base"); - } - abstract describe(when: string): void; -} -class Panel extends Widget { - label: string; - width: number; - visible: boolean; - origin: Point; - items: number[]; - options: { depth: number }; - key: string | number; - index: Map; - tags: Set; - count = 5; - constructor(label: string, readonly id: string) { - super(); - this.label = label; - this.width = 10; - this.visible = true; - this.origin = new Point(1); - this.items = [1, 2]; - this.options = { depth: 2 }; - this.key = 7; - this.index = new Map([["a", 1]]); - this.tags = new Set(["t"]); - this.describe("ctor"); - } - describe(when: string): void { - console.log(when, this.label, this.width, this.visible, this.origin, this.items, this.options, this.key, this.count, this.id); - console.log(when, `${this.label}:${this.width}`, this.width + 1, this.count * 2, this.label === undefined, typeof this.items, this.origin?.x, this.options?.depth ?? -1, this.index?.size, this.tags?.has("t")); - console.log(when, show(this.label, this.width), String(this.key), [this.label].length); - try { - console.log(this.label.length, this.items.length); - } catch (error) { - console.log(when, (error as Error).message); - } - } - grow(): number { - this.count++; - this.width += 5; - this.label += "!"; - this.items.push(this.width); - return this.count + this.width + this.items.length; - } -} -const panel = new Panel("main", "p1"); -console.log(panel.grow(), panel.label, panel.items, panel.count, panel.width); - -class Plain { - total: number; - name: string; - constructor() { - this.total = 1; - this.name = "plain"; - this.bump(); - } - bump(): void { - this.total++; - } -} -const plain = new Plain(); -console.log(plain.total, plain.name); - -class Toucher { - constructor() { - this.touch(); - } - touch(): void {} -} -class Counter extends Toucher { - count = 0; - label: string; - items: string[]; - constructor() { - super(); - this.label = "c"; - this.items = []; - } - override touch(): void { - this.count++; - this.label += "x"; - console.log(this.count, this.label, Number.isNaN(this.count)); - try { - this.items.push("a"); - } catch (error) { - console.log(error instanceof TypeError); - } - } -} -const c = new Counter(); -c.touch(); -console.log(c.count, c.label, c.items); - -class Shower { - constructor() { - this.show(); - } - show(): void {} -} -class Box extends Shower { - label: string; - value: T; - constructor(value: T) { - super(); - this.label = "box"; - this.value = value; - } - override show(): void { - console.log("show", this.label, this.value); - } -} -new Box(3).show(); -new Box("s").show(); - -type Ctor = new (...args: any[]) => T; -class Viewer { - constructor() { - this.show(); - } - show(): void {} -} -function Labeled>(Parent: B) { - return class extends Parent { - label = "mixed"; - override show(): void { - console.log("show", this.label); - } - }; -} -const L = Labeled(Viewer); -new L().show(); - -interface Crash { loc: number; message: string } -class Setupper { - constructor() { - this.setup(); - } - setup(): void {} -} -class Host extends Setupper { - crash: Crash | null; - name: string | null; - constructor(c: Crash | null) { - super(); - this.crash = c; - this.name = "host"; - } - override setup(): void { - console.log("setup", this.crash, this.name); - if (this.crash !== null) { - try { - console.log(this.crash.loc); - } catch (error) { - console.log(error instanceof TypeError); - } - } - } - report(): string { - if (this.crash) return `${this.crash.message} at ${this.crash.loc}`; - return this.name === null ? "none" : this.name.toUpperCase(); - } -} -console.log(new Host({ loc: 3, message: "boom" }).report(), new Host(null).report()); diff --git a/tests/corpus/5944-js-fields-read-before-assignment.js b/tests/corpus/5944-js-fields-read-before-assignment.js deleted file mode 100644 index e313a12b0..000000000 --- a/tests/corpus/5944-js-fields-read-before-assignment.js +++ /dev/null @@ -1,28 +0,0 @@ -// JavaScript class fields that the constructor creates after it calls a -// method, or after a base constructor that does, hold undefined until -// they are assigned, as in Node. -class Widget { - constructor(name) { - this.render(); - this.name = name; - this.size = 3; - this.title = "widget"; - this.parts = ["a"]; - this.meta = { id: 1 }; - } - render() { - console.log("render", this.name, this.title, this.size, this.parts, this.meta, this.size + 1, `${this.name}!`); - } -} -class Big extends Widget { - constructor() { - super("big"); - this.extra = 2; - } - render() { - super.render(); - console.log("big", this.extra); - } -} -new Widget("w").render(); -new Big().render(); diff --git a/tests/corpus/5976-builtin-class-aliases.ts b/tests/corpus/5976-builtin-class-aliases.ts new file mode 100644 index 000000000..692b3928d --- /dev/null +++ b/tests/corpus/5976-builtin-class-aliases.ts @@ -0,0 +1,109 @@ +// Builtin classes through const aliases: EventEmitter, the stream classes, +// the error classes, Map and Set construct, extend and test instanceof +// through an alias, and typeof, .name and === answer as in Node for the +// aliases and the direct spellings. +import { EventEmitter } from "node:events"; +import * as events from "node:events"; +import { PassThrough, Readable, Transform } from "node:stream"; +import * as stream from "node:stream"; +import { AssertionError } from "node:assert"; +import * as assert from "node:assert"; + +function lateEmitter(): boolean { + return new LateEmitter() instanceof EventEmitter && typeof LateMap === "function"; +} + +function localHoist(): string { + function inner(): string { + return `${typeof LocalMap} ${LocalMap.name} ${new LocalMap().size}`; + } + const LocalMap = Map; + return inner(); +} + +const Emitter = EventEmitter; +const NsEmitter = events.EventEmitter; +const Again = Emitter; +const LateEmitter = events.EventEmitter; +const LateMap = Map; + +const e = new Emitter(); +e.on("ping", (n: number) => console.log("ping", n)); +e.emit("ping", 1); +console.log("emitter", e instanceof Emitter, e instanceof NsEmitter, e instanceof Again); +console.log("hoisted use", lateEmitter(), localHoist()); + +class Bus extends Again { + send(text: string): void { + this.emit("text", text); + } +} +const bus = new Bus(); +bus.on("text", (text: string) => console.log("bus", text)); +bus.send("hello"); +console.log("subclass", bus instanceof Emitter, bus instanceof Bus); + +const Pass = PassThrough; +const R = Readable; +const T = stream.Transform; +const pass = new Pass(); +console.log("stream", pass instanceof Pass, pass instanceof R, pass instanceof T); +console.log("readable", new R({ read() {} }) instanceof Pass, e instanceof R); + +class Upper extends T { + override _transform( + chunk: Buffer, + _encoding: BufferEncoding, + callback: (error: Error | null, data?: string) => void, + ): void { + callback(null, chunk.toString().toUpperCase()); + } +} +const upper = new Upper(); +upper.on("data", (chunk: Buffer) => console.log("upper", chunk.toString())); +upper.end("quiet"); + +const TE = TypeError; +const RE = RangeError; +class LimitError extends RE { + constructor(limit: number) { + super(`over ${limit}`); + this.name = "LimitError"; + } +} +try { + throw new TE("bad input"); +} catch (err) { + console.log("error", err instanceof TE, err instanceof RE, err instanceof Error, (err as Error).message); +} +try { + throw new LimitError(3); +} catch (err) { + console.log("subclass error", err instanceof RE, err instanceof LimitError, (err as Error).message); +} +const AE = AssertionError; +try { + assert.strictEqual(1, 2); +} catch (err) { + console.log("assertion", err instanceof AE, err instanceof TE); +} +const viaNamespace = new assert.AssertionError({ message: "custom" }); +console.log("namespace error", viaNamespace.message, viaNamespace instanceof AE); + +const M = Map; +const S = Set; +const m = new M([ + ["a", 1], + ["b", 2], +]); +m.set("c", 3); +const s = new S([1, 2, 2, 3]); +console.log("collections", m.get("b"), m.size, s.size, m instanceof M, s instanceof S, s instanceof M); +const groups = M.groupBy([1, 2, 3, 4, 5], (n: number) => (n % 2 === 0 ? "even" : "odd")); +console.log("groupBy", groups.get("even"), groups.get("odd")); + +console.log("typeof", typeof Emitter, typeof R, typeof TE, typeof M, typeof EventEmitter, typeof Readable); +console.log("name", Emitter.name, NsEmitter.name, Pass.name, T.name, TE.name, AE.name, M.name, S.name); +console.log("direct name", EventEmitter.name, stream.Duplex.name, Error.name, RangeError.name, Map.name, Set.name); +console.log("identity", Emitter === EventEmitter, Again === NsEmitter, T === Transform, R === stream.Readable); +console.log("other class", (Pass as unknown) === (R as unknown), (M as unknown) !== (S as unknown), (TE as unknown) === (RE as unknown)); diff --git a/tests/corpus/5977-builtin-class-aliases.cjs b/tests/corpus/5977-builtin-class-aliases.cjs new file mode 100644 index 000000000..c149884ba --- /dev/null +++ b/tests/corpus/5977-builtin-class-aliases.cjs @@ -0,0 +1,45 @@ +// Builtin classes through const aliases in a CommonJS file. A copy of a +// name (or of a globalThis property) keeps its value, so it can be stored +// and compared; construction, instanceof, typeof and .name resolve to the +// class. Builtin module members construct through the alias. +const { EventEmitter } = require("node:events"); +const events = require("node:events"); +const zlib = require("node:zlib"); +const { PassThrough } = require("node:stream"); + +const Emitter = EventEmitter; +const NsEmitter = events.EventEmitter; +const Gz = zlib.Gzip; +const Gz2 = Gz; +const M = Map; +const GlobalSet = globalThis.Set; +const Pass = PassThrough; +const TE = TypeError; + +const e = new Emitter(); +e.on("ping", (n) => console.log("ping", n)); +e.emit("ping", 2); +console.log("emitter", e instanceof Emitter, e instanceof NsEmitter, e instanceof EventEmitter); + +const m = new M(); +m.set("k", 1); +const s = new GlobalSet(["a", "b", "a"]); +console.log("collections", m.get("k"), s.size, m instanceof M, s instanceof GlobalSet, m instanceof GlobalSet); + +const gz = new Gz2(); +console.log("zlib", gz instanceof Gz, gz instanceof zlib.Gunzip, Gz2 === zlib.Gzip); + +const pass = new Pass(); +console.log("stream", pass instanceof Pass, pass instanceof EventEmitter); + +try { + throw new TE("typed"); +} catch (err) { + console.log("error", err instanceof TE, err.message); +} + +const stored = [Emitter, M, GlobalSet, Pass]; +console.log("stored", stored.length, stored[0] === EventEmitter, stored[1] === Map, stored[2] === Set); +console.log("typeof", typeof Emitter, typeof NsEmitter, typeof Gz2, typeof M, typeof GlobalSet, typeof TE); +console.log("name", Emitter.name, NsEmitter.name, Gz2.name, M.name, GlobalSet.name, Pass.name, TE.name); +console.log("identity", Emitter === NsEmitter, M === Map, GlobalSet === Set, M === GlobalSet); diff --git a/tests/corpus/5978-builtin-class-alias-modules/classes.ts b/tests/corpus/5978-builtin-class-alias-modules/classes.ts new file mode 100644 index 000000000..73590d1e8 --- /dev/null +++ b/tests/corpus/5978-builtin-class-alias-modules/classes.ts @@ -0,0 +1,8 @@ +import { EventEmitter } from "node:events"; +import { PassThrough } from "node:stream"; + +export const Emitter = EventEmitter; +export const Pass = PassThrough; +export const Dict = Map; +export const Bag = Set; +export const Failure = TypeError; diff --git a/tests/corpus/5978-builtin-class-alias-modules/main.ts b/tests/corpus/5978-builtin-class-alias-modules/main.ts new file mode 100644 index 000000000..cf39cd712 --- /dev/null +++ b/tests/corpus/5978-builtin-class-alias-modules/main.ts @@ -0,0 +1,33 @@ +// Builtin class aliases exported from one module and used through named +// and namespace imports: construction, instanceof, typeof, .name and === +// resolve to the builtin class in the importing module. +import { EventEmitter } from "node:events"; +import { Emitter, Pass, Dict, Bag, Failure } from "./classes.ts"; +import * as classes from "./classes.ts"; + +const e = new Emitter(); +e.on("tick", (n: number) => console.log("tick", n)); +e.emit("tick", 7); +console.log("emitter", e instanceof Emitter, e instanceof classes.Emitter, e instanceof EventEmitter); + +const viaNs = new classes.Emitter(); +console.log("namespace emitter", viaNs instanceof Emitter, Emitter === classes.Emitter); + +const pass = new classes.Pass(); +console.log("stream", pass instanceof Pass, pass instanceof classes.Emitter); + +const dict = new Dict([["k", 1]]); +const viaNsDict = new classes.Dict(); +viaNsDict.set("a", "b"); +const bag = new classes.Bag([1, 1, 2]); +console.log("collections", dict.get("k"), viaNsDict.get("a"), bag.size, dict instanceof classes.Dict, bag instanceof Bag); + +try { + throw new classes.Failure("wrong type"); +} catch (err) { + console.log("error", err instanceof Failure, err instanceof classes.Failure, (err as Error).message); +} + +console.log("typeof", typeof Emitter, typeof classes.Pass, typeof Dict, typeof classes.Bag, typeof Failure); +console.log("name", Emitter.name, classes.Pass.name, Dict.name, classes.Bag.name, Failure.name); +console.log("identity", Dict === Map, classes.Bag === Set, Failure === TypeError, (Dict as unknown) === (Bag as unknown)); diff --git a/tests/corpus/5980-cycle-components.ts b/tests/corpus/5980-cycle-components.ts new file mode 100644 index 000000000..05e8ffed1 --- /dev/null +++ b/tests/corpus/5980-cycle-components.ts @@ -0,0 +1,101 @@ +// Collector passes whose candidates mix dead structures with live ones: the +// ring under construction when a pass runs, owners that keep an inner node +// of an otherwise dead ring, trees whose child arrays have holes and sparse +// slots, and a live structure too large for a pass to record. Each dead +// structure is reclaimed whole, and everything still reachable survives +// with its contents intact. The sanitized lane's RC audit fails if anything +// below leaks or is freed while reachable. + +class Node { + next: Node | null = null; + kids: (Node | undefined)[] = []; + owner: Node | null = null; + value: number; + constructor(value: number) { + this.value = value; + } +} + +function ring(size: number, base: number): Node { + const first = new Node(base); + let last = first; + for (let i = 1; i < size; i++) { + const node = new Node(base + i); + last.next = node; + last = node; + } + last.next = first; + return first; +} + +function tree(depth: number, owner: Node | null, value: number): Node { + const node = new Node(value); + node.owner = owner; + if (depth > 0) for (let i = 0; i < 3; i++) node.kids.push(tree(depth - 1, node, value * 3 + i)); + return node; +} + +function total(start: Node): number { + const seen = new Set(); + const stack: Node[] = [start]; + let sum = 0; + while (stack.length > 0) { + const node = stack.pop()!; + if (seen.has(node)) continue; + seen.add(node); + sum = (sum + node.value) % 1000003; + if (node.next) stack.push(node.next); + if (node.owner) stack.push(node.owner); + for (const kid of node.kids) if (kid) stack.push(kid); + } + return sum * 1000 + (seen.size % 1000); +} + +// Dead rings dropped while the next one is being built; a few are kept. +let rings = 0; +const kept: Node[] = []; +for (let round = 0; round < 3000; round++) { + const r = ring(5 + (round % 7), round); + rings = (rings + r.next!.value) % 1000003; + if (round % 500 === 0) kept.push(r); +} +console.log("rings", rings, kept.map(total).join(" ")); + +// Rings whose head is dropped while an owner keeps a node in the middle: +// the ring stays reachable through it and must survive whole. +const anchors: Node[] = []; +for (let round = 0; round < 400; round++) { + let mid = ring(30, round * 100); + for (let i = 0; i < 15; i++) mid = mid.next!; + if (round % 4 === 0) anchors.push(mid); +} +let anchored = 0; +for (const anchor of anchors) anchored = (anchored + total(anchor)) % 1000003; +console.log("anchored", anchors.length, anchored); + +// Trees with holes and sparse slots in their child arrays. +let trees = 0; +const shaped: Node[] = []; +for (let round = 0; round < 300; round++) { + const t = tree(4, null, round); + if (round % 3 === 0) t.kids.length = 6; + if (round % 5 === 0) t.kids[200] = tree(2, t, round + 7); + if (round % 7 === 0) t.kids[0]!.kids.length = 1; + trees = (trees + total(t)) % 1000003; + if (round % 60 === 0) shaped.push(t.kids[2]!); +} +console.log("trees", trees, shaped.map(total).join(" ")); + +// A live tree larger than one pass records, built and read while dead +// structures keep dying around it, then dropped. +const big = tree(10, null, 1); +let churn = 0; +for (let round = 0; round < 200; round++) { + const r = ring(50, round); + const branch = big.kids[round % 3]!.kids[(round >> 2) % 3]!; + branch.next = r; + churn = (churn + total(r)) % 1000003; + if (round % 50 === 0) churn = (churn + total(big)) % 1000003; + branch.next = null; +} +console.log("big", churn, total(big)); diff --git a/tests/corpus/5993-array-length-loop-resizing.ts b/tests/corpus/5993-array-length-loop-resizing.ts new file mode 100644 index 000000000..a1d5f6d11 --- /dev/null +++ b/tests/corpus/5993-array-length-loop-resizing.ts @@ -0,0 +1,167 @@ +// Index loops bounded by an array's length whose bodies resize that array, +// directly, through an alias, a helper or a closure: the bound is re-read on +// every pass, exactly like Node. + +function pushInBody(): string { + const a = [1, 2, 3]; + const seen: number[] = []; + for (let i = 0; i < a.length; i++) { + seen.push(a[i]); + if (a.length < 6) a.push(a[i] * 10); + } + return `${seen.join(",")} | ${a.length}`; +} + +function stridedPush(): string { + const a = [0, 1, 2, 3]; + let visits = 0; + for (let i = 0; i < a.length; i += 2) { + visits++; + if (a.length < 9) a.push(i); + } + return `${visits} ${a.join(",")}`; +} + +function truncateInBody(): string { + const a = [5, 6, 7, 8, 9, 10]; + let sum = 0; + for (let i = 0; i < a.length; i++) { + sum += a[i]; + if (i === 2) a.length = 4; + } + return `${sum} ${a.length}`; +} + +function growLengthInBody(): string { + const a = [1, 2]; + let holes = 0; + for (let i = 0; i < a.length; i++) { + if (a[i] === undefined) holes++; + if (i === 0) a.length = 5; + } + return `${holes} ${a.length}`; +} + +function helperPush(xs: number[], v: number): void { + if (xs.length < 7) xs.push(v); +} + +function pushThroughHelper(): string { + const a = [1, 2, 3]; + let sum = 0; + for (let i = 0; i < a.length; i++) { + sum += a[i]; + helperPush(a, i + 100); + } + return `${sum} ${a.length}`; +} + +function pushThroughClosure(): string { + const a = [1, 2, 3]; + const grow = (v: number): void => { + if (a.length < 6) a.push(v); + }; + let sum = 0; + for (let i = 0; i < a.length; i++) { + sum += a[i]; + grow(i); + } + return `${sum} ${a.length}`; +} + +function pushThroughAlias(): string { + const a = [1, 2, 3]; + const alias = a; + let sum = 0; + for (let i = 0; i < a.length; i++) { + sum += a[i]; + if (alias.length < 8) alias.push(i * 2); + } + return `${sum} ${a.length}`; +} + +function aliasedParameters(a: number[], out: number[]): string { + // `out` may be `a` itself. + let n = 0; + for (let i = 0; i < a.length; i++) { + n++; + if (out.length < 10) out.push(a[i]); + } + return `${n} ${a.length} ${out.length}`; +} + +function writePastEnd(): string { + const a = [1, 2, 3]; + let passes = 0; + for (let i = 0; i < a.length; i++) { + passes++; + if (i + 1 === a.length && a.length < 6) a[i + 1] = i; + } + return `${passes} ${a.join(",")}`; +} + +function shiftInBody(): string { + const a = [1, 2, 3, 4, 5, 6]; + const seen: number[] = []; + for (let i = 0; i < a.length; i++) { + seen.push(a[i]); + a.shift(); + } + return `${seen.join(",")} | ${a.join(",")}`; +} + +function spliceInDescendingLoop(): string { + const a = [1, 2, 3, 4, 5, 6, 7]; + for (let i = a.length - 1; i >= 0; i--) if (a[i] % 2 === 0) a.splice(i, 1); + return a.join(","); +} + +function whilePush(): string { + const a = [3, 1]; + let i = 0; + let sum = 0; + while (i < a.length) { + sum += a[i]; + if (a.length < 5) a.push(sum); + i++; + } + return `${sum} ${i} ${a.join(",")}`; +} + +function nestedResize(): string { + const rows = [[1], [2, 3], [4, 5, 6]]; + let total = 0; + for (let r = 0; r < rows.length; r++) { + const row = rows[r]; + for (let c = 0; c < row.length; c++) { + total += row[c]; + if (row.length < 3) row.push(c); + } + } + return `${total} ${rows.map((row) => row.length).join(",")}`; +} + +function pushIntoOther(): string { + const a = [2, 4, 6]; + const out: number[] = []; + for (let i = 0; i < a.length; i++) out.push(a[i] + 1); + for (let i = 0; i < out.length; i += 2) out.push(-1); + return `${out.join(",")}`; +} + +console.log(pushInBody()); +console.log(stridedPush()); +console.log(truncateInBody()); +console.log(growLengthInBody()); +console.log(pushThroughHelper()); +console.log(pushThroughClosure()); +console.log(pushThroughAlias()); +const shared = [7, 8, 9]; +console.log(aliasedParameters(shared, shared)); +console.log(aliasedParameters([7, 8, 9], [])); +console.log(writePastEnd()); +console.log(shiftInBody()); +console.log(spliceInDescendingLoop()); +console.log(whilePush()); +console.log(nestedResize()); +console.log(pushIntoOther()); diff --git a/tests/corpus/5994-array-length-sparse-loops.ts b/tests/corpus/5994-array-length-sparse-loops.ts new file mode 100644 index 000000000..589c1df60 --- /dev/null +++ b/tests/corpus/5994-array-length-sparse-loops.ts @@ -0,0 +1,133 @@ +// Index loops over sparse arrays with lengths beyond 2^31, holes, and arrays +// that become sparse while a loop runs. Dense-loop fast paths must leave all +// of these on the general path. + +function partialAscending(a: number[], limit: number): string { + let sum = 0; + let passes = 0; + for (let i = 0; i < a.length; i++) { + if (passes++ === limit) break; + sum += a[i] === undefined ? -1 : a[i]; + } + return `${passes} ${sum}`; +} + +function partialDescending(a: number[], limit: number): string { + let last = -1; + let passes = 0; + for (let i = a.length - 1; i >= 0; i--) { + if (passes++ === limit) break; + last = i; + } + return `${passes} ${last}`; +} + +function stridedFromEnd(a: number[]): string { + const hits: number[] = []; + for (let i = a.length - 1; i >= 0 && hits.length < 3; i -= 1000000000) hits.push(i); + return hits.join(","); +} + +function whileWithLimit(a: number[], limit: number): number { + let i = 0; + let present = 0; + while (i < a.length && i < limit) { + if (a[i] !== undefined) present++; + i++; + } + return present * 1000 + i; +} + +function writeAtCounter(a: number[], limit: number): string { + for (let i = 0; i < a.length && i < limit; i++) a[i] = i * 2; + return `${a.length} ${a[0]} ${a[limit - 1]} ${a[limit]}`; +} + +function holeSum(a: number[]): string { + let sum = 0; + let nan = 0; + for (let i = 0; i < a.length; i++) { + const v = a[i]; + if (Number.isNaN(v + 0)) nan++; + else sum += v; + } + return `${sum} ${nan}`; +} + +function firstValues(a: number[]): string { + let s = 0; + for (let i = 0; i < a.length; i++) { + if (i === 4) break; + s += a[i]; + } + let t = 0; + for (let i = a.length - 1; i >= 0; i--) { + if (i < a.length - 2) break; + t += a[i]; + } + let u = 0; + for (let i = 0; i < a.length; i += 2) { + if (i > 2) break; + u += a[i]; + } + return `${s} ${t} ${u}`; +} + +const huge: number[] = [1, 2, 3]; +huge.length = 3e9; +console.log(huge.length); +console.log(partialAscending(huge, 5)); +console.log(firstValues(huge), firstValues([1, 2, 3, 4, 5])); +console.log(partialDescending(huge, 4)); +console.log(stridedFromEnd(huge)); +console.log(whileWithLimit(huge, 6)); +console.log(writeAtCounter(huge, 4)); +console.log(huge.length); + +const edge: number[] = []; +edge[4294967294] = 7; +console.log(edge.length, partialDescending(edge, 2), partialAscending(edge, 3)); + +const holey = [1, 2]; +holey[3] = 4; +holey[5] = 6; +console.log(holeSum(holey)); +const sparse: number[] = [1, 2]; +sparse[10] = 5; +console.log(holeSum(sparse)); + +// Becomes sparse mid-loop: a write far past the end through another binding. +function sparseMidLoop(): string { + const a = [1, 2, 3, 4]; + const b = a; + let sum = 0; + for (let i = 0; i < a.length; i++) { + if (i === 1) b[3000000000] = 9; + if (i > 6) break; + sum += a[i] === undefined ? 100 : a[i]; + } + return `${sum} ${a.length}`; +} +console.log(sparseMidLoop()); + +function holeMidLoop(): string { + const a = [1, 2, 3, 4, 5]; + let sum = 0; + for (let i = 0; i < a.length; i++) { + if (i === 1) a.length = 3; + if (i === 2) a.length = 5; + sum += a[i] === undefined ? 100 : a[i]; + } + return `${sum}`; +} +console.log(holeMidLoop()); + +const objects: { v: number }[] = [{ v: 1 }, { v: 2 }, { v: 3 }]; +objects.length = 5; +try { + let total = 0; + for (let i = 0; i < objects.length; i++) total += objects[i].v; + console.log(total); +} catch (e) { + console.log(e instanceof TypeError ? "TypeError" : "other"); +} diff --git a/tests/corpus/5995-array-dense-loop-boundaries.ts b/tests/corpus/5995-array-dense-loop-boundaries.ts new file mode 100644 index 000000000..5efad78ec --- /dev/null +++ b/tests/corpus/5995-array-dense-loop-boundaries.ts @@ -0,0 +1,173 @@ +// Index loops whose array length the loop never changes, run over dense, +// holey, partially filled, aliased and short arrays, with fractional, +// negative and offset bounds, ascending, strided and descending counters. + +interface Point { + x: number; + y: number; +} + +function sum(a: number[]): number { + let s = 0; + for (let i = 0; i < a.length; i++) s += a[i]; + return s; +} + +function scale(a: number[], b: number[]): string { + for (let i = 0; i < a.length; i++) b[i] = a[i] * 2; + return `${b.length}:${b.join(",")}`; +} + +function inPlace(a: number[]): string { + for (let i = 0; i < a.length; i++) a[i] = a[i] + 1; + return a.join(","); +} + +function strided(a: number[], step: number): string { + let s = 0; + if (step === 2) for (let i = 0; i < a.length; i += 2) s += a[i]; + else for (let i = 1; i < a.length; i += 3) s += a[i]; + return `${s}`; +} + +function descending(a: number[]): string { + const out: number[] = []; + for (let i = a.length - 1; i >= 0; i--) out.push(a[i]); + return out.join(","); +} + +function descendingPairs(a: number[]): number { + let s = 0; + for (let i = a.length - 2; i > 0; i -= 2) s += a[i] - a[i + 1]; + return s; +} + +function offsetBounds(a: number[]): string { + let inner = 0; + for (let i = 1; i < a.length - 1; i++) inner += a[i]; + let inclusive = 0; + for (let i = 0; i <= a.length - 1; i++) inclusive += a[i]; + let diffs = 0; + for (let i = 0; i < a.length - 1; i++) diffs += a[i + 1] - a[i]; + return `${inner} ${inclusive} ${diffs}`; +} + +function fractionalBounds(a: number[]): string { + const seen: number[] = []; + for (let i = 0.5; i < a.length; i++) seen.push(a[i] === undefined ? -1 : 1); + for (let i = -2; i < a.length - 3; i++) seen.push(a[i] === undefined ? -2 : a[i]); + for (let i = 0; i < a.length - 0.5; i++) seen.push(i); + return seen.join(","); +} + +function whileLoop(a: number[]): number { + let i = 0; + let s = 0; + while (i < a.length) { + s += a[i] * i; + i++; + } + return s; +} + +function constLength(a: number[]): number { + const n = a.length; + let s = 0; + for (let i = 0; i < n; i++) s += a[i]; + return s; +} + +function points(ps: Point[]): string { + let s = 0; + for (let i = 0; i < ps.length; i++) s += ps[i].x * ps[i].y; + return `${s}`; +} + +function flags(bs: boolean[]): number { + let n = 0; + for (let i = 0; i < bs.length; i++) if (bs[i]) n++; + return n; +} + +function names(ss: string[]): string { + let out = ""; + for (let i = ss.length - 1; i >= 0; i--) out += ss[i]; + return out; +} + +function maybe(xs: (number | undefined)[]): string { + let s = 0; + let missing = 0; + for (let i = 0; i < xs.length; i++) { + const v = xs[i]; + if (v === undefined) missing++; + else s += v; + } + return `${s} ${missing}`; +} + +function nested(g: number[][]): number { + let s = 0; + for (let r = 0; r < g.length; r++) { + const row = g[r]; + for (let c = 0; c < row.length; c++) s += row[c] * (r + 1); + } + return s; +} + +function counterAsNumber(a: number[]): string { + let mixed = 0; + for (let i = 0; i < a.length; i++) mixed += i / 2 + (i | 0) + a[i] * -0; + return `${mixed} ${Object.is(mixed, -0)}`; +} + +function earlyExit(a: number[], target: number): number { + for (let i = 0; i < a.length; i++) if (a[i] === target) return i; + return -1; +} + +const dense = [1, 2, 3, 4, 5, 6, 7, 8]; +console.log(sum(dense), sum([]), sum([0.5, -0.25])); +console.log(scale(dense, new Array(8).fill(0))); +console.log(scale(dense, [9, 9])); +console.log(scale([1, 2, 3], [])); +const self = [1, 2, 3, 4]; +console.log(scale(self, self)); +console.log(inPlace([10, 20, 30])); +console.log(strided(dense, 2), strided(dense, 3)); +console.log(descending(dense), descending([])); +console.log(descendingPairs(dense), descendingPairs([1])); +console.log(offsetBounds(dense), offsetBounds([4]), offsetBounds([])); +console.log(fractionalBounds([1, 2, 3, 4, 5])); +console.log(whileLoop(dense)); +console.log(constLength(dense)); +console.log(points([{ x: 1, y: 2 }, { x: 3, y: 4 }])); +console.log(flags([true, false, true, true])); +console.log(names(["a", "b", "c"])); +console.log(maybe([1, undefined, 3])); +console.log(nested([[1, 2], [3], [], [4, 5, 6]])); +console.log(counterAsNumber([1, 2, 3])); +console.log(earlyExit(dense, 5), earlyExit(dense, 50)); + +// Holes and partially filled storage take the checked path. +const holey = [1]; +holey[2] = 3; +holey[4] = 5; +console.log(sum(holey), descending(holey), whileLoop(holey)); +const target = [0, 0, 0]; +target[4] = 0; +console.log(scale(dense.slice(0, 5), target)); +const gap: number[] = []; +gap[3] = 1; +console.log(sum(gap), scale([1, 2, 3, 4], gap)); +const shortened = [1, 2, 3, 4, 5, 6]; +shortened.length = 3; +shortened.length = 6; +console.log(sum(shortened), constLength(shortened)); +const holeyPoints: Point[] = [{ x: 1, y: 1 }]; +holeyPoints[2] = { x: 2, y: 2 }; +try { + console.log(points(holeyPoints)); +} catch (e) { + console.log(e instanceof TypeError ? "TypeError" : "other"); +} diff --git a/tests/corpus/5996-rejections-between-macrotasks.cjs b/tests/corpus/5996-rejections-between-macrotasks.cjs new file mode 100644 index 000000000..1afd8c8e8 --- /dev/null +++ b/tests/corpus/5996-rejections-between-macrotasks.cjs @@ -0,0 +1,33 @@ +'use strict'; +// Unhandled rejections are reported after the callback that left them, +// before the next due timer or the next immediate runs (Node's +// runNextTicks between macrotasks), also when several timers expire in +// the same turn. +process.on('unhandledRejection', (reason) => { + console.log('unhandledRejection', reason instanceof Error ? reason.message : JSON.stringify(reason)); +}); +process.on('uncaughtException', (err, origin) => console.log(origin, err)); + +const start = Date.now(); +setTimeout(() => void Promise.reject({ a: 1 }), 1); +setTimeout(() => void Promise.reject(new TypeError('typed reason')), 2); +setTimeout(() => { + throw 'thrown string'; +}, 3); +setTimeout(() => { + console.log('timer 4'); + Promise.reject('handled later').catch((e) => console.log('caught', e)); +}, 4); +setTimeout(() => console.log('timer 5'), 5); +// Every timer above is due once this returns. +while (Date.now() - start < 20) {} + +setTimeout(() => { + setImmediate(() => void Promise.reject('first immediate')); + setImmediate(() => { + console.log('second immediate'); + process.nextTick(() => console.log('tick')); + void Promise.reject('second immediate'); + }); + setImmediate(() => console.log('third immediate')); +}, 30); diff --git a/tests/corpus/6100-record-get-or-create.ts b/tests/corpus/6100-record-get-or-create.ts new file mode 100644 index 000000000..807cb303e --- /dev/null +++ b/tests/corpus/6100-record-get-or-create.ts @@ -0,0 +1,119 @@ +// Get-or-create over nested index-signature records without +// noUncheckedIndexedAccess: a missing key reads as undefined when the next +// statement tests the binding for absence, in every spelling of that test, +// at module scope and inside functions. A write with a computed key through +// a binding that may hold a missing element throws Node's TypeError after +// the key and the value evaluated. +type ExportMap = Record>; + +interface Target { + subpath: string; + condition: string; + target: string; +} + +function build(targets: Target[]): { exports: ExportMap; unmapped: string[] } { + const exports: ExportMap = {}; + const unmapped: string[] = []; + for (const t of targets) { + if (t.target === "") { + unmapped.push(t.subpath); + continue; + } + let entry = exports[t.subpath]; + if (entry === undefined) { + entry = {}; + exports[t.subpath] = entry; + } + entry[t.condition] = `./${t.target}`; + entry["types"] = `./${t.target}`; + } + return { exports: exports, unmapped: unmapped }; +} + +const result = build([ + { subpath: ".", condition: "import", target: "src/index.ts" }, + { subpath: ".", condition: "require", target: "src/index.ts" }, + { subpath: "./x", condition: "default", target: "" }, + { subpath: "./y", condition: "node", target: "src/y.ts" }, +]); +console.log(JSON.stringify(result.exports), result.unmapped.join(",")); + +function group(words: string[]): Record { + const byInitial: Record = {}; + for (const w of words) { + const k = w.slice(0, 1); + let list = byInitial[k]; + if (list == null) { + list = []; + byInitial[k] = list; + } + list.push(w); + } + return byInitial; +} +console.log(JSON.stringify(group(["apple", "avocado", "banana", "blueberry", "cherry"]))); + +function count(words: string[]): Record { + const counts: Record = {}; + for (const w of words) { + const n = counts[w]; + if (typeof n === "undefined") counts[w] = 1; + else counts[w] = n + 1; + } + return counts; +} +console.log(JSON.stringify(count(["a", "b", "a", "c", "a"]))); + +function lookup(table: Record, key: string): string { + const hit = table[key]; + if (hit !== undefined) return hit; + return `<${key} missing>`; +} +console.log(lookup({ a: "A" }, "a"), lookup({ a: "A" }, "b")); + +function rows(table: Record>, key: string, col: string): number { + const row = table[key]; + if (row != undefined) return row[col] ?? -1; + return -2; +} +console.log(rows({ r: { c: 3 } }, "r", "c"), rows({ r: { c: 3 } }, "r", "d"), rows({}, "x", "c")); + +// Module scope. +const nested: Record> = {}; +const key = process.argv.length > 99 ? "unused" : "k"; +let slot = nested[key]; +if (slot === undefined) { + slot = {}; + nested[key] = slot; +} +slot[key] = 1; +slot["fixed"] = 2; +console.log(JSON.stringify(nested)); + +const names: Record = { present: "yes" }; +const missing = names[key]; +if (!missing) console.log("falsy guard:", missing); +const present = names["present"]; +if (present === void 0) console.log("unreachable"); +else console.log("present:", present); + +// A write with a computed key through a binding that may hold a missing +// array element throws Node's TypeError naming the key, after evaluating +// the key and the value. +function probe(tables: Record[]): void { + const firstTable = tables[0]; + const lastTable = tables[5]; + const pick = (k: string): string => { + console.log("key", k); + return k; + }; + firstTable[pick("b")] = pick("second"); + console.log(JSON.stringify(firstTable)); + try { + lastTable[pick("c")] = pick("value"); + } catch (e) { + console.log((e as Error).name, (e as Error).message); + } +} +probe([{ a: "first" }]); diff --git a/tests/corpus/6101-record-get-or-create-unchecked/main.ts b/tests/corpus/6101-record-get-or-create-unchecked/main.ts new file mode 100644 index 000000000..a2b4ef705 --- /dev/null +++ b/tests/corpus/6101-record-get-or-create-unchecked/main.ts @@ -0,0 +1,70 @@ +// Get-or-create over nested index-signature records under +// noUncheckedIndexedAccess: after the presence guard the binding is still +// stored with its undefined arm, and reads, writes and absence probes with +// computed keys go through it (an internal compiler error before). +type ExportMap = Record>; + +function build(rows: { k: string; c: string; v: string }[]): ExportMap { + const map: ExportMap = {}; + for (const r of rows) { + let entry = map[r.k]; + if (entry === undefined) { + entry = {}; + map[r.k] = entry; + } + entry[r.c] = r.v; + } + return map; +} +console.log( + JSON.stringify( + build([ + { k: "a", c: "x", v: "1" }, + { k: "a", c: "y", v: "2" }, + { k: "b", c: "x", v: "3" }, + ]), + ), +); + +function touch(map: ExportMap, k: string): string { + let e = map[k]; + if (e === undefined) { + e = { [k]: "init" }; + map[k] = e; + } + const before = e[k] ?? "none"; + e[k] = e[k] + "!"; + return before + (k in e ? "+" : "-"); +} +const m: ExportMap = {}; +console.log(touch(m, "a"), touch(m, "a"), touch(m, "b"), JSON.stringify(m)); + +function tally(counts: Record>, group: string, item: string): void { + let row = counts[group]; + if (!row) { + row = {}; + counts[group] = row; + } + row[item] = (row[item] ?? 0) + 1; +} +const counts: Record> = {}; +for (const [g, i] of [ + ["fruit", "apple"], + ["fruit", "pear"], + ["veg", "kale"], + ["fruit", "apple"], +]) { + tally(counts, g!, i!); +} +console.log(JSON.stringify(counts)); + +// Module scope. +const top: ExportMap = {}; +const key = process.argv.length > 99 ? "unused" : "k"; +let slot = top[key]; +if (slot === undefined) { + slot = {}; + top[key] = slot; +} +slot[key] = "v"; +console.log(JSON.stringify(top), slot[key], slot["other"] ?? "missing"); diff --git a/tests/corpus/6101-record-get-or-create-unchecked/tsconfig.json b/tests/corpus/6101-record-get-or-create-unchecked/tsconfig.json new file mode 100644 index 000000000..f2e1c10cd --- /dev/null +++ b/tests/corpus/6101-record-get-or-create-unchecked/tsconfig.json @@ -0,0 +1,6 @@ +{ + "compilerOptions": { + "strict": true, + "noUncheckedIndexedAccess": true + } +} diff --git a/tests/corpus/6103-typeof-builtin-bindings.cjs b/tests/corpus/6103-typeof-builtin-bindings.cjs new file mode 100644 index 000000000..ecef94dd3 --- /dev/null +++ b/tests/corpus/6103-typeof-builtin-bindings.cjs @@ -0,0 +1,15 @@ +// typeof on bindings destructured from Node builtin modules: Node answers by +// the export's kind ("function" for functions and classes, "object" for +// namespace objects), never by how the binding is represented. +const { inspect, format, promisify, deprecate, isDeepStrictEqual } = require("util"); +const { join, resolve } = require("path"); +const { readFileSync, promises } = require("fs"); +const { EventEmitter } = require("events"); +const { createHash } = require("crypto"); +const util = require("node:util"); +const { inspect: aliased } = util; +console.log(typeof inspect, typeof format, typeof promisify, typeof deprecate, typeof isDeepStrictEqual); +console.log(typeof join, typeof resolve, typeof readFileSync, typeof promises); +console.log(typeof EventEmitter, typeof createHash, typeof aliased, typeof util.inspect); +console.log(typeof inspect === "function" ? "callable" : "not callable"); +console.log(join("a", "b"), format("%s=%d", "x", 1), inspect({ a: [1, 2] })); diff --git a/tests/corpus/6104-typeof-builtin-bindings.mjs b/tests/corpus/6104-typeof-builtin-bindings.mjs new file mode 100644 index 000000000..dd2976775 --- /dev/null +++ b/tests/corpus/6104-typeof-builtin-bindings.mjs @@ -0,0 +1,11 @@ +// typeof on named, default-destructured and namespace-destructured builtin +// bindings in an ES module: Node answers by the export's kind. +import util, { inspect as namedInspect, format } from "node:util"; +import * as path from "node:path"; +import { createHash } from "node:crypto"; +import { promises } from "node:fs"; +const { inspect, promisify } = util; +const { join } = path; +console.log(typeof inspect, typeof promisify, typeof namedInspect, typeof format); +console.log(typeof join, typeof createHash, typeof util.inspect, typeof promises); +console.log(namedInspect("s"), format("%d", 2), join("x", "y")); diff --git a/tests/corpus/6105-cjs-top-level-this/lib.js b/tests/corpus/6105-cjs-top-level-this/lib.js new file mode 100644 index 000000000..7addc1840 --- /dev/null +++ b/tests/corpus/6105-cjs-top-level-this/lib.js @@ -0,0 +1,11 @@ +// A required CommonJS module sees its own exports object as `this`, and the +// UMD test for a CommonJS host takes the module.exports branch. +const kind = typeof this; +let host = "global"; +if (typeof module !== "undefined" && module.exports) host = "commonjs"; +(function (root) { + console.log("lib wrapper root:", typeof root, root ? "truthy" : "falsy"); +})(this); +exports.describe = function () { + return kind + " " + host; +}; diff --git a/tests/corpus/6105-cjs-top-level-this/main.js b/tests/corpus/6105-cjs-top-level-this/main.js new file mode 100644 index 000000000..1da910171 --- /dev/null +++ b/tests/corpus/6105-cjs-top-level-this/main.js @@ -0,0 +1,12 @@ +// Top-level `this` in a CommonJS module is module.exports (an object), also +// inside arrow functions at module scope. +const lib = require("./lib.js"); +const self = this; +const arrow = () => this; +console.log(typeof this, self === this, arrow() === this, this ? "truthy" : "falsy"); +console.log(lib.describe()); +(function () { + console.log("call(this):", typeof this); +}).call(this); +if ((typeof module !== "undefined" && module !== null) && module.exports) console.log("commonjs host"); +else console.log("browser host"); diff --git a/tests/corpus/6105-cjs-top-level-this/package.json b/tests/corpus/6105-cjs-top-level-this/package.json new file mode 100644 index 000000000..a3c15a7a6 --- /dev/null +++ b/tests/corpus/6105-cjs-top-level-this/package.json @@ -0,0 +1 @@ +{ "type": "commonjs" } diff --git a/tests/corpus/6106-esm-top-level-this.mjs b/tests/corpus/6106-esm-top-level-this.mjs new file mode 100644 index 000000000..6079ecc1e --- /dev/null +++ b/tests/corpus/6106-esm-top-level-this.mjs @@ -0,0 +1,4 @@ +// Top-level `this` in an ES module stays undefined, also inside arrow +// functions at module scope. +const arrow = () => this; +console.log(typeof this, this === undefined, typeof arrow()); diff --git a/tests/corpus/6110-cjs-speculative-wrapper-fences.cjs b/tests/corpus/6110-cjs-speculative-wrapper-fences.cjs new file mode 100644 index 000000000..1b3546518 --- /dev/null +++ b/tests/corpus/6110-cjs-speculative-wrapper-fences.cjs @@ -0,0 +1,30 @@ +// @deferred-fences: 9 +// Function expressions lowered speculatively — the `.call(this)` and +// `.apply` receivers of CommonJS wrappers, and a computed class base — keep +// every statement of their bodies. A statement with no static lowering +// (here `module.exports` read as a value) compiles to its runtime fence +// instead of vanishing; these never run, so the binary matches Node. +// Coverage counts the three fenced statements nine times: each lowering of +// a wrapper body counts its statements, and the `.call` and `.apply` +// wrapper bodies are each lowered four times. +const verbose = process.argv.length > 99; +(function () { + var root = this; + console.log("call wrapper"); + if (verbose) console.log(root === module.exports); + console.log("call wrapper done"); +}).call(this); +(function (label) { + console.log("apply wrapper", label); + if (verbose) console.log(typeof module.exports); +}).apply(null, ["ok"]); +class Base { + hello() { + return "hello"; + } +} +class Derived extends (function () { + if (verbose) console.log(module.exports); + return Base; +})() {} +console.log(new Derived().hello()); diff --git a/tests/corpus/6120-type-only-import-graph/flags.ts b/tests/corpus/6120-type-only-import-graph/flags.ts new file mode 100644 index 000000000..5af639c0a --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/flags.ts @@ -0,0 +1,2 @@ +export type Flag = boolean; +console.log("flags evaluated"); diff --git a/tests/corpus/6120-type-only-import-graph/main.ts b/tests/corpus/6120-type-only-import-graph/main.ts new file mode 100644 index 000000000..251ce6809 --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/main.ts @@ -0,0 +1,21 @@ +// Node's type stripping erases `import type` and `export type … from`, so +// the modules and packages they name are never loaded: a package with no +// runtime entry, a module importing a builtin scriptc does not compile, and +// modules with load-time output. An inline `type` specifier keeps an empty +// import clause and still evaluates its module, as does a mixed clause. +import type { Frozen, Shape } from "types-only-pkg"; +import type { Label } from "./uses-vm.ts"; +import type * as Models from "./models.ts"; +import type { Reexported } from "./reexports.ts"; +import { type Mode, describe } from "./mixed.ts"; +import { type Flag } from "./flags.ts"; + +const shape: Shape = { width: 3, height: 4 }; +const frozen: Frozen = { width: 5, height: 6 }; +const label: Label = "area"; +const model: Models.Model = { id: 1 }; +const payload: Reexported = { ok: true }; +const mode: Mode = "fast"; +const flag: Flag = true; +console.log(label, shape.width * shape.height, model.id, payload.ok); +console.log(describe(mode), flag, frozen.width + frozen.height); diff --git a/tests/corpus/6120-type-only-import-graph/mixed.ts b/tests/corpus/6120-type-only-import-graph/mixed.ts new file mode 100644 index 000000000..ec5a08bbb --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/mixed.ts @@ -0,0 +1,8 @@ +export type { Payload } from "./payload.ts"; + +export type Mode = "fast" | "slow"; + +export function describe(mode: Mode): string { + return `mode=${mode}`; +} +console.log("mixed evaluated"); diff --git a/tests/corpus/6120-type-only-import-graph/models.ts b/tests/corpus/6120-type-only-import-graph/models.ts new file mode 100644 index 000000000..195cf9a17 --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/models.ts @@ -0,0 +1,4 @@ +export interface Model { + id: number; +} +console.log("models evaluated"); diff --git a/tests/corpus/6120-type-only-import-graph/node_modules/types-only-pkg/index.d.ts b/tests/corpus/6120-type-only-import-graph/node_modules/types-only-pkg/index.d.ts new file mode 100644 index 000000000..e9c1b5497 --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/node_modules/types-only-pkg/index.d.ts @@ -0,0 +1,6 @@ +export interface Shape { + width: number; + height: number; +} + +export type Frozen = { readonly [K in keyof T]: T[K] }; diff --git a/tests/corpus/6120-type-only-import-graph/node_modules/types-only-pkg/package.json b/tests/corpus/6120-type-only-import-graph/node_modules/types-only-pkg/package.json new file mode 100644 index 000000000..c2be952b9 --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/node_modules/types-only-pkg/package.json @@ -0,0 +1,5 @@ +{ + "name": "types-only-pkg", + "version": "1.0.0", + "types": "./index.d.ts" +} diff --git a/tests/corpus/6120-type-only-import-graph/payload.ts b/tests/corpus/6120-type-only-import-graph/payload.ts new file mode 100644 index 000000000..beb97c5ac --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/payload.ts @@ -0,0 +1,4 @@ +export interface Payload { + ok: boolean; +} +console.log("payload evaluated"); diff --git a/tests/corpus/6120-type-only-import-graph/reexports.ts b/tests/corpus/6120-type-only-import-graph/reexports.ts new file mode 100644 index 000000000..52e86f294 --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/reexports.ts @@ -0,0 +1,2 @@ +export type { Payload as Reexported } from "./payload.ts"; +console.log("reexports evaluated"); diff --git a/tests/corpus/6120-type-only-import-graph/tsconfig.json b/tests/corpus/6120-type-only-import-graph/tsconfig.json new file mode 100644 index 000000000..aee0ec940 --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/tsconfig.json @@ -0,0 +1,5 @@ +{ + "compilerOptions": { + "strict": true + } +} diff --git a/tests/corpus/6120-type-only-import-graph/uses-vm.ts b/tests/corpus/6120-type-only-import-graph/uses-vm.ts new file mode 100644 index 000000000..bed6646bf --- /dev/null +++ b/tests/corpus/6120-type-only-import-graph/uses-vm.ts @@ -0,0 +1,4 @@ +import * as vm from "node:vm"; + +export type Label = string; +console.log("uses-vm evaluated", vm.runInThisContext("1 + 1")); diff --git a/tests/corpus/6121-type-specifier-edges/a.ts b/tests/corpus/6121-type-specifier-edges/a.ts new file mode 100644 index 000000000..59d6385da --- /dev/null +++ b/tests/corpus/6121-type-specifier-edges/a.ts @@ -0,0 +1,10 @@ +import { b } from "./b.ts"; + +export interface AInfo { + tag: string; +} + +export function start(): string { + return `a+${b()}`; +} +console.log("a evaluated"); diff --git a/tests/corpus/6121-type-specifier-edges/b.ts b/tests/corpus/6121-type-specifier-edges/b.ts new file mode 100644 index 000000000..a71f62def --- /dev/null +++ b/tests/corpus/6121-type-specifier-edges/b.ts @@ -0,0 +1,7 @@ +import { type AInfo } from "./a.ts"; + +export function b(): string { + const info: AInfo = { tag: "b" }; + return info.tag; +} +console.log("b evaluated"); diff --git a/tests/corpus/6121-type-specifier-edges/config-barrel.ts b/tests/corpus/6121-type-specifier-edges/config-barrel.ts new file mode 100644 index 000000000..dc461f654 --- /dev/null +++ b/tests/corpus/6121-type-specifier-edges/config-barrel.ts @@ -0,0 +1,2 @@ +export { type Config } from "./config.ts"; +console.log("barrel evaluated"); diff --git a/tests/corpus/6121-type-specifier-edges/config.ts b/tests/corpus/6121-type-specifier-edges/config.ts new file mode 100644 index 000000000..1a42a9093 --- /dev/null +++ b/tests/corpus/6121-type-specifier-edges/config.ts @@ -0,0 +1,4 @@ +export interface Config { + name: string; +} +console.log("config evaluated"); diff --git a/tests/corpus/6121-type-specifier-edges/main.ts b/tests/corpus/6121-type-specifier-edges/main.ts new file mode 100644 index 000000000..23855b470 --- /dev/null +++ b/tests/corpus/6121-type-specifier-edges/main.ts @@ -0,0 +1,10 @@ +// An inline `type` specifier keeps its module request under Node's type +// stripping: `export { type X } from` and `import { type X } from` both +// evaluate the named module, once, in source order. A cycle closed only by +// such an import binds nothing at runtime and runs like a side-effect +// import. +import { type Config } from "./config-barrel.ts"; +import { start } from "./a.ts"; + +const config: Config = { name: "demo" }; +console.log("main", config.name, start()); diff --git a/tests/corpus/6122-punycode-conversions.ts b/tests/corpus/6122-punycode-conversions.ts new file mode 100644 index 000000000..061987a78 --- /dev/null +++ b/tests/corpus/6122-punycode-conversions.ts @@ -0,0 +1,45 @@ +// @exit: 1 +// @no-node-shims +// node:punycode's conversions (punycode.js 2.1.0): encode/decode of bare +// labels, toASCII/toUnicode over domains and email addresses (only the part +// after '@' maps; every RFC 3490 separator splits labels), ucs2 code point +// arrays with astral pairs and lone surrogates, and the RangeErrors of +// malformed input. A static import links the module before anything runs, +// so its DEP0040 report precedes the program and no 'warning' listener the +// program registers sees it (the Node side runs without the preload shim, +// which would delay that report past the program's first turn). The report +// carries the process id, so the @exit directive keeps stderr out of the +// comparison. +import * as punycode from "node:punycode"; + +process.on("warning", (w) => console.log("listener", w.name)); + +const show = (label: string, f: () => unknown): void => { + try { + console.log(label, JSON.stringify(f())); + } catch (e) { + const err = e as Error; + console.log(label, "threw", err instanceof RangeError, err.name, err.message); + } +}; + +console.log(punycode.version); +for (const s of ["mañana", "bücher", "ü", "abc", "", "-> $1.00 <-", "ليهمابتكلموشعربي؟", "他们为什么不说中文", "😀x", "MaJiでKoiする5秒前"]) { + show(`encode ${s}`, () => punycode.encode(s)); + show(`roundtrip ${s}`, () => punycode.decode(punycode.encode(s))); +} +for (const s of ["maana-pta", "bcher-kva", "tda", "abc-", "-abc", "ABC-kva", "a-b-c-kva", "ü-kva", "kva-ü", "99999999999", "zzzzzzzzzzzzz", "a-", "-"]) { + show(`decode ${s}`, () => punycode.decode(s)); +} +for (const d of ["mañana.com", "bücher.de", "example.com", "", "a..b", "ex@mple@bücher.de", "user@bücher.de", "日本。jp", "a.b。c", "ÄÖÜ.test", "a.b."]) { + show(`toASCII ${d}`, () => punycode.toASCII(d)); +} +for (const d of ["xn--maana-pta.com", "xn--bcher-kva.de", "XN--bcher-kva.de", "xn--BCHER-KVA.de", "example.com", "user@xn--bcher-kva.de", "xn--.x", "xn--ü.x", "xn--zzzzzzzzzzzzz.x", "xn--a-Kva.x"]) { + show(`toUnicode ${d}`, () => punycode.toUnicode(d)); +} +show("ucs2.decode", () => punycode.ucs2.decode("aé😀\ud800b\udc00")); +show("ucs2.encode", () => punycode.ucs2.encode([97, 0xe9, 0x1f600, 0xd800, 0xdc00, 98])); +show("ucs2.encode join", () => punycode.ucs2.encode([0xd83d, 0xde00]).length); +show("ucs2.encode range", () => punycode.ucs2.encode([0x110000])); +show("ucs2 roundtrip", () => punycode.ucs2.encode(punycode.ucs2.decode("x😀\udfffy")) === "x😀\udfffy"); +process.exitCode = 1; diff --git a/tests/corpus/6124-punycode-require-warnings.cjs b/tests/corpus/6124-punycode-require-warnings.cjs new file mode 100644 index 000000000..c3255e6b7 --- /dev/null +++ b/tests/corpus/6124-punycode-require-warnings.cjs @@ -0,0 +1,32 @@ +// DEP0040 for require("punycode"): reported once, on the next tick after +// the requiring turn, to the stderr report and the 'warning' listeners; +// require("node:punycode") is the same module and does not report again. +// process.noDeprecation drops the report and the listener call. The program +// re-runs itself per case and prints each child's output with process ids +// normalized. +"use strict"; +const { spawnSync } = require("child_process"); + +const mode = process.argv[2]; +if (mode === "direct") { + process.on("warning", (w) => console.log("listener", w.name, w.code, w.message)); + process.nextTick(() => console.log("earlier tick")); + console.log("before require"); + const direct = require("punycode"); + console.log("after require", direct.toASCII("bücher.de")); + const prefixed = require("node:punycode"); + console.log("prefixed", prefixed.toUnicode("xn--bcher-kva.de")); + setTimeout(() => console.log("timer"), 0); +} else if (mode === "silenced") { + process.on("warning", (w) => console.log("listener", w.code)); + process.noDeprecation = true; + const silencedModule = require("punycode"); + console.log("silenced", silencedModule.encode("ü")); +} else { + for (const child of ["direct", "silenced"]) { + const run = spawnSync(process.execPath, [process.argv[1], child], { encoding: "utf8" }); + console.log(`== ${child}: status ${run.status}`); + console.log(run.stdout); + console.log(run.stderr.replace(/\(node:\d+\)/g, "(node:)")); + } +} diff --git a/tests/corpus/6125-constants-module.cjs b/tests/corpus/6125-constants-module.cjs new file mode 100644 index 000000000..d7a547ac5 --- /dev/null +++ b/tests/corpus/6125-constants-module.cjs @@ -0,0 +1,26 @@ +// node:constants (deprecated in documentation only, so no warning): one +// frozen object merging os.constants' dlopen, errno, priority and signals +// groups with fs.constants and crypto.constants, in that order. Members +// read like the owning modules' constants, and a constant the platform +// lacks is absent (graceful-fs probes O_SYMLINK with hasOwnProperty). +const constants = require("constants"); +const nodeConstants = require("node:constants"); +const os = require("os"); + +console.log(typeof constants, Object.isFrozen(constants), constants === nodeConstants); +console.log(constants.O_RDONLY, constants.O_WRONLY, constants.O_RDWR); +console.log(constants.O_WRONLY | constants.O_CREAT | constants.O_TRUNC); +console.log(constants.S_IFMT, constants.S_IFREG, constants.S_IFDIR, constants.S_IRWXU); +console.log(constants.F_OK, constants.R_OK, constants.W_OK, constants.X_OK); +console.log(constants.COPYFILE_EXCL, constants.UV_FS_COPYFILE_FICLONE, constants.UV_DIRENT_DIR); +console.log(constants.SIGTERM === os.constants.signals.SIGTERM, constants.SIGINT); +console.log(constants.ENOENT === os.constants.errno.ENOENT, constants.EACCES); +console.log(constants.PRIORITY_HIGH, constants.PRIORITY_LOW, constants.RTLD_LAZY); +console.log(constants.TLS1_2_VERSION, constants.RSA_PKCS1_PADDING, typeof constants.defaultCoreCipherList); +console.log(constants.hasOwnProperty("O_SYMLINK") === ("O_SYMLINK" in constants)); +console.log(constants.hasOwnProperty("O_RDWR"), "SIGKILL" in constants, "UV_UDP_REUSEADDR" in constants); +console.log(constants.hasOwnProperty("NOT_A_CONSTANT"), "dlopen" in constants, "errno" in constants); +const keys = Object.keys(constants); +console.log(keys.indexOf("RTLD_LAZY") < keys.indexOf("E2BIG"), keys.indexOf("E2BIG") < keys.indexOf("PRIORITY_LOW")); +console.log(keys.indexOf("PRIORITY_LOW") < keys.indexOf("SIGHUP"), keys.indexOf("SIGHUP") < keys.indexOf("O_RDONLY")); +console.log(keys.indexOf("O_RDONLY") < keys.indexOf("OPENSSL_VERSION_NUMBER"), keys.includes("POINT_CONVERSION_HYBRID")); diff --git a/tests/corpus/6126-punycode-dependency/main.cjs b/tests/corpus/6126-punycode-dependency/main.cjs new file mode 100644 index 000000000..530503a71 --- /dev/null +++ b/tests/corpus/6126-punycode-dependency/main.cjs @@ -0,0 +1,12 @@ +// A dependency under node_modules that requires node:punycode: Node's +// isInsideNodeModules() rule skips DEP0040 for it, so nothing reaches stderr +// or the 'warning' listeners, and the program's own later require of the +// cached module does not report either. +"use strict"; +process.on("warning", (w) => console.log("listener", w.code)); +const toAscii = require("puny-dep"); +const punycode = require("punycode"); + +console.log(toAscii("mañana.com"), toAscii("example.org")); +console.log(punycode.toUnicode("xn--maana-pta.com")); +setTimeout(() => console.log("timer"), 0); diff --git a/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/index.d.ts b/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/index.d.ts new file mode 100644 index 000000000..89b712210 --- /dev/null +++ b/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/index.d.ts @@ -0,0 +1,2 @@ +declare function toAscii(domain: string): string; +export = toAscii; diff --git a/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/index.js b/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/index.js new file mode 100644 index 000000000..417c44889 --- /dev/null +++ b/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/index.js @@ -0,0 +1,8 @@ +// A dependency requiring node:punycode: Node's isInsideNodeModules() rule +// skips DEP0040 for it. +"use strict"; +const punycode = require("punycode"); + +module.exports = function toAscii(domain) { + return punycode.toASCII(domain); +}; diff --git a/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/package.json b/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/package.json new file mode 100644 index 000000000..71edeb25c --- /dev/null +++ b/tests/corpus/6126-punycode-dependency/node_modules/puny-dep/package.json @@ -0,0 +1,5 @@ +{ + "name": "puny-dep", + "version": "1.0.0", + "main": "index.js" +} diff --git a/tests/corpus/js-builder-string-method-names/builder.js b/tests/corpus/js-builder-string-method-names/builder.js new file mode 100644 index 000000000..36824c317 --- /dev/null +++ b/tests/corpus/js-builder-string-method-names/builder.js @@ -0,0 +1,22 @@ +// Prototype getters named like String.prototype's HTML methods (bold, +// italics, link, big): each read builds a callable that wraps its input. +const wrap = (tag) => ({ + get() { + const parent = this; + const builder = (text) => builder.prefix(text); + builder.prefix = (text) => parent.prefix(`<${tag}>` + text + ``); + Object.setPrototypeOf(builder, proto); + return builder; + }, +}); +const proto = Object.defineProperties(() => {}, { + bold: wrap("b"), + italics: wrap("i"), + link: wrap("a"), + big: wrap("big"), + red: wrap("red"), +}); +const root = (text) => text; +root.prefix = (text) => text; +Object.setPrototypeOf(root, proto); +export default root; diff --git a/tests/corpus/js-builder-string-method-names/main.ts b/tests/corpus/js-builder-string-method-names/main.ts new file mode 100644 index 000000000..5cab8e4ca --- /dev/null +++ b/tests/corpus/js-builder-string-method-names/main.ts @@ -0,0 +1,14 @@ +// A TypeScript caller reaching JavaScript builders whose member names +// collide with String.prototype methods: the receiver is a function, so +// each call reads the function's inherited getter, not the string method. +import builder from "./builder.js"; + +// TypeScript infers no members for the builder's prototype getters. +const tags: any = builder; + +console.log(tags.bold("strong")); +console.log(tags.red.bold("nested")); +console.log(tags.italics.link("deep").length); +console.log(tags.big.red.italics("three")); +const chain = tags.link; +console.log(chain.bold("from a binding")); diff --git a/tests/corpus/js-callable-builder.js b/tests/corpus/js-callable-builder.js new file mode 100644 index 000000000..3697d788d --- /dev/null +++ b/tests/corpus/js-callable-builder.js @@ -0,0 +1,110 @@ +// A chalk-style builder: every style read through a function's runtime +// prototype chain runs a getter that creates a new callable builder, +// links it to a shared prototype with Object.setPrototypeOf, and caches +// it on the receiver with Object.defineProperty. +const GENERATOR = Symbol("GENERATOR"); +const STYLER = Symbol("STYLER"); +const IS_EMPTY = Symbol("IS_EMPTY"); +const codes = { red: [31, 39], green: [32, 39], bold: [1, 22], underline: [4, 24] }; +let getterRuns = 0; + +const styles = Object.create(null); +for (const [name, [open, close]] of Object.entries(codes)) { + styles[name] = { + get() { + getterRuns++; + const builder = createBuilder(this, createStyler(`<${open}>`, ``, this[STYLER]), this[IS_EMPTY]); + Object.defineProperty(this, name, { value: builder }); + return builder; + }, + }; +} +styles.visible = { + get() { + const builder = createBuilder(this, this[STYLER], true); + Object.defineProperty(this, "visible", { value: builder }); + return builder; + }, +}; + +const proto = Object.defineProperties(() => {}, { + ...styles, + level: { + enumerable: true, + get() { + return this[GENERATOR].level; + }, + set(level) { + this[GENERATOR].level = level; + }, + }, +}); + +function createStyler(open, close, parent) { + if (parent === undefined) return { open, close, openAll: open, closeAll: close, parent }; + return { open, close, openAll: parent.openAll + open, closeAll: close + parent.closeAll, parent }; +} + +function createBuilder(self, styler, isEmpty) { + const builder = (...args) => applyStyle(builder, args.length === 1 ? "" + args[0] : args.join(" ")); + Object.setPrototypeOf(builder, proto); + builder[GENERATOR] = self; + builder[STYLER] = styler; + builder[IS_EMPTY] = isEmpty; + return builder; +} + +function applyStyle(self, text) { + if (self.level <= 0 || !text) return self[IS_EMPTY] ? "" : text; + let styler = self[STYLER]; + if (styler === undefined) return text; + const { openAll, closeAll } = styler; + if (text.includes(" strings.join(" "); + chalk.level = options && options.level !== undefined ? options.level : 1; + Object.setPrototypeOf(chalk, createChalk.prototype); + return chalk; +} +Object.setPrototypeOf(createChalk.prototype, Function.prototype); +Object.defineProperties(createChalk.prototype, styles); + +const chalk = createChalk(); +console.log(chalk.red("x")); +console.log(chalk.red.bold("nested")); +console.log(chalk.green("a " + chalk.red("b") + " c")); +console.log(chalk.underline.green.bold("three", "args", 3)); +console.log(JSON.stringify(chalk.red("")), chalk.visible("v"), chalk.red.visible("rv")); +console.log(chalk("plain", "call"), typeof chalk, typeof chalk.red, typeof chalk.red.bold); +console.log(chalk.red === chalk.red, chalk.red.bold === chalk.red.bold, getterRuns); +console.log(Object.getPrototypeOf(chalk) === createChalk.prototype); +console.log(Object.getPrototypeOf(chalk.red) === proto, Object.getPrototypeOf(chalk.red.bold) === proto); +console.log(Object.getPrototypeOf(createChalk.prototype) === Function.prototype); +console.log(chalk.apply(null, ["via", "apply"])); +console.log(Object.keys(chalk).join(","), Object.hasOwn(chalk, "red"), "red" in chalk); + +const red = chalk.red; +chalk.level = 0; +console.log(red("off"), red.level, chalk.level, JSON.stringify(red.visible("hidden"))); +red.level = 2; +console.log(chalk.level, chalk.green.level, red("on")); +const quiet = createChalk({ level: 0 }); +console.log(quiet.red.bold("quiet"), quiet.level, chalk.level); diff --git a/tests/corpus/js-class-constructor-returns/factory.js b/tests/corpus/js-class-constructor-returns/factory.js new file mode 100644 index 000000000..9fe0b84c4 --- /dev/null +++ b/tests/corpus/js-class-constructor-returns/factory.js @@ -0,0 +1,35 @@ +// Classes whose constructor returns a value: `new` answers that value. +const make = (options = {}) => { + const fn = (...parts) => parts.join(options.separator ?? " "); + fn.level = options.level ?? 1; + return fn; +}; + +export class Joiner { + constructor(options) { + return make(options); + } +} + +export class Record { + constructor(name, size) { + const value = { name, size, kind: "record" }; + if (size > 10) return { ...value, large: true }; + return value; + } +} + +let constructed = 0; +export class Counted { + constructor() { + constructed++; + return { id: constructed }; + } +} + +const j = new Joiner({ separator: "-", level: 3 }); +console.log(typeof j, j("a", "b", "c"), j.level); +const plain = new Joiner(); +console.log(plain("x", "y"), plain.level); +console.log(JSON.stringify(new Record("box", 4)), JSON.stringify(new Record("crate", 40))); +console.log(new Counted().id, new Counted().id, constructed); diff --git a/tests/corpus/js-class-constructor-returns/main.ts b/tests/corpus/js-class-constructor-returns/main.ts new file mode 100644 index 000000000..800016329 --- /dev/null +++ b/tests/corpus/js-class-constructor-returns/main.ts @@ -0,0 +1,11 @@ +// The same classes constructed from TypeScript: the values come from the +// constructors' returns. +import { Counted, Joiner, Record } from "./factory.js"; + +// TypeScript types `new` by the class; the values are what the constructors return. +const joiner: any = new Joiner({ separator: "+" }); +console.log(joiner("1", "2")); +const record = new Record("tube", 11); +console.log(JSON.stringify(record)); +const counted: any = new Counted(); +console.log(counted.id); diff --git a/tests/corpus/js-function-prototype-value.js b/tests/corpus/js-function-prototype-value.js new file mode 100644 index 000000000..3efe4540e --- /dev/null +++ b/tests/corpus/js-function-prototype-value.js @@ -0,0 +1,17 @@ +// Function.prototype as a value: a callable answering undefined, with an +// empty name and length 0, usable as a prototype-chain link. +const fp = Function.prototype; +console.log(typeof fp, fp === Function.prototype, JSON.stringify(fp.name), fp.length); +console.log(fp(), fp(1, 2, 3)); + +function Base() {} +Object.setPrototypeOf(Base.prototype, Function.prototype); +console.log(Object.getPrototypeOf(Base.prototype) === Function.prototype); + +const callable = (...args) => args.length; +Object.setPrototypeOf(callable, Base.prototype); +console.log(callable(1, 2), Object.getPrototypeOf(callable) === Base.prototype); +console.log(callable.apply(null, [1, 2, 3]), callable.call(null, 1)); + +const plain = Object.setPrototypeOf({ own: 1 }, Function.prototype); +console.log(typeof plain, plain.own, Object.getPrototypeOf(plain) === Function.prototype); diff --git a/tests/corpus/js-number-parse-dynamic.js b/tests/corpus/js-number-parse-dynamic.js new file mode 100644 index 000000000..e4b050046 --- /dev/null +++ b/tests/corpus/js-number-parse-dynamic.js @@ -0,0 +1,8 @@ +// Number.parseInt / Number.parseFloat over checked-dynamic values apply +// the parsers' own ToString, as supports-color does with FORCE_COLOR. +const env = JSON.parse('{"FORCE_COLOR":"2","DEPTH":" 3.5px","EMPTY":"","HEX":"ff"}'); +console.log(Number.parseInt(env.FORCE_COLOR, 10), Math.min(Number.parseInt(env.FORCE_COLOR, 10), 3)); +console.log(Number.parseInt(env.MISSING, 10), Number.parseInt(env.EMPTY, 10), Number.parseInt(env.HEX, 16)); +console.log(Number.parseFloat(env.DEPTH), Number.parseFloat(env.MISSING)); +const values = JSON.parse("[12, true, null]"); +console.log(Number.parseInt(values[0], 10), Number.parseInt(values[1], 36), Number.parseInt(values[2], 36)); diff --git a/tests/corpus/js-process-destructure.mjs b/tests/corpus/js-process-destructure.mjs new file mode 100644 index 000000000..81660e22c --- /dev/null +++ b/tests/corpus/js-process-destructure.mjs @@ -0,0 +1,12 @@ +// Data members of the process object taken off by destructuring. +import process from "node:process"; + +const { env, argv, platform } = process; +console.log(env.SCRIPTC_TEST_ENV, "SCRIPTC_TEST_ENV" in env, "SCRIPTC_NOT_SET_ANYWHERE" in env); +console.log(Array.isArray(argv), typeof argv[0], platform === process.platform); + +function readFlags() { + const { env: environment, pid } = process; + return [typeof environment.SCRIPTC_TEST_ENV, pid === process.pid]; +} +console.log(readFlags().join(" ")); diff --git a/tests/corpus/js-reshaped-object-literal.js b/tests/corpus/js-reshaped-object-literal.js new file mode 100644 index 000000000..c184cc388 --- /dev/null +++ b/tests/corpus/js-reshaped-object-literal.js @@ -0,0 +1,42 @@ +// An object literal the program reshapes at run time (runtime-keyed +// writes, defineProperty, delete): every read sees the live object, not +// the layout TypeScript inferred for the literal (ansi-styles' +// assembleStyles shape). +const styles = { + modifier: { reset: [0, 0], bold: [1, 22] }, + color: { red: [31, 39], green: [32, 39] }, +}; + +function assemble() { + const codes = new Map(); + for (const [groupName, group] of Object.entries(styles)) { + for (const [styleName, style] of Object.entries(group)) { + styles[styleName] = { open: `<${style[0]}>`, close: `` }; + group[styleName] = styles[styleName]; + codes.set(style[0], style[1]); + } + Object.defineProperty(styles, groupName, { value: group, enumerable: false }); + } + Object.defineProperty(styles, "codes", { value: codes, enumerable: false }); + styles.color.close = ""; + styles.color.ansi = (code) => `[${code}]`; + Object.defineProperties(styles, { + double: { value: (n) => n * 2, enumerable: false }, + }); + return styles; +} + +const ansi = assemble(); +console.log(Object.keys(ansi).join(",")); +console.log(ansi.red.open, ansi.color.red.close, ansi.color.close, ansi.color.ansi(5)); +console.log(ansi.codes.get(31), ansi.codes.size, ansi.double(21)); +console.log(ansi.modifier.bold === ansi.bold, Object.keys(ansi.color).join(",")); +for (const [name, style] of Object.entries(ansi)) console.log(name, style.open, style.close); + +const table = { a: 1, b: 2, c: 3 }; +const drop = (key) => { + delete table[key]; +}; +drop("b"); +table["d" + 1] = 4; +console.log(JSON.stringify(table), Object.keys(table).length, "b" in table); diff --git a/tests/corpus/scratch-string-slices.ts b/tests/corpus/scratch-string-slices.ts new file mode 100644 index 000000000..487883d6e --- /dev/null +++ b/tests/corpus/scratch-string-slices.ts @@ -0,0 +1,116 @@ +// Slices read only by comparisons, numeric parsers, collection probes, +// switch discriminants and lent helper parameters, including the keys a +// map or set keeps after the slice that produced them is gone. +const ascii = "GET /api/users?id=42 200 17.5ms"; +const mixed = "Aé水x😀yz-12.5e1 ok"; +const bounds = [-Infinity, -40, -8, -2.5, -1, -0.5, -0, 0, 0.5, 1, 2, 3, 4.9, 5, 7, 10, 40, Infinity, NaN]; + +function compare(source: string, start: number, end: number): string { + const s = source.slice(start, end); + const t = source.substring(start, end); + const parts = [ + String(s === "GET"), + String(t === "GET"), + String(source.slice(start, end) === source.substring(start, end)), + String(source.slice(start, end) < "M"), + String(source.substring(start, end) >= "a"), + String(s.length), + String(t.length), + String(s.charCodeAt(0)), + String(t.charCodeAt(t.length - 1)), + String(s.indexOf("a")), + String(t.includes("😀")), + String(s.startsWith("A")), + String(t.endsWith("z")), + ]; + return parts.join(","); +} +for (const source of [ascii, mixed]) { + for (const start of bounds) { + const row: string[] = []; + for (const end of bounds) row.push(compare(source, start, end)); + console.log(row.join(" | ")); + } +} + +// Numeric parsing of slices. +const numbers = " 42|-0|0x1F|1e3|.5|Infinity|-Infinity|12px|abc|+7.25| 9 |0b11|1_000|"; +let offset = 0; +while (offset < numbers.length) { + const bar = numbers.indexOf("|", offset); + const field = numbers.slice(offset, bar); + console.log( + JSON.stringify(field), + Number(field), + +numbers.substring(offset, bar), + parseInt(field), + parseInt(numbers.slice(offset, bar), 16), + parseFloat(field), + Object.is(Number(numbers.slice(offset, bar)), -0), + ); + offset = bar + 1; +} +console.log(Number(mixed.slice(9, 16)), parseFloat(mixed.slice(9)), parseInt(mixed.slice(10, 12))); + +// Collection probes and insertions keyed by slices. The stored keys must +// outlive every later slice written to the same site. +const counts = new Map(); +const seen = new Set(); +const words = "alpha beta gamma alpha delta beta alpha epsilon-with-a-much-longer-key-than-the-stack-buffer-holds-alpha zeta é-key é-key x x"; +for (let round = 0; round < 3; round++) { + let start = 0; + while (start < words.length) { + let end = words.indexOf(" ", start); + if (end < 0) end = words.length; + const word = words.slice(start, end); + counts.set(word, (counts.get(word) ?? 0) + 1); + if (!seen.has(word)) seen.add(word); + if (counts.get(words.substring(start, end)) === 2) seen.delete(word); + start = end + 1; + } +} +console.log([...counts.entries()].map(([key, value]) => `${key}=${value}`).join(",")); +console.log([...seen].join(",")); +console.log(counts.has(ascii.slice(0, 3)), counts.has(words.slice(0, 5)), seen.has(words.slice(6, 10))); +const index = new Map(); +for (let i = 0; i + 3 <= ascii.length; i++) index.set(ascii.slice(i, i + 3), i); +console.log(index.size, index.get("api"), index.get(ascii.substring(5, 8)), [...index.keys()].slice(0, 6).join("/")); +const reads = new Map([["api", "first"], ["users", "second"]]); +console.log(reads.get(ascii.slice(5, 8)), reads.get(ascii.slice(9, 14)) ?? "none", reads.get(ascii.slice(0, 1)) ?? "none"); + +// Switch discriminants. +function method(line: string): number { + switch (line.slice(0, line.indexOf(" "))) { + case "GET": + return 1; + case "POST": + return 2; + case "é": + return 3; + default: + return 0; + } +} +console.log(method(ascii), method("POST /x"), method("é y"), method("PATCH /x"), method("nospace")); + +// Lent helper parameters, including recursion through a lent parameter. +function isKeyword(text: string): boolean { + return text === "let" || text === "const" || text === "return"; +} +function depth(text: string, n: number): number { + if (n === 0) return text.length; + return depth(text, n - 1) + (text === "abc" ? 1 : 0); +} +function keep(text: string): string[] { + return [text]; +} +const program = "let x = 1; const y = 2; return x + y; letter"; +let keywords = 0; +const kept: string[][] = []; +for (let i = 0; i < program.length; i++) { + for (let j = i + 2; j <= Math.min(program.length, i + 6); j++) { + if (isKeyword(program.slice(i, j))) keywords++; + } + if (i % 9 === 0) kept.push(keep(program.slice(i, i + 4))); +} +console.log(keywords, depth("abcdef".slice(0, 3), 4), kept.map((k) => k[0]).join("|")); diff --git a/tests/corpus/truthy-object-logical.ts b/tests/corpus/truthy-object-logical.ts new file mode 100644 index 000000000..0b84b760b --- /dev/null +++ b/tests/corpus/truthy-object-logical.ts @@ -0,0 +1,31 @@ +// `&&` and `||` whose left operand is an object, array or function value: +// every object is truthy, so `o && x` evaluates `o` and answers `x`, and +// `o || x` answers `o` without evaluating `x`. +interface Stream { + isTTY: boolean; + columns: number; +} +let reads = 0; +const stream = (): Stream => { + reads++; + return { isTTY: reads % 2 === 1, columns: 80 }; +}; +function tty(s: Stream): boolean { + return s && s.isTTY; +} +console.log(tty({ isTTY: true, columns: 1 }), tty({ isTTY: false, columns: 2 })); +const width = stream() && stream().columns; +console.log(width, reads); +const list = [3, 1, 2]; +const first = list && list.length; +console.log(first); +const fn = (n: number): number => n * 2; +const doubled = fn && fn(21); +console.log(doubled); +let evaluated = false; +const fallback = (): Stream => { + evaluated = true; + return { isTTY: false, columns: 0 }; +}; +const chosen = stream() || fallback(); +console.log(chosen.columns, evaluated, reads); diff --git a/tests/coverage-fixtures/tier-demotions.ts b/tests/coverage-fixtures/tier-demotions.ts new file mode 100644 index 000000000..52f2058f1 --- /dev/null +++ b/tests/coverage-fixtures/tier-demotions.ts @@ -0,0 +1,25 @@ +// The report-only tier assignment of `scriptc coverage --explain-dynamic`. +class Point { + constructor( + public x: number, + public y: number, + ) {} +} + +// Reflection on a class: the dynamic tier would give Point a prototype object. +(Point.prototype as any).norm = function () { + return 0; +}; + +// Already dynamic: a property read on an `any` value. +function readName(text: string): string { + const data: any = JSON.parse(text); + return String(data.name); +} + +// An operation without a static lowering, on a parameter. +function precise(n: number): string { + return n.toPrecision(3); +} + +console.log(readName('{"name":"a"}'), precise(1.5), new Point(1, 2).x); diff --git a/tests/diagnostics/builtin-class-alias-escape.ts b/tests/diagnostics/builtin-class-alias-escape.ts new file mode 100644 index 000000000..b3f9be5d3 --- /dev/null +++ b/tests/diagnostics/builtin-class-alias-escape.ts @@ -0,0 +1,15 @@ +// A builtin class alias owns no runtime value in a TypeScript file: +// construction, instanceof, typeof and .name compile, other uses report +// at the use. A let binding of a class is not an alias. +import { EventEmitter } from "node:events"; + +const Emitter = EventEmitter; +const kinds = [Emitter]; +console.log(kinds.length); + +const M = Map; +const boxed: unknown = M; +console.log(boxed !== undefined); + +let Mutable = EventEmitter; +console.log(typeof Mutable); diff --git a/tests/diagnostics/class-values.ts b/tests/diagnostics/class-values.ts index 02c5360fa..3a53e5185 100644 --- a/tests/diagnostics/class-values.ts +++ b/tests/diagnostics/class-values.ts @@ -18,8 +18,10 @@ class Unrelated { static tag = "unrelated"; } -// Builtin classes as values: their construction is libCall-shaped. +// Builtin classes as values: a const alias resolves at compile time, and +// an array of builtin classes has no element type. const E = Error; +const errorClasses = [E]; // Classes extending builtin bases as values (the error-message completion // rule has no thunk ABI). diff --git a/tests/diagnostics/local-class-type-query-brand.ts b/tests/diagnostics/local-class-type-query-brand.ts new file mode 100644 index 000000000..566304627 --- /dev/null +++ b/tests/diagnostics/local-class-type-query-brand.ts @@ -0,0 +1,14 @@ +// The same speculative element-type query, reaching a function-local class +// with a private brand through the function's returned array. +function make() { + class Box { + #v = 1; + read(): number { + return this.#v; + } + } + return [new Box()]; +} +const xs = make(); +const x = xs[0]; +console.log(x === undefined ? -1 : x.read()); diff --git a/tests/diagnostics/local-class-type-query-js.js b/tests/diagnostics/local-class-type-query-js.js new file mode 100644 index 000000000..2da619133 --- /dev/null +++ b/tests/diagnostics/local-class-type-query-js.js @@ -0,0 +1,26 @@ +// A whole-program analysis that queries types before reachability is known +// (the computed-key write census here, through `holder.p`) names a +// function-local class whose collection is refused. The refusal is reported, +// once, at the class; it used to crash the compiler with an uncaught +// PoisonError. +function main() { + const Project = class _Project { + constructor() { + this.name = "p"; + } + close() { + if (this.missingFilesMap) { + this.missingFilesMap.clear(); + this.missingFilesMap = void 0; + } + } + add(k) { + (this.missingFilesMap || (this.missingFilesMap = new Map())).set(k, 1); + } + }; + const holder = { p: new Project() }; + const key = process.argv.length > 5 ? String(process.argv.length) : "name"; + holder.p[key] = "q"; + console.log(holder.p.name); +} +main(); diff --git a/tests/diagnostics/local-class-type-query.ts b/tests/diagnostics/local-class-type-query.ts new file mode 100644 index 000000000..217653ff2 --- /dev/null +++ b/tests/diagnostics/local-class-type-query.ts @@ -0,0 +1,14 @@ +// A whole-program analysis queries the element type of `items` before any +// body lowers. The function-local class it names is refused: the refusal is +// reported once, at the class (it used to crash the compiler with an uncaught +// PoisonError). +function make() { + class Box { + static count = 0; + v = 1; + } + return new Box(); +} +const items = [make()]; +const first = items[0]; +console.log(first ? first.v : 0); diff --git a/tests/diagnostics/speculative-function-refusal.ts b/tests/diagnostics/speculative-function-refusal.ts new file mode 100644 index 000000000..fc48d9935 --- /dev/null +++ b/tests/diagnostics/speculative-function-refusal.ts @@ -0,0 +1,14 @@ +// Function expressions lowered speculatively — an `.apply` receiver and a +// computed class base — report a refused statement in their body like any +// other function: the statement never disappears from the build. +(function () { + console.log("wrapper"); + console.log(eval("1 + 1")); +}).apply(null, []); + +class Base {} +class Derived extends (function () { + console.log(new Function("return 1")()); + return Base; +})() {} +console.log(typeof Derived); diff --git a/tests/fixtures/node-test/cases/root-after-throws.ts b/tests/fixtures/node-test/cases/root-after-throws.ts new file mode 100644 index 000000000..f7c20b9b0 --- /dev/null +++ b/tests/fixtures/node-test/cases/root-after-throws.ts @@ -0,0 +1,10 @@ +// A throwing module-level after hook: the run reports the entry file as a +// failed result with the hook's location, after the top-level results, +// without counting it, and the exit code stays 0, as in Node. +import { after, test } from "node:test"; + +after(() => { + throw new Error("root after failed"); +}); + +test("t", () => {}); diff --git a/tests/fixtures/node-test/cases/root-before-throws.ts b/tests/fixtures/node-test/cases/root-before-throws.ts new file mode 100644 index 000000000..b12c50060 --- /dev/null +++ b/tests/fixtures/node-test/cases/root-before-throws.ts @@ -0,0 +1,30 @@ +// @exit: 1 +// A throwing module-level before hook: each top-level test fails with its +// error without running its body or beforeEach hooks, and each top-level +// suite fails with it while the tests and suites in it are cancelled +// before they start. Later module-level before hooks still run at their +// before() calls, and the first failure is the one reported. +import { before, beforeEach, describe, it, test } from "node:test"; + +before(() => { + throw new Error("root before failed"); +}); +before(async () => { + await Promise.resolve(); + throw new Error("a later failure"); +}); +beforeEach(() => { + throw new Error("beforeEach never runs"); +}); + +describe("s", () => { + it("x", () => {}); + describe("inner", () => { + it("y", () => {}); + }); + it("z", () => {}); +}); + +test("t", () => { + throw new Error("the body never runs"); +}); diff --git a/tests/fixtures/node-test/cases/root-hooks.ts b/tests/fixtures/node-test/cases/root-hooks.ts new file mode 100644 index 000000000..8159aa7bb --- /dev/null +++ b/tests/fixtures/node-test/cases/root-hooks.ts @@ -0,0 +1,44 @@ +// Module-level before hooks run at their before() calls, with the root's +// context (named ""), and the first top-level test waits for the +// async ones; module-level after hooks run after the last top-level test +// or suite, and module-level beforeEach hooks run before each test, with +// that test's context. +import { after, before, beforeEach, describe, it, test } from "node:test"; +import * as assert from "node:assert"; + +const seen: string[] = []; + +before((c) => { + seen.push(`before: ${c.name}`); +}); +seen.push("after the before() call"); +before(async () => { + seen.push("async before starts"); + await new Promise((resolve) => setTimeout(resolve, 5)); + seen.push("async before ends"); +}); +beforeEach((c) => { + seen.push(`beforeEach: ${c.name}`); +}); +after(() => { + // Throws, and so fails the run visibly, unless every hook ran in order. + assert.deepStrictEqual(seen, [ + "before: ", + "after the before() call", + "async before starts", + "suite body", + "async before ends", + "beforeEach: x", + "beforeEach: t", + "report", + ]); +}); + +describe("s", () => { + seen.push("suite body"); + it("x", () => {}); +}); + +test("t", () => { + seen.push("report"); +}); diff --git a/tests/fixtures/npm-static/node_modules/@types/twinned/index.d.ts b/tests/fixtures/npm-static/node_modules/@types/twinned/index.d.ts new file mode 100644 index 000000000..ef7c0d6c6 --- /dev/null +++ b/tests/fixtures/npm-static/node_modules/@types/twinned/index.d.ts @@ -0,0 +1,5 @@ +export interface Greeting { + text: string; +} + +export default function wait(text: string): string; diff --git a/tests/fixtures/npm-static/node_modules/@types/twinned/package.json b/tests/fixtures/npm-static/node_modules/@types/twinned/package.json new file mode 100644 index 000000000..2bdbbfa6e --- /dev/null +++ b/tests/fixtures/npm-static/node_modules/@types/twinned/package.json @@ -0,0 +1,5 @@ +{ + "name": "@types/twinned", + "version": "1.0.0", + "types": "index.d.ts" +} diff --git a/tests/fixtures/npm-static/node_modules/twinned/index.js b/tests/fixtures/npm-static/node_modules/twinned/index.js new file mode 100644 index 000000000..0aa0a79f0 --- /dev/null +++ b/tests/fixtures/npm-static/node_modules/twinned/index.js @@ -0,0 +1,9 @@ +// A CommonJS package without its own declarations: its types live in the +// separate @types/twinned package, and it depends on the untyped ms. +'use strict'; + +const ms = require('ms'); + +module.exports = function wait(text) { + return 'twinned:' + text + ':' + ms(90000); +}; diff --git a/tests/fixtures/npm-static/node_modules/twinned/package.json b/tests/fixtures/npm-static/node_modules/twinned/package.json new file mode 100644 index 000000000..14cce5ab2 --- /dev/null +++ b/tests/fixtures/npm-static/node_modules/twinned/package.json @@ -0,0 +1,5 @@ +{ + "name": "twinned", + "version": "1.0.0", + "main": "index.js" +} diff --git a/tests/fixtures/npm-static/twinned-cli.ts b/tests/fixtures/npm-static/twinned-cli.ts new file mode 100644 index 000000000..e24383184 --- /dev/null +++ b/tests/fixtures/npm-static/twinned-cli.ts @@ -0,0 +1,5 @@ +// twinned's declarations come from @types/twinned; the package itself and +// its untyped ms dependency ship JavaScript only. +import wait from "twinned"; + +console.log(wait("ready")); diff --git a/tests/fixtures/npm-static/type-only-cli.ts b/tests/fixtures/npm-static/type-only-cli.ts new file mode 100644 index 000000000..9fd332331 --- /dev/null +++ b/tests/fixtures/npm-static/type-only-cli.ts @@ -0,0 +1,7 @@ +// `import type` erases under Node's type stripping: twinned is never +// loaded, so no package is selected or refused, and its declared +// interface types a value the program builds itself. +import type { Greeting } from "twinned"; + +const greeting: Greeting = { text: "hello" }; +console.log(greeting.text); diff --git a/tests/fixtures/npm/cases/chalk5/main.ts b/tests/fixtures/npm/cases/chalk5/main.ts new file mode 100644 index 000000000..6a33a1466 --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/main.ts @@ -0,0 +1,64 @@ +// chalk 5 (the real package, pinned in node_modules): the callable +// builder whose styles are prototype getters on a function chain, every +// color model, nested styles, chalk.level and new Chalk({ level }) at all +// four levels. chalk.test.ts runs the same program under FORCE_COLOR and +// the --color flags. +import chalk, { Chalk, chalkStderr, supportsColor, modifierNames, foregroundColorNames, backgroundColorNames, colorNames } from "chalk"; + +const show = (label: string, value: string): void => { + console.log(label + " " + JSON.stringify(value)); +}; + +console.log("default level " + String(chalk.level) + " stderr " + String(chalkStderr.level)); +console.log("supportsColor " + JSON.stringify(supportsColor)); +console.log(modifierNames.join(",")); +console.log(foregroundColorNames.length + " " + backgroundColorNames.length + " " + colorNames.length); + +show("red", chalk.red("red")); +show("red.bold", chalk.red.bold("bold red")); +show("nested", chalk.blue("a " + chalk.red("b") + " c")); +show("multi-arg", chalk.green("one", "two", 3)); +show("number", chalk.yellow(42 as unknown as string)); +show("empty", chalk.red("")); +show("newline", chalk.magenta.underline("line1\nline2\r\nline3")); +show("bg", chalk.bgCyan.black("bg")); +show("bright", chalk.redBright.bgBlueBright("bright")); +show("modifiers", chalk.italic.inverse.strikethrough.overline("mods")); +show("hidden-visible", chalk.hidden("h") + "|" + chalk.visible("v")); +show("rgb", chalk.rgb(10, 20, 30)("rgb")); +show("hex", chalk.hex("#ff8800").bold("hex")); +show("hex-short", chalk.bgHex("#0f0")("bg hex")); +show("ansi256", chalk.ansi256(201)("ansi256")); +show("bgAnsi256", chalk.bgAnsi256(17).white("bgAnsi256")); +show("callable", chalk("plain", "args")); +show("gray-grey", chalk.gray("x") + chalk.grey("y")); + +for (const level of [0, 1, 2, 3] as const) { + const c = new Chalk({ level }); + show(`L${level} red.bold`, c.red.bold("x")); + show(`L${level} hex`, c.hex("#123456")("x")); + show(`L${level} rgb`, c.rgb(255, 255, 255).bgRgb(0, 0, 0)("x")); + show(`L${level} ansi256`, c.ansi256(100).bgAnsi256(200)("x")); + show(`L${level} visible`, c.visible.red("v")); + show(`L${level} nested`, c.green("g" + c.yellow("y") + "g")); +} + +const mutable = new Chalk({ level: 3 }); +const red = mutable.red; +show("before", red("r")); +mutable.level = 0; +show("after level 0", red("r") + "|" + mutable.blue("b")); +console.log("level via builder " + String(red.level)); +red.level = 2; +console.log("shared level " + String(mutable.level)); +show("restored", mutable.cyan("c")); + +try { + new Chalk({ level: 5 as 0 }); +} catch (error) { + console.log("error " + (error as Error).message); +} + +chalk.level = 1; +show("global after set", chalk.red.bold("x")); +console.log("global level " + String(chalk.level)); diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/license b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/license new file mode 100644 index 000000000..fa7ceba3e --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/license @@ -0,0 +1,9 @@ +MIT License + +Copyright (c) Sindre Sorhus (https://sindresorhus.com) + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/package.json b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/package.json new file mode 100644 index 000000000..c9e0dc52b --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/package.json @@ -0,0 +1,83 @@ +{ + "name": "chalk", + "version": "5.6.2", + "description": "Terminal string styling done right", + "license": "MIT", + "repository": "chalk/chalk", + "funding": "https://github.com/chalk/chalk?sponsor=1", + "type": "module", + "main": "./source/index.js", + "exports": "./source/index.js", + "imports": { + "#ansi-styles": "./source/vendor/ansi-styles/index.js", + "#supports-color": { + "node": "./source/vendor/supports-color/index.js", + "default": "./source/vendor/supports-color/browser.js" + } + }, + "types": "./source/index.d.ts", + "sideEffects": false, + "engines": { + "node": "^12.17.0 || ^14.13 || >=16.0.0" + }, + "scripts": { + "test": "xo && c8 ava && tsd", + "bench": "matcha benchmark.js" + }, + "files": [ + "source", + "!source/index.test-d.ts" + ], + "keywords": [ + "color", + "colour", + "colors", + "terminal", + "console", + "cli", + "string", + "ansi", + "style", + "styles", + "tty", + "formatting", + "rgb", + "256", + "shell", + "xterm", + "log", + "logging", + "command-line", + "text" + ], + "devDependencies": { + "@types/node": "^16.11.10", + "ava": "^3.15.0", + "c8": "^7.10.0", + "color-convert": "^2.0.1", + "execa": "^6.0.0", + "log-update": "^5.0.0", + "matcha": "^0.7.0", + "tsd": "^0.19.0", + "xo": "^0.57.0", + "yoctodelay": "^2.0.0" + }, + "xo": { + "rules": { + "unicorn/prefer-string-slice": "off", + "@typescript-eslint/consistent-type-imports": "off", + "@typescript-eslint/consistent-type-exports": "off", + "@typescript-eslint/consistent-type-definitions": "off", + "unicorn/expiring-todo-comments": "off" + } + }, + "c8": { + "reporter": [ + "text", + "lcov" + ], + "exclude": [ + "source/vendor" + ] + } +} diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/index.d.ts b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/index.d.ts new file mode 100644 index 000000000..8295d9252 --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/index.d.ts @@ -0,0 +1,325 @@ +// TODO: Make it this when TS suports that. +// import {ModifierName, ForegroundColor, BackgroundColor, ColorName} from '#ansi-styles'; +// import {ColorInfo, ColorSupportLevel} from '#supports-color'; +import { + ModifierName, + ForegroundColorName, + BackgroundColorName, + ColorName, +} from './vendor/ansi-styles/index.js'; +import {ColorInfo, ColorSupportLevel} from './vendor/supports-color/index.js'; + +export interface Options { + /** + Specify the color support for Chalk. + + By default, color support is automatically detected based on the environment. + + Levels: + - `0` - All colors disabled. + - `1` - Basic 16 colors support. + - `2` - ANSI 256 colors support. + - `3` - Truecolor 16 million colors support. + */ + readonly level?: ColorSupportLevel; +} + +/** +Return a new Chalk instance. +*/ +export const Chalk: new (options?: Options) => ChalkInstance; // eslint-disable-line @typescript-eslint/naming-convention + +export interface ChalkInstance { + (...text: unknown[]): string; + + /** + The color support for Chalk. + + By default, color support is automatically detected based on the environment. + + Levels: + - `0` - All colors disabled. + - `1` - Basic 16 colors support. + - `2` - ANSI 256 colors support. + - `3` - Truecolor 16 million colors support. + */ + level: ColorSupportLevel; + + /** + Use RGB values to set text color. + + @example + ``` + import chalk from 'chalk'; + + chalk.rgb(222, 173, 237); + ``` + */ + rgb: (red: number, green: number, blue: number) => this; + + /** + Use HEX value to set text color. + + @param color - Hexadecimal value representing the desired color. + + @example + ``` + import chalk from 'chalk'; + + chalk.hex('#DEADED'); + ``` + */ + hex: (color: string) => this; + + /** + Use an [8-bit unsigned number](https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit) to set text color. + + @example + ``` + import chalk from 'chalk'; + + chalk.ansi256(201); + ``` + */ + ansi256: (index: number) => this; + + /** + Use RGB values to set background color. + + @example + ``` + import chalk from 'chalk'; + + chalk.bgRgb(222, 173, 237); + ``` + */ + bgRgb: (red: number, green: number, blue: number) => this; + + /** + Use HEX value to set background color. + + @param color - Hexadecimal value representing the desired color. + + @example + ``` + import chalk from 'chalk'; + + chalk.bgHex('#DEADED'); + ``` + */ + bgHex: (color: string) => this; + + /** + Use a [8-bit unsigned number](https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit) to set background color. + + @example + ``` + import chalk from 'chalk'; + + chalk.bgAnsi256(201); + ``` + */ + bgAnsi256: (index: number) => this; + + /** + Modifier: Reset the current style. + */ + readonly reset: this; + + /** + Modifier: Make the text bold. + */ + readonly bold: this; + + /** + Modifier: Make the text have lower opacity. + */ + readonly dim: this; + + /** + Modifier: Make the text italic. *(Not widely supported)* + */ + readonly italic: this; + + /** + Modifier: Put a horizontal line below the text. *(Not widely supported)* + */ + readonly underline: this; + + /** + Modifier: Put a horizontal line above the text. *(Not widely supported)* + */ + readonly overline: this; + + /** + Modifier: Invert background and foreground colors. + */ + readonly inverse: this; + + /** + Modifier: Print the text but make it invisible. + */ + readonly hidden: this; + + /** + Modifier: Puts a horizontal line through the center of the text. *(Not widely supported)* + */ + readonly strikethrough: this; + + /** + Modifier: Print the text only when Chalk has a color level above zero. + + Can be useful for things that are purely cosmetic. + */ + readonly visible: this; + + readonly black: this; + readonly red: this; + readonly green: this; + readonly yellow: this; + readonly blue: this; + readonly magenta: this; + readonly cyan: this; + readonly white: this; + + /* + Alias for `blackBright`. + */ + readonly gray: this; + + /* + Alias for `blackBright`. + */ + readonly grey: this; + + readonly blackBright: this; + readonly redBright: this; + readonly greenBright: this; + readonly yellowBright: this; + readonly blueBright: this; + readonly magentaBright: this; + readonly cyanBright: this; + readonly whiteBright: this; + + readonly bgBlack: this; + readonly bgRed: this; + readonly bgGreen: this; + readonly bgYellow: this; + readonly bgBlue: this; + readonly bgMagenta: this; + readonly bgCyan: this; + readonly bgWhite: this; + + /* + Alias for `bgBlackBright`. + */ + readonly bgGray: this; + + /* + Alias for `bgBlackBright`. + */ + readonly bgGrey: this; + + readonly bgBlackBright: this; + readonly bgRedBright: this; + readonly bgGreenBright: this; + readonly bgYellowBright: this; + readonly bgBlueBright: this; + readonly bgMagentaBright: this; + readonly bgCyanBright: this; + readonly bgWhiteBright: this; +} + +/** +Main Chalk object that allows to chain styles together. + +Call the last one as a method with a string argument. + +Order doesn't matter, and later styles take precedent in case of a conflict. + +This simply means that `chalk.red.yellow.green` is equivalent to `chalk.green`. +*/ +declare const chalk: ChalkInstance; + +export const supportsColor: ColorInfo; + +export const chalkStderr: typeof chalk; +export const supportsColorStderr: typeof supportsColor; + +export { + ModifierName, ForegroundColorName, BackgroundColorName, ColorName, + modifierNames, foregroundColorNames, backgroundColorNames, colorNames, +// } from '#ansi-styles'; +} from './vendor/ansi-styles/index.js'; + +export { + ColorInfo, + ColorSupport, + ColorSupportLevel, +// } from '#supports-color'; +} from './vendor/supports-color/index.js'; + +// TODO: Remove these aliases in the next major version +/** +@deprecated Use `ModifierName` instead. + +Basic modifier names. +*/ +export type Modifiers = ModifierName; + +/** +@deprecated Use `ForegroundColorName` instead. + +Basic foreground color names. + +[More colors here.](https://github.com/chalk/chalk/blob/main/readme.md#256-and-truecolor-color-support) +*/ +export type ForegroundColor = ForegroundColorName; + +/** +@deprecated Use `BackgroundColorName` instead. + +Basic background color names. + +[More colors here.](https://github.com/chalk/chalk/blob/main/readme.md#256-and-truecolor-color-support) +*/ +export type BackgroundColor = BackgroundColorName; + +/** +@deprecated Use `ColorName` instead. + +Basic color names. The combination of foreground and background color names. + +[More colors here.](https://github.com/chalk/chalk/blob/main/readme.md#256-and-truecolor-color-support) +*/ +export type Color = ColorName; + +/** +@deprecated Use `modifierNames` instead. + +Basic modifier names. +*/ +export const modifiers: readonly Modifiers[]; + +/** +@deprecated Use `foregroundColorNames` instead. + +Basic foreground color names. +*/ +export const foregroundColors: readonly ForegroundColor[]; + +/** +@deprecated Use `backgroundColorNames` instead. + +Basic background color names. +*/ +export const backgroundColors: readonly BackgroundColor[]; + +/** +@deprecated Use `colorNames` instead. + +Basic color names. The combination of foreground and background color names. +*/ +export const colors: readonly Color[]; + +export default chalk; diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/index.js b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/index.js new file mode 100644 index 000000000..8bc993da5 --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/index.js @@ -0,0 +1,225 @@ +import ansiStyles from '#ansi-styles'; +import supportsColor from '#supports-color'; +import { // eslint-disable-line import/order + stringReplaceAll, + stringEncaseCRLFWithFirstIndex, +} from './utilities.js'; + +const {stdout: stdoutColor, stderr: stderrColor} = supportsColor; + +const GENERATOR = Symbol('GENERATOR'); +const STYLER = Symbol('STYLER'); +const IS_EMPTY = Symbol('IS_EMPTY'); + +// `supportsColor.level` → `ansiStyles.color[name]` mapping +const levelMapping = [ + 'ansi', + 'ansi', + 'ansi256', + 'ansi16m', +]; + +const styles = Object.create(null); + +const applyOptions = (object, options = {}) => { + if (options.level && !(Number.isInteger(options.level) && options.level >= 0 && options.level <= 3)) { + throw new Error('The `level` option should be an integer from 0 to 3'); + } + + // Detect level if not set manually + const colorLevel = stdoutColor ? stdoutColor.level : 0; + object.level = options.level === undefined ? colorLevel : options.level; +}; + +export class Chalk { + constructor(options) { + // eslint-disable-next-line no-constructor-return + return chalkFactory(options); + } +} + +const chalkFactory = options => { + const chalk = (...strings) => strings.join(' '); + applyOptions(chalk, options); + + Object.setPrototypeOf(chalk, createChalk.prototype); + + return chalk; +}; + +function createChalk(options) { + return chalkFactory(options); +} + +Object.setPrototypeOf(createChalk.prototype, Function.prototype); + +for (const [styleName, style] of Object.entries(ansiStyles)) { + styles[styleName] = { + get() { + const builder = createBuilder(this, createStyler(style.open, style.close, this[STYLER]), this[IS_EMPTY]); + Object.defineProperty(this, styleName, {value: builder}); + return builder; + }, + }; +} + +styles.visible = { + get() { + const builder = createBuilder(this, this[STYLER], true); + Object.defineProperty(this, 'visible', {value: builder}); + return builder; + }, +}; + +const getModelAnsi = (model, level, type, ...arguments_) => { + if (model === 'rgb') { + if (level === 'ansi16m') { + return ansiStyles[type].ansi16m(...arguments_); + } + + if (level === 'ansi256') { + return ansiStyles[type].ansi256(ansiStyles.rgbToAnsi256(...arguments_)); + } + + return ansiStyles[type].ansi(ansiStyles.rgbToAnsi(...arguments_)); + } + + if (model === 'hex') { + return getModelAnsi('rgb', level, type, ...ansiStyles.hexToRgb(...arguments_)); + } + + return ansiStyles[type][model](...arguments_); +}; + +const usedModels = ['rgb', 'hex', 'ansi256']; + +for (const model of usedModels) { + styles[model] = { + get() { + const {level} = this; + return function (...arguments_) { + const styler = createStyler(getModelAnsi(model, levelMapping[level], 'color', ...arguments_), ansiStyles.color.close, this[STYLER]); + return createBuilder(this, styler, this[IS_EMPTY]); + }; + }, + }; + + const bgModel = 'bg' + model[0].toUpperCase() + model.slice(1); + styles[bgModel] = { + get() { + const {level} = this; + return function (...arguments_) { + const styler = createStyler(getModelAnsi(model, levelMapping[level], 'bgColor', ...arguments_), ansiStyles.bgColor.close, this[STYLER]); + return createBuilder(this, styler, this[IS_EMPTY]); + }; + }, + }; +} + +const proto = Object.defineProperties(() => {}, { + ...styles, + level: { + enumerable: true, + get() { + return this[GENERATOR].level; + }, + set(level) { + this[GENERATOR].level = level; + }, + }, +}); + +const createStyler = (open, close, parent) => { + let openAll; + let closeAll; + if (parent === undefined) { + openAll = open; + closeAll = close; + } else { + openAll = parent.openAll + open; + closeAll = close + parent.closeAll; + } + + return { + open, + close, + openAll, + closeAll, + parent, + }; +}; + +const createBuilder = (self, _styler, _isEmpty) => { + // Single argument is hot path, implicit coercion is faster than anything + // eslint-disable-next-line no-implicit-coercion + const builder = (...arguments_) => applyStyle(builder, (arguments_.length === 1) ? ('' + arguments_[0]) : arguments_.join(' ')); + + // We alter the prototype because we must return a function, but there is + // no way to create a function with a different prototype + Object.setPrototypeOf(builder, proto); + + builder[GENERATOR] = self; + builder[STYLER] = _styler; + builder[IS_EMPTY] = _isEmpty; + + return builder; +}; + +const applyStyle = (self, string) => { + if (self.level <= 0 || !string) { + return self[IS_EMPTY] ? '' : string; + } + + let styler = self[STYLER]; + + if (styler === undefined) { + return string; + } + + const {openAll, closeAll} = styler; + if (string.includes('\u001B')) { + while (styler !== undefined) { + // Replace any instances already present with a re-opening code + // otherwise only the part of the string until said closing code + // will be colored, and the rest will simply be 'plain'. + string = stringReplaceAll(string, styler.close, styler.open); + + styler = styler.parent; + } + } + + // We can move both next actions out of loop, because remaining actions in loop won't have + // any/visible effect on parts we add here. Close the styling before a linebreak and reopen + // after next line to fix a bleed issue on macOS: https://github.com/chalk/chalk/pull/92 + const lfIndex = string.indexOf('\n'); + if (lfIndex !== -1) { + string = stringEncaseCRLFWithFirstIndex(string, closeAll, openAll, lfIndex); + } + + return openAll + string + closeAll; +}; + +Object.defineProperties(createChalk.prototype, styles); + +const chalk = createChalk(); +export const chalkStderr = createChalk({level: stderrColor ? stderrColor.level : 0}); + +export { + modifierNames, + foregroundColorNames, + backgroundColorNames, + colorNames, + + // TODO: Remove these aliases in the next major version + modifierNames as modifiers, + foregroundColorNames as foregroundColors, + backgroundColorNames as backgroundColors, + colorNames as colors, +} from './vendor/ansi-styles/index.js'; + +export { + stdoutColor as supportsColor, + stderrColor as supportsColorStderr, +}; + +export default chalk; diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/utilities.js b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/utilities.js new file mode 100644 index 000000000..4366dee0d --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/utilities.js @@ -0,0 +1,33 @@ +// TODO: When targeting Node.js 16, use `String.prototype.replaceAll`. +export function stringReplaceAll(string, substring, replacer) { + let index = string.indexOf(substring); + if (index === -1) { + return string; + } + + const substringLength = substring.length; + let endIndex = 0; + let returnValue = ''; + do { + returnValue += string.slice(endIndex, index) + substring + replacer; + endIndex = index + substringLength; + index = string.indexOf(substring, endIndex); + } while (index !== -1); + + returnValue += string.slice(endIndex); + return returnValue; +} + +export function stringEncaseCRLFWithFirstIndex(string, prefix, postfix, index) { + let endIndex = 0; + let returnValue = ''; + do { + const gotCR = string[index - 1] === '\r'; + returnValue += string.slice(endIndex, (gotCR ? index - 1 : index)) + prefix + (gotCR ? '\r\n' : '\n') + postfix; + endIndex = index + 1; + index = string.indexOf('\n', endIndex); + } while (index !== -1); + + returnValue += string.slice(endIndex); + return returnValue; +} diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/ansi-styles/index.d.ts b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/ansi-styles/index.d.ts new file mode 100644 index 000000000..58f133abe --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/ansi-styles/index.d.ts @@ -0,0 +1,236 @@ +export interface CSPair { // eslint-disable-line @typescript-eslint/naming-convention + /** + The ANSI terminal control sequence for starting this style. + */ + readonly open: string; + + /** + The ANSI terminal control sequence for ending this style. + */ + readonly close: string; +} + +export interface ColorBase { + /** + The ANSI terminal control sequence for ending this color. + */ + readonly close: string; + + ansi(code: number): string; + + ansi256(code: number): string; + + ansi16m(red: number, green: number, blue: number): string; +} + +export interface Modifier { + /** + Resets the current color chain. + */ + readonly reset: CSPair; + + /** + Make text bold. + */ + readonly bold: CSPair; + + /** + Emitting only a small amount of light. + */ + readonly dim: CSPair; + + /** + Make text italic. (Not widely supported) + */ + readonly italic: CSPair; + + /** + Make text underline. (Not widely supported) + */ + readonly underline: CSPair; + + /** + Make text overline. + + Supported on VTE-based terminals, the GNOME terminal, mintty, and Git Bash. + */ + readonly overline: CSPair; + + /** + Inverse background and foreground colors. + */ + readonly inverse: CSPair; + + /** + Prints the text, but makes it invisible. + */ + readonly hidden: CSPair; + + /** + Puts a horizontal line through the center of the text. (Not widely supported) + */ + readonly strikethrough: CSPair; +} + +export interface ForegroundColor { + readonly black: CSPair; + readonly red: CSPair; + readonly green: CSPair; + readonly yellow: CSPair; + readonly blue: CSPair; + readonly cyan: CSPair; + readonly magenta: CSPair; + readonly white: CSPair; + + /** + Alias for `blackBright`. + */ + readonly gray: CSPair; + + /** + Alias for `blackBright`. + */ + readonly grey: CSPair; + + readonly blackBright: CSPair; + readonly redBright: CSPair; + readonly greenBright: CSPair; + readonly yellowBright: CSPair; + readonly blueBright: CSPair; + readonly cyanBright: CSPair; + readonly magentaBright: CSPair; + readonly whiteBright: CSPair; +} + +export interface BackgroundColor { + readonly bgBlack: CSPair; + readonly bgRed: CSPair; + readonly bgGreen: CSPair; + readonly bgYellow: CSPair; + readonly bgBlue: CSPair; + readonly bgCyan: CSPair; + readonly bgMagenta: CSPair; + readonly bgWhite: CSPair; + + /** + Alias for `bgBlackBright`. + */ + readonly bgGray: CSPair; + + /** + Alias for `bgBlackBright`. + */ + readonly bgGrey: CSPair; + + readonly bgBlackBright: CSPair; + readonly bgRedBright: CSPair; + readonly bgGreenBright: CSPair; + readonly bgYellowBright: CSPair; + readonly bgBlueBright: CSPair; + readonly bgCyanBright: CSPair; + readonly bgMagentaBright: CSPair; + readonly bgWhiteBright: CSPair; +} + +export interface ConvertColor { + /** + Convert from the RGB color space to the ANSI 256 color space. + + @param red - (`0...255`) + @param green - (`0...255`) + @param blue - (`0...255`) + */ + rgbToAnsi256(red: number, green: number, blue: number): number; + + /** + Convert from the RGB HEX color space to the RGB color space. + + @param hex - A hexadecimal string containing RGB data. + */ + hexToRgb(hex: string): [red: number, green: number, blue: number]; + + /** + Convert from the RGB HEX color space to the ANSI 256 color space. + + @param hex - A hexadecimal string containing RGB data. + */ + hexToAnsi256(hex: string): number; + + /** + Convert from the ANSI 256 color space to the ANSI 16 color space. + + @param code - A number representing the ANSI 256 color. + */ + ansi256ToAnsi(code: number): number; + + /** + Convert from the RGB color space to the ANSI 16 color space. + + @param red - (`0...255`) + @param green - (`0...255`) + @param blue - (`0...255`) + */ + rgbToAnsi(red: number, green: number, blue: number): number; + + /** + Convert from the RGB HEX color space to the ANSI 16 color space. + + @param hex - A hexadecimal string containing RGB data. + */ + hexToAnsi(hex: string): number; +} + +/** +Basic modifier names. +*/ +export type ModifierName = keyof Modifier; + +/** +Basic foreground color names. + +[More colors here.](https://github.com/chalk/chalk/blob/main/readme.md#256-and-truecolor-color-support) +*/ +export type ForegroundColorName = keyof ForegroundColor; + +/** +Basic background color names. + +[More colors here.](https://github.com/chalk/chalk/blob/main/readme.md#256-and-truecolor-color-support) +*/ +export type BackgroundColorName = keyof BackgroundColor; + +/** +Basic color names. The combination of foreground and background color names. + +[More colors here.](https://github.com/chalk/chalk/blob/main/readme.md#256-and-truecolor-color-support) +*/ +export type ColorName = ForegroundColorName | BackgroundColorName; + +/** +Basic modifier names. +*/ +export const modifierNames: readonly ModifierName[]; + +/** +Basic foreground color names. +*/ +export const foregroundColorNames: readonly ForegroundColorName[]; + +/** +Basic background color names. +*/ +export const backgroundColorNames: readonly BackgroundColorName[]; + +/* +Basic color names. The combination of foreground and background color names. +*/ +export const colorNames: readonly ColorName[]; + +declare const ansiStyles: { + readonly modifier: Modifier; + readonly color: ColorBase & ForegroundColor; + readonly bgColor: ColorBase & BackgroundColor; + readonly codes: ReadonlyMap; +} & ForegroundColor & BackgroundColor & Modifier & ConvertColor; + +export default ansiStyles; diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/ansi-styles/index.js b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/ansi-styles/index.js new file mode 100644 index 000000000..eaa7bed6c --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/ansi-styles/index.js @@ -0,0 +1,223 @@ +const ANSI_BACKGROUND_OFFSET = 10; + +const wrapAnsi16 = (offset = 0) => code => `\u001B[${code + offset}m`; + +const wrapAnsi256 = (offset = 0) => code => `\u001B[${38 + offset};5;${code}m`; + +const wrapAnsi16m = (offset = 0) => (red, green, blue) => `\u001B[${38 + offset};2;${red};${green};${blue}m`; + +const styles = { + modifier: { + reset: [0, 0], + // 21 isn't widely supported and 22 does the same thing + bold: [1, 22], + dim: [2, 22], + italic: [3, 23], + underline: [4, 24], + overline: [53, 55], + inverse: [7, 27], + hidden: [8, 28], + strikethrough: [9, 29], + }, + color: { + black: [30, 39], + red: [31, 39], + green: [32, 39], + yellow: [33, 39], + blue: [34, 39], + magenta: [35, 39], + cyan: [36, 39], + white: [37, 39], + + // Bright color + blackBright: [90, 39], + gray: [90, 39], // Alias of `blackBright` + grey: [90, 39], // Alias of `blackBright` + redBright: [91, 39], + greenBright: [92, 39], + yellowBright: [93, 39], + blueBright: [94, 39], + magentaBright: [95, 39], + cyanBright: [96, 39], + whiteBright: [97, 39], + }, + bgColor: { + bgBlack: [40, 49], + bgRed: [41, 49], + bgGreen: [42, 49], + bgYellow: [43, 49], + bgBlue: [44, 49], + bgMagenta: [45, 49], + bgCyan: [46, 49], + bgWhite: [47, 49], + + // Bright color + bgBlackBright: [100, 49], + bgGray: [100, 49], // Alias of `bgBlackBright` + bgGrey: [100, 49], // Alias of `bgBlackBright` + bgRedBright: [101, 49], + bgGreenBright: [102, 49], + bgYellowBright: [103, 49], + bgBlueBright: [104, 49], + bgMagentaBright: [105, 49], + bgCyanBright: [106, 49], + bgWhiteBright: [107, 49], + }, +}; + +export const modifierNames = Object.keys(styles.modifier); +export const foregroundColorNames = Object.keys(styles.color); +export const backgroundColorNames = Object.keys(styles.bgColor); +export const colorNames = [...foregroundColorNames, ...backgroundColorNames]; + +function assembleStyles() { + const codes = new Map(); + + for (const [groupName, group] of Object.entries(styles)) { + for (const [styleName, style] of Object.entries(group)) { + styles[styleName] = { + open: `\u001B[${style[0]}m`, + close: `\u001B[${style[1]}m`, + }; + + group[styleName] = styles[styleName]; + + codes.set(style[0], style[1]); + } + + Object.defineProperty(styles, groupName, { + value: group, + enumerable: false, + }); + } + + Object.defineProperty(styles, 'codes', { + value: codes, + enumerable: false, + }); + + styles.color.close = '\u001B[39m'; + styles.bgColor.close = '\u001B[49m'; + + styles.color.ansi = wrapAnsi16(); + styles.color.ansi256 = wrapAnsi256(); + styles.color.ansi16m = wrapAnsi16m(); + styles.bgColor.ansi = wrapAnsi16(ANSI_BACKGROUND_OFFSET); + styles.bgColor.ansi256 = wrapAnsi256(ANSI_BACKGROUND_OFFSET); + styles.bgColor.ansi16m = wrapAnsi16m(ANSI_BACKGROUND_OFFSET); + + // From https://github.com/Qix-/color-convert/blob/3f0e0d4e92e235796ccb17f6e85c72094a651f49/conversions.js + Object.defineProperties(styles, { + rgbToAnsi256: { + value(red, green, blue) { + // We use the extended greyscale palette here, with the exception of + // black and white. normal palette only has 4 greyscale shades. + if (red === green && green === blue) { + if (red < 8) { + return 16; + } + + if (red > 248) { + return 231; + } + + return Math.round(((red - 8) / 247) * 24) + 232; + } + + return 16 + + (36 * Math.round(red / 255 * 5)) + + (6 * Math.round(green / 255 * 5)) + + Math.round(blue / 255 * 5); + }, + enumerable: false, + }, + hexToRgb: { + value(hex) { + const matches = /[a-f\d]{6}|[a-f\d]{3}/i.exec(hex.toString(16)); + if (!matches) { + return [0, 0, 0]; + } + + let [colorString] = matches; + + if (colorString.length === 3) { + colorString = [...colorString].map(character => character + character).join(''); + } + + const integer = Number.parseInt(colorString, 16); + + return [ + /* eslint-disable no-bitwise */ + (integer >> 16) & 0xFF, + (integer >> 8) & 0xFF, + integer & 0xFF, + /* eslint-enable no-bitwise */ + ]; + }, + enumerable: false, + }, + hexToAnsi256: { + value: hex => styles.rgbToAnsi256(...styles.hexToRgb(hex)), + enumerable: false, + }, + ansi256ToAnsi: { + value(code) { + if (code < 8) { + return 30 + code; + } + + if (code < 16) { + return 90 + (code - 8); + } + + let red; + let green; + let blue; + + if (code >= 232) { + red = (((code - 232) * 10) + 8) / 255; + green = red; + blue = red; + } else { + code -= 16; + + const remainder = code % 36; + + red = Math.floor(code / 36) / 5; + green = Math.floor(remainder / 6) / 5; + blue = (remainder % 6) / 5; + } + + const value = Math.max(red, green, blue) * 2; + + if (value === 0) { + return 30; + } + + // eslint-disable-next-line no-bitwise + let result = 30 + ((Math.round(blue) << 2) | (Math.round(green) << 1) | Math.round(red)); + + if (value === 2) { + result += 60; + } + + return result; + }, + enumerable: false, + }, + rgbToAnsi: { + value: (red, green, blue) => styles.ansi256ToAnsi(styles.rgbToAnsi256(red, green, blue)), + enumerable: false, + }, + hexToAnsi: { + value: hex => styles.ansi256ToAnsi(styles.hexToAnsi256(hex)), + enumerable: false, + }, + }); + + return styles; +} + +const ansiStyles = assembleStyles(); + +export default ansiStyles; diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/browser.d.ts b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/browser.d.ts new file mode 100644 index 000000000..0e8a9ccb8 --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/browser.d.ts @@ -0,0 +1 @@ +export {default} from './index.js'; diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/browser.js b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/browser.js new file mode 100644 index 000000000..fbb6ce0fc --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/browser.js @@ -0,0 +1,34 @@ +/* eslint-env browser */ + +const level = (() => { + if (!('navigator' in globalThis)) { + return 0; + } + + if (globalThis.navigator.userAgentData) { + const brand = navigator.userAgentData.brands.find(({brand}) => brand === 'Chromium'); + if (brand && brand.version > 93) { + return 3; + } + } + + if (/\b(Chrome|Chromium)\//.test(globalThis.navigator.userAgent)) { + return 1; + } + + return 0; +})(); + +const colorSupport = level !== 0 && { + level, + hasBasic: true, + has256: level >= 2, + has16m: level >= 3, +}; + +const supportsColor = { + stdout: colorSupport, + stderr: colorSupport, +}; + +export default supportsColor; diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/index.d.ts b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/index.d.ts new file mode 100644 index 000000000..db44a7851 --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/index.d.ts @@ -0,0 +1,55 @@ +import type {WriteStream} from 'node:tty'; + +export type Options = { + /** + Whether `process.argv` should be sniffed for `--color` and `--no-color` flags. + + @default true + */ + readonly sniffFlags?: boolean; +}; + +/** +Levels: +- `0` - All colors disabled. +- `1` - Basic 16 colors support. +- `2` - ANSI 256 colors support. +- `3` - Truecolor 16 million colors support. +*/ +export type ColorSupportLevel = 0 | 1 | 2 | 3; + +/** +Detect whether the terminal supports color. +*/ +export type ColorSupport = { + /** + The color level. + */ + level: ColorSupportLevel; + + /** + Whether basic 16 colors are supported. + */ + hasBasic: boolean; + + /** + Whether ANSI 256 colors are supported. + */ + has256: boolean; + + /** + Whether Truecolor 16 million colors are supported. + */ + has16m: boolean; +}; + +export type ColorInfo = ColorSupport | false; + +export function createSupportsColor(stream?: WriteStream, options?: Options): ColorInfo; + +declare const supportsColor: { + stdout: ColorInfo; + stderr: ColorInfo; +}; + +export default supportsColor; diff --git a/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/index.js b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/index.js new file mode 100644 index 000000000..265d7f858 --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/node_modules/chalk/source/vendor/supports-color/index.js @@ -0,0 +1,190 @@ +import process from 'node:process'; +import os from 'node:os'; +import tty from 'node:tty'; + +// From: https://github.com/sindresorhus/has-flag/blob/main/index.js +/// function hasFlag(flag, argv = globalThis.Deno?.args ?? process.argv) { +function hasFlag(flag, argv = globalThis.Deno ? globalThis.Deno.args : process.argv) { + const prefix = flag.startsWith('-') ? '' : (flag.length === 1 ? '-' : '--'); + const position = argv.indexOf(prefix + flag); + const terminatorPosition = argv.indexOf('--'); + return position !== -1 && (terminatorPosition === -1 || position < terminatorPosition); +} + +const {env} = process; + +let flagForceColor; +if ( + hasFlag('no-color') + || hasFlag('no-colors') + || hasFlag('color=false') + || hasFlag('color=never') +) { + flagForceColor = 0; +} else if ( + hasFlag('color') + || hasFlag('colors') + || hasFlag('color=true') + || hasFlag('color=always') +) { + flagForceColor = 1; +} + +function envForceColor() { + if ('FORCE_COLOR' in env) { + if (env.FORCE_COLOR === 'true') { + return 1; + } + + if (env.FORCE_COLOR === 'false') { + return 0; + } + + return env.FORCE_COLOR.length === 0 ? 1 : Math.min(Number.parseInt(env.FORCE_COLOR, 10), 3); + } +} + +function translateLevel(level) { + if (level === 0) { + return false; + } + + return { + level, + hasBasic: true, + has256: level >= 2, + has16m: level >= 3, + }; +} + +function _supportsColor(haveStream, {streamIsTTY, sniffFlags = true} = {}) { + const noFlagForceColor = envForceColor(); + if (noFlagForceColor !== undefined) { + flagForceColor = noFlagForceColor; + } + + const forceColor = sniffFlags ? flagForceColor : noFlagForceColor; + + if (forceColor === 0) { + return 0; + } + + if (sniffFlags) { + if (hasFlag('color=16m') + || hasFlag('color=full') + || hasFlag('color=truecolor')) { + return 3; + } + + if (hasFlag('color=256')) { + return 2; + } + } + + // Check for Azure DevOps pipelines. + // Has to be above the `!streamIsTTY` check. + if ('TF_BUILD' in env && 'AGENT_NAME' in env) { + return 1; + } + + if (haveStream && !streamIsTTY && forceColor === undefined) { + return 0; + } + + const min = forceColor || 0; + + if (env.TERM === 'dumb') { + return min; + } + + if (process.platform === 'win32') { + // Windows 10 build 10586 is the first Windows release that supports 256 colors. + // Windows 10 build 14931 is the first release that supports 16m/TrueColor. + const osRelease = os.release().split('.'); + if ( + Number(osRelease[0]) >= 10 + && Number(osRelease[2]) >= 10_586 + ) { + return Number(osRelease[2]) >= 14_931 ? 3 : 2; + } + + return 1; + } + + if ('CI' in env) { + if (['GITHUB_ACTIONS', 'GITEA_ACTIONS', 'CIRCLECI'].some(key => key in env)) { + return 3; + } + + if (['TRAVIS', 'APPVEYOR', 'GITLAB_CI', 'BUILDKITE', 'DRONE'].some(sign => sign in env) || env.CI_NAME === 'codeship') { + return 1; + } + + return min; + } + + if ('TEAMCITY_VERSION' in env) { + return /^(9\.(0*[1-9]\d*)\.|\d{2,}\.)/.test(env.TEAMCITY_VERSION) ? 1 : 0; + } + + if (env.COLORTERM === 'truecolor') { + return 3; + } + + if (env.TERM === 'xterm-kitty') { + return 3; + } + + if (env.TERM === 'xterm-ghostty') { + return 3; + } + + if (env.TERM === 'wezterm') { + return 3; + } + + if ('TERM_PROGRAM' in env) { + const version = Number.parseInt((env.TERM_PROGRAM_VERSION || '').split('.')[0], 10); + + switch (env.TERM_PROGRAM) { + case 'iTerm.app': { + return version >= 3 ? 3 : 2; + } + + case 'Apple_Terminal': { + return 2; + } + // No default + } + } + + if (/-256(color)?$/i.test(env.TERM)) { + return 2; + } + + if (/^screen|^xterm|^vt100|^vt220|^rxvt|color|ansi|cygwin|linux/i.test(env.TERM)) { + return 1; + } + + if ('COLORTERM' in env) { + return 1; + } + + return min; +} + +export function createSupportsColor(stream, options = {}) { + const level = _supportsColor(stream, { + streamIsTTY: stream && stream.isTTY, + ...options, + }); + + return translateLevel(level); +} + +const supportsColor = { + stdout: createSupportsColor({isTTY: tty.isatty(1)}), + stderr: createSupportsColor({isTTY: tty.isatty(2)}), +}; + +export default supportsColor; diff --git a/tests/fixtures/npm/cases/chalk5/package.json b/tests/fixtures/npm/cases/chalk5/package.json new file mode 100644 index 000000000..3804daa8d --- /dev/null +++ b/tests/fixtures/npm/cases/chalk5/package.json @@ -0,0 +1 @@ +{ "name": "chalk5-case", "private": true, "type": "module" } diff --git a/tests/harness/__snapshots__/builtin-class-alias-escape.ts.txt b/tests/harness/__snapshots__/builtin-class-alias-escape.ts.txt new file mode 100644 index 000000000..9d1d981e2 --- /dev/null +++ b/tests/harness/__snapshots__/builtin-class-alias-escape.ts.txt @@ -0,0 +1,22 @@ +builtin-class-alias-escape.ts:7:16 - error SC1090: a builtin class alias as an escaping value (use 'Emitter' with new, instanceof, typeof or .name) is not supported yet + + 6 | const Emitter = EventEmitter; + 7 | const kinds = [Emitter]; + | ^~~~~~~ + 8 | console.log(kinds.length); + +builtin-class-alias-escape.ts:11:24 - error SC1090: a builtin class alias as an escaping value (use 'M' with new, instanceof, typeof or .name) is not supported yet + + 10 | const M = Map; + 11 | const boxed: unknown = M; + | ^ + 12 | console.log(boxed !== undefined); + +builtin-class-alias-escape.ts:14:15 - error SC2020: 'events.EventEmitter' is part of the standard library types but has no scriptc lowering yet + + 13 | + 14 | let Mutable = EventEmitter; + | ^~~~~~~~~~~~ + 15 | console.log(typeof Mutable); + + hint: the type checker sees the full standard library, but only the supported surface compiles (https://scriptc.dev/limitations) \ No newline at end of file diff --git a/tests/harness/__snapshots__/class-values.ts.txt b/tests/harness/__snapshots__/class-values.ts.txt index 9d5a43c1f..953d1c2d1 100644 --- a/tests/harness/__snapshots__/class-values.ts.txt +++ b/tests/harness/__snapshots__/class-values.ts.txt @@ -1,52 +1,52 @@ -class-values.ts:22:7 - error SC2020: 'ErrorConstructor' is part of the standard library types but has no scriptc lowering yet +class-values.ts:24:7 - error SC2009: values of type 'ErrorConstructor[]' cannot be compiled: the array shape is supported, but its element type 'ErrorConstructor' does not compile - 21 | // Builtin classes as values: their construction is libCall-shaped. - 22 | const E = Error; - | ^ - 23 | + 23 | const E = Error; + 24 | const errorClasses = [E]; + | ^~~~~~~~~~~~ + 25 | - hint: the type checker sees the full standard library, but only the supported surface compiles (https://scriptc.dev/limitations) + hint: the outer shape is supported — the named component is the blocker; restate it inside the compilable set (number, string, boolean, arrays, Maps, Sets, RegExp, functions, classes, records, unions of those, and 'unknown') -class-values.ts:22:11 - error SC2020: 'Error' is part of the standard library types but has no scriptc lowering yet +class-values.ts:24:22 - error SC2009: values of type 'ErrorConstructor[]' cannot be compiled: the array shape is supported, but its element type 'ErrorConstructor' does not compile - 21 | // Builtin classes as values: their construction is libCall-shaped. - 22 | const E = Error; - | ^~~~~ - 23 | + 23 | const E = Error; + 24 | const errorClasses = [E]; + | ^~~ + 25 | - hint: the type checker sees the full standard library, but only the supported surface compiles (https://scriptc.dev/limitations) + hint: the outer shape is supported — the named component is the blocker; restate it inside the compilable set (number, string, boolean, arrays, Maps, Sets, RegExp, functions, classes, records, unions of those, and 'unknown') -class-values.ts:44:1 - error SC1090: assigning the static 'tag' through a class value (JS creates an OWN property on the runtime class — assign through the declaring class's name, which subclasses here) is not supported yet +class-values.ts:46:1 - error SC1090: assigning the static 'tag' through a class value (JS creates an OWN property on the runtime class — assign through the declaring class's name, which subclasses here) is not supported yet - 43 | let someClass: typeof Base = Base; - 44 | someClass.tag = "write"; + 45 | let someClass: typeof Base = Base; + 46 | someClass.tag = "write"; | ^~~~~~~~~~~~~ - 45 | + 47 | -class-values.ts:48:13 - error SC1090: reading the static member 'tag' through a class value (a subclass of 'Base' redeclares it, so the runtime class decides which declaration answers) is not supported yet +class-values.ts:50:13 - error SC1090: reading the static member 'tag' through a class value (a subclass of 'Base' redeclares it, so the runtime class decides which declaration answers) is not supported yet - 47 | // runtime class decides which declaration answers. - 48 | console.log(someClass.tag); + 49 | // runtime class decides which declaration answers. + 50 | console.log(someClass.tag); | ^~~~~~~~~~~~~ - 49 | + 51 | -class-values.ts:52:10 - error SC1090: function-local classes with statics, decorators, or generic members are not supported yet +class-values.ts:54:10 - error SC1090: function-local classes with statics, decorators, or generic members are not supported yet - 51 | function make(): unknown { - 52 | return class { static value = 1; }; + 53 | function make(): unknown { + 54 | return class { static value = 1; }; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ - 53 | } + 55 | } -class-values.ts:59:18 - error SC1090: class expressions with static initializers or static blocks outside a whole-initializer position (their declaration-time code must run exactly where the expression evaluates — bind the class in its own top-level `const C = class …` statement) is not supported yet +class-values.ts:61:18 - error SC1090: class expressions with static initializers or static blocks outside a whole-initializer position (their declaration-time code must run exactly where the expression evaluates — bind the class in its own top-level `const C = class …` statement) is not supported yet - 58 | // evaluates. - 59 | const pair = [1, class WithStatic { static x = 1; }] as const; + 60 | // evaluates. + 61 | const pair = [1, class WithStatic { static x = 1; }] as const; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - 60 | + 62 | -class-values.ts:79:56 - error SC1090: receiver-dependent static methods through a polymorphic class value is not supported yet +class-values.ts:81:56 - error SC1090: receiver-dependent static methods through a polymorphic class value is not supported yet - 78 | class OtherThis extends UsesThis {} - 79 | function self(value: typeof UsesThis): string { return value.self(); } + 80 | class OtherThis extends UsesThis {} + 81 | function self(value: typeof UsesThis): string { return value.self(); } | ^~~~~~~~~~ - 80 | console.log(self(OtherThis)); \ No newline at end of file + 82 | console.log(self(OtherThis)); \ No newline at end of file diff --git a/tests/harness/__snapshots__/coverage-divergences.txt b/tests/harness/__snapshots__/coverage-divergences.txt index 67c9b57e3..2f811f892 100644 --- a/tests/harness/__snapshots__/coverage-divergences.txt +++ b/tests/harness/__snapshots__/coverage-divergences.txt @@ -2,6 +2,7 @@ scriptc coverage tests/coverage-fixtures/divergences.ts statements analyzed 18 compile statically 18 (100%) + in dynamic tier 1 (operate on values typed only at runtime; --explain-dynamic lists them) fully static — all analyzed statements compile to native code. diff --git a/tests/harness/__snapshots__/coverage-dynamic-mix.txt b/tests/harness/__snapshots__/coverage-dynamic-mix.txt index a3e714bde..2ad84f0af 100644 --- a/tests/harness/__snapshots__/coverage-dynamic-mix.txt +++ b/tests/harness/__snapshots__/coverage-dynamic-mix.txt @@ -2,6 +2,7 @@ scriptc coverage tests/coverage-fixtures/dynamic-mix.ts statements analyzed 8 compile statically 7 (87%) + in dynamic tier 2 (operate on values typed only at runtime; --explain-dynamic lists them) blockers: ×1 SC2012 '.toPrecision()' on numbers cannot compile statically diff --git a/tests/harness/__snapshots__/coverage-explain-dynamic.txt b/tests/harness/__snapshots__/coverage-explain-dynamic.txt new file mode 100644 index 000000000..1c86a808c --- /dev/null +++ b/tests/harness/__snapshots__/coverage-explain-dynamic.txt @@ -0,0 +1,16 @@ + dynamic tier (tier assignment, report-only; --tier=auto) + 4 functions already operate on dynamic values (1 statement) + readName tests/coverage-fixtures/tier-demotions.ts:15:1: %dyn.error.stringConstructor at tests/coverage-fixtures/tier-demotions.ts:17:10 + %dyn.error.stringConstructor.0 tests/coverage-fixtures/tier-demotions.ts:17:10: dynCall at tests/coverage-fixtures/tier-demotions.ts:17:10 + %dyn.class.property.0 tests/coverage-fixtures/tier-demotions.ts:17:17: %dyn.class.readPropertiesOrEmpty at tests/coverage-fixtures/tier-demotions.ts:17:17 + %method.value:%%Error.toString tests/coverage-fixtures/tier-demotions.ts:17:10: dyn.this at tests/coverage-fixtures/tier-demotions.ts:17:10 + + 2 demotion requests (refusals and fences the dynamic tier would replace) + class Point tests/coverage-fixtures/tier-demotions.ts:2:1 (1 request) + <- SC1090 reflection at tests/coverage-fixtures/tier-demotions.ts:10:1: assignment to non-variables are not supported yet + -> function turns dynamic + function tests/coverage-fixtures/tier-demotions.ts:1:1 (1 site inside) + function precise tests/coverage-fixtures/tier-demotions.ts:21:1 (1 site inside) + binding n in precise tests/coverage-fixtures/tier-demotions.ts:21:18 (1 request) + <- SC2012 operation at tests/coverage-fixtures/tier-demotions.ts:22:10: '.toPrecision()' on numbers cannot compile statically + -> function precise turns dynamic \ No newline at end of file diff --git a/tests/harness/__snapshots__/coverage-js-gap.txt b/tests/harness/__snapshots__/coverage-js-gap.txt index 3150216b6..3b6fa678c 100644 --- a/tests/harness/__snapshots__/coverage-js-gap.txt +++ b/tests/harness/__snapshots__/coverage-js-gap.txt @@ -2,5 +2,6 @@ scriptc coverage tests/coverage-fixtures/js-gap/gap.js statements analyzed 5 compile statically 5 (100%) + in dynamic tier 3 (operate on values typed only at runtime; --explain-dynamic lists them) fully static — all analyzed statements compile to native code. \ No newline at end of file diff --git a/tests/harness/__snapshots__/coverage-mixed.txt b/tests/harness/__snapshots__/coverage-mixed.txt index 88b61c39b..c3b348f9c 100644 --- a/tests/harness/__snapshots__/coverage-mixed.txt +++ b/tests/harness/__snapshots__/coverage-mixed.txt @@ -2,6 +2,7 @@ scriptc coverage tests/coverage-fixtures/mixed.ts statements analyzed 13 compile statically 12 (92%) + in dynamic tier 1 (operate on values typed only at runtime; --explain-dynamic lists them) blockers: ×1 SC1031 object destructuring of non-record values (the source is array-typed) diff --git a/tests/harness/__snapshots__/coverage-npm-packages.txt b/tests/harness/__snapshots__/coverage-npm-packages.txt index 0ba05286a..475b10e45 100644 --- a/tests/harness/__snapshots__/coverage-npm-packages.txt +++ b/tests/harness/__snapshots__/coverage-npm-packages.txt @@ -2,6 +2,7 @@ scriptc coverage tests/corpus/native-npm-imports/main.ts statements analyzed 14 compile statically 14 (100%) + in dynamic tier 3 (operate on values typed only at runtime; --explain-dynamic lists them) npm package compilation: mathkit static diff --git a/tests/harness/__snapshots__/coverage-tier-demotions.txt b/tests/harness/__snapshots__/coverage-tier-demotions.txt new file mode 100644 index 000000000..7b52f9d63 --- /dev/null +++ b/tests/harness/__snapshots__/coverage-tier-demotions.txt @@ -0,0 +1,13 @@ +scriptc coverage tests/coverage-fixtures/tier-demotions.ts + + statements analyzed 5 + compile statically 3 (60%) + in dynamic tier 1 (operate on values typed only at runtime; --explain-dynamic lists them) + + blockers: + ×1 SC1090 assignment to non-variables + at tests/coverage-fixtures/tier-demotions.ts:10:1 + hint: this form is valid TypeScript but does not compile yet — rewrite it with a supported equivalent of the construct the message names (https://scriptc.dev/limitations lists the boundary) + ×1 SC2012 '.toPrecision()' on numbers cannot compile statically + at tests/coverage-fixtures/tier-demotions.ts:22:10 + hint: use a supported API or rewrite this operation for native compilation \ No newline at end of file diff --git a/tests/harness/__snapshots__/coverage-width-copies.txt b/tests/harness/__snapshots__/coverage-width-copies.txt index dd99d4bb5..46e8e949d 100644 --- a/tests/harness/__snapshots__/coverage-width-copies.txt +++ b/tests/harness/__snapshots__/coverage-width-copies.txt @@ -2,6 +2,7 @@ scriptc coverage tests/coverage-fixtures/width-copies.ts statements analyzed 14 compile statically 14 (100%) + in dynamic tier 1 (operate on values typed only at runtime; --explain-dynamic lists them) fully static — all analyzed statements compile to native code. diff --git a/tests/harness/__snapshots__/local-class-type-query-brand.ts.txt b/tests/harness/__snapshots__/local-class-type-query-brand.ts.txt new file mode 100644 index 000000000..6db3418a8 --- /dev/null +++ b/tests/harness/__snapshots__/local-class-type-query-brand.ts.txt @@ -0,0 +1,6 @@ +local-class-type-query-brand.ts:4:3 - error SC1090: function-local classes with private brands are not supported yet + + 3 | function make() { + 4 | class Box { + | ^~~~~~~~~~~ + 5 | #v = 1; \ No newline at end of file diff --git a/tests/harness/__snapshots__/local-class-type-query-js.js.txt b/tests/harness/__snapshots__/local-class-type-query-js.js.txt new file mode 100644 index 000000000..60422a5dc --- /dev/null +++ b/tests/harness/__snapshots__/local-class-type-query-js.js.txt @@ -0,0 +1,6 @@ +local-class-type-query-js.js:14:9 - error SC1090: fields assigned outside the constructor's top level ('this.missingFilesMap' would be undefined until the first assignment runs — assign it unconditionally at the top of the constructor) is not supported yet + + 13 | this.missingFilesMap.clear(); + 14 | this.missingFilesMap = void 0; + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + 15 | } \ No newline at end of file diff --git a/tests/harness/__snapshots__/local-class-type-query.ts.txt b/tests/harness/__snapshots__/local-class-type-query.ts.txt new file mode 100644 index 000000000..87489fcd5 --- /dev/null +++ b/tests/harness/__snapshots__/local-class-type-query.ts.txt @@ -0,0 +1,6 @@ +local-class-type-query.ts:6:3 - error SC1090: function-local classes with statics, decorators, or generic members are not supported yet + + 5 | function make() { + 6 | class Box { + | ^~~~~~~~~~~ + 7 | static count = 0; \ No newline at end of file diff --git a/tests/harness/__snapshots__/speculative-function-refusal.ts.txt b/tests/harness/__snapshots__/speculative-function-refusal.ts.txt new file mode 100644 index 000000000..c8b0cd28c --- /dev/null +++ b/tests/harness/__snapshots__/speculative-function-refusal.ts.txt @@ -0,0 +1,17 @@ +speculative-function-refusal.ts:6:15 - error SC2020: 'eval' is part of the standard library types but has no scriptc lowering yet + + 5 | console.log("wrapper"); + 6 | console.log(eval("1 + 1")); + | ^~~~ + 7 | }).apply(null, []); + + hint: runtime code evaluation cannot be compiled ahead of time + +speculative-function-refusal.ts:11:15 - error SC2020: 'new Function' is part of the standard library types but has no scriptc lowering yet + + 10 | class Derived extends (function () { + 11 | console.log(new Function("return 1")()); + | ^~~~~~~~~~~~~~~~~~~~~~~~ + 12 | return Base; + + hint: runtime code generation cannot be compiled ahead of time (the eval stance) — write the function \ No newline at end of file diff --git a/tests/harness/__snapshots__/zlib-class-alias-escape.ts.txt b/tests/harness/__snapshots__/zlib-class-alias-escape.ts.txt index ea9d76089..703038025 100644 --- a/tests/harness/__snapshots__/zlib-class-alias-escape.ts.txt +++ b/tests/harness/__snapshots__/zlib-class-alias-escape.ts.txt @@ -1,4 +1,4 @@ -zlib-class-alias-escape.ts:7:18 - error SC1090: a zlib class alias as an escaping value (use 'Gz' with new, instanceof, typeof or .name) is not supported yet +zlib-class-alias-escape.ts:7:18 - error SC1090: a builtin class alias as an escaping value (use 'Gz' with new, instanceof, typeof or .name) is not supported yet 6 | const Gz = zlib.Gzip; 7 | const classes = [Gz]; diff --git a/tests/harness/chalk.test.ts b/tests/harness/chalk.test.ts new file mode 100644 index 000000000..4d99bf3ee --- /dev/null +++ b/tests/harness/chalk.test.ts @@ -0,0 +1,97 @@ +/* chalk 5 compiled from its published JavaScript (tests/fixtures/npm/cases/ + * chalk5): the program runs under Node and as a native binary with each + * color-detection input supports-color reads — FORCE_COLOR values and the + * --color/--no-color flags — and stdout, stderr and the exit status must + * match byte for byte. npm.test.ts runs the same case once in the + * harness's own environment. */ +import { execFile } from "node:child_process"; +import { mkdirSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { promisify } from "node:util"; +import { beforeAll, describe, expect, test } from "vitest"; +import { compile } from "@scriptc/compiler"; + +const execFileAsync = promisify(execFile); +const repoRoot = join(import.meta.dirname, "../.."); +const entry = join(repoRoot, "tests/fixtures/npm/cases/chalk5/main.ts"); +const outDir = join(repoRoot, "node_modules/.cache/scriptc-tests/chalk5"); +const sanitize = process.env["SCRIPTC_SAN"] === "1"; + +interface RunResult { + stdout: string; + stderr: string; + exitCode: number; +} + +async function run(cmd: string, args: string[], env: NodeJS.ProcessEnv): Promise { + try { + const pending = execFileAsync(cmd, args, { cwd: dirname(entry), env, encoding: "utf8" }); + pending.child.stdin?.end(); + const { stdout, stderr } = await pending; + return { stdout, stderr, exitCode: 0 }; + } catch (err) { + const e = err as { code?: unknown; stdout?: string; stderr?: string }; + if (typeof e.stdout !== "string" || typeof e.stderr !== "string") throw err; + return { + stdout: e.stdout, + stderr: e.stderr, + exitCode: typeof e.code === "number" ? e.code : -1, + }; + } +} + +/** The harness environment without the color inputs, plus `force`. */ +function colorEnv(force: string | undefined): NodeJS.ProcessEnv { + const env = { ...process.env }; + delete env["FORCE_COLOR"]; + delete env["NO_COLOR"]; + delete env["NODE_DISABLE_COLORS"]; + if (force !== undefined) env["FORCE_COLOR"] = force; + return env; +} + +const cases: { name: string; force: string | undefined; argv: string[] }[] = [ + { name: "FORCE_COLOR unset", force: undefined, argv: [] }, + ...["0", "1", "2", "3", "true", "false", ""].map((force) => ({ + name: `FORCE_COLOR=${JSON.stringify(force)}`, + force, + argv: [], + })), + ...[ + ["--no-color"], + ["--color"], + ["--color=256"], + ["--color=16m"], + ["--color", "--", "--no-color"], + ].map((argv) => ({ name: argv.join(" "), force: undefined, argv })), + { name: "FORCE_COLOR=3 --no-color", force: "3", argv: ["--no-color"] }, +]; + +let binary = ""; + +beforeAll(async () => { + mkdirSync(outDir, { recursive: true }); + const result = await compile(entry, { outPath: join(outDir, "program"), outDir, sanitize }); + if (!result.ok) + throw new Error(result.diagnostics.map((d) => `${d.code}: ${d.message}`).join("\n")); + binary = result.binaryPath; +}, 300_000); + +describe("chalk 5 color detection matches Node", () => { + for (const c of cases) { + test( + c.name, + async () => { + const env = colorEnv(c.force); + const [node, native] = await Promise.all([ + run(process.execPath, [entry, ...c.argv], env), + run(binary, c.argv, env), + ]); + expect(native.stdout).toBe(node.stdout); + expect(native.stderr).toBe(node.stderr); + expect(native.exitCode).toBe(node.exitCode); + }, + 60_000, + ); + } +}); diff --git a/tests/harness/corpus-costs.json b/tests/harness/corpus-costs.json index 9f3c007f3..18147374d 100644 --- a/tests/harness/corpus-costs.json +++ b/tests/harness/corpus-costs.json @@ -1341,6 +1341,8 @@ "4451-int32-field-storage.ts": 3457, "4452-int32-field-dynamic-writes.ts": 3861, "4453-int32-closed-locals.ts": 2280, + "4454-int32-field-reflection-reads.ts": 3951, + "4455-live-view-reads-without-stores.ts": 3598, "4471-unassigned-field-reads.ts": 3309, "4476-string-ascii-header-fact.ts": 2846, "4477-buffer-write-ascii-runs.ts": 2125, @@ -1509,6 +1511,8 @@ "5020-lifetime-region-large.ts": 3840, "5021-lifetime-region-opaque-holders.ts": 2702, "5022-lifetime-region-workers.ts": 3127, + "5023-lifetime-region-await.ts": 3656, + "5024-lifetime-region-await-forms.ts": 3342, "503-array-functions.ts": 2610, "504-array-rc-stress.ts": 2247, "5040-promise-all-race-turns.ts": 3415, @@ -1690,6 +1694,16 @@ "5248-event-target-subclass-warnings.ts": 2743, "5249-event-target-subclass-unconstructed.ts": 1632, "525-map-rc-stress.ts": 3853, + "5250-cipher-aes-gcm.ts": 5771, + "5251-cipher-aes-cbc.ts": 3823, + "5252-cipher-aes-ctr.ts": 2580, + "5253-cipher-chacha20-poly1305.ts": 3418, + "5254-cipher-errors.js": 4743, + "5255-cipher-encodings.cjs": 2250, + "5256-cipher-deprecation.cjs": 3088, + "5257-cipher-vectors.ts": 3572, + "5258-cipher-handles.js": 2980, + "5259-cipher-chunked.ts": 2586, "526-set-basics.ts": 1919, "5260-record-unknown-identity.ts": 2937, "5261-optional-field-identity.ts": 2860, @@ -1729,6 +1743,14 @@ "5319-map-iteration-values.ts": 2689, "532-record-arrays-rc-stress.ts": 2437, "5320-map-iterate-across-compaction.ts": 3526, + "5321-integer-closed-arithmetic-loops.ts": 3349, + "5321-json-parse-cycles.ts": 3308, + "5322-integer-variable-step-loops.ts": 2801, + "5322-json-stringify-parsed-trees.ts": 3139, + "5323-dyn-numeric-index-reads.ts": 2266, + "5323-int32-nonnegative-slot-facts.ts": 2381, + "5324-int32-initializer-locals.ts": 3123, + "5324-structured-clone-documents.ts": 3223, "5325-record-boundary-returns.ts": 1763, "5326-record-boundary-params.ts": 2634, "5327-record-boundary-identity.ts": 2323, @@ -1737,14 +1759,36 @@ "533-array-element-cycles.ts": 2932, "5330-record-boundary-fields.ts": 1977, "5331-record-boundary-methods.ts": 3130, + "5332-int32-narrowed-slot-facts.ts": 2932, + "5333-integer-guarded-while-counters.ts": 2773, "5333-record-boundary-nullable-argument.ts": 3012, "5334-fetch-mentions-without-calls.ts": 2532, + "5335-stream-operators-values.ts": 3448, "5335-top-level-await-pending-exit-code.ts": 1919, + "5336-stream-operators-promise-timing.ts": 3509, "5336-top-level-await-pending-exit-listener.ts": 3734, "5337-dc-subscriber-throw-next-tick.cjs": 2604, + "5337-stream-operators-concurrency.ts": 2478, "5338-js-tdz-function-element.js": 2562, + "5338-stream-operators-checks.cjs": 3991, "5339-array-buffer-constructor.js": 2502, + "5339-stream-for-await-hops.ts": 2919, "534-record-width-subtyping.ts": 2615, + "5340-readable-from-options.ts": 1980, + "5341-compound-any-operand.ts": 3314, + "5341-generator-callback-params.ts": 3955, + "5341-integer-recursion-boundaries.ts": 2534, + "5341-readable-from-dynamic.cjs": 3057, + "5342-compound-any-operand.cjs": 3386, + "5342-generator-scalar-channels.ts": 3184, + "5342-integer-recursion-stack-and-mutual.ts": 3097, + "5342-stream-compose.ts": 2940, + "5343-any-flow-compound.ts": 2063, + "5343-integer-recursion-array-index.ts": 2905, + "5343-iterator-result-records.ts": 3426, + "5343-stream-compose-edges.ts": 3078, + "5344-json-parse-undeclared-keys.ts": 4193, + "5344-stream-pipeline-timing.ts": 3177, "5345-object-values-for-of-unknown.ts": 3008, "5346-object-values-length-dynamic.ts": 3055, "5347-object-values-loop-control.ts": 3030, @@ -1753,6 +1797,12 @@ "535-object-statics.ts": 2312, "5350-object-values-wide-record.ts": 2790, "5351-object-values-methods-index.ts": 3275, + "5352-final-field-loop-reads.ts": 2894, + "5353-captured-const-values.ts": 3727, + "5354-captured-const-tdz.ts": 2243, + "5355-captured-function-declarations.ts": 2917, + "5356-captured-tdz-ordering.ts": 2726, + "5357-init-only-module-bindings.ts": 1956, "536-map-seed-array.ts": 2712, "5360-array-sort-comparator-calls.ts": 3175, "5361-array-sort-inconsistent-comparators.ts": 3161, @@ -1761,6 +1811,16 @@ "5364-array-sort-union-and-boolean-elements.ts": 8514, "5365-array-sort-dynamic-values.js": 2402, "537-map-iter-drains.ts": 2867, + "5370-zlib-stream-roundtrip.ts": 3486, + "5371-zlib-stream-chunking.cjs": 4025, + "5372-zlib-stream-errors.cjs": 2794, + "5373-zlib-stream-options.cjs": 3210, + "5374-zlib-stream-flush-params.ts": 3572, + "5375-zlib-stream-promises.ts": 3156, + "5376-zlib-constants.cjs": 2490, + "5377-zlib-stream-backpressure.cjs": 2917, + "5378-zlib-sync-options.cjs": 2777, + "5379-zlib-callback-options.ts": 2861, "538-map-set-for-of.ts": 2756, "5390-interface-loop-element-lifetimes.ts": 2999, "5391-interface-dispatch-hierarchies.ts": 3168, @@ -1782,7 +1842,18 @@ "5411-inspect-class-hierarchies.ts": 2712, "5412-inspect-dynamic-collections.ts": 2792, "5413-inspect-class-values.cjs": 3020, + "5415-zlib-dynamic-inputs.cjs": 3606, "542-union-element-arrays.ts": 2846, + "5420-readable-from-generators.ts": 3459, + "5421-readable-from-generators.cjs": 3156, + "5422-stream-async-push-wakeup.ts": 3051, + "5423-stream-end-and-eos-timing.ts": 3065, + "5424-stream-write-buffer-encoding.ts": 2687, + "5425-pipeline-generator-stage.ts": 2503, + "5426-pipeline-generator-errors.ts": 3640, + "5427-pipeline-generator-backpressure.ts": 3096, + "5428-pipeline-generator-stages.cjs": 2057, + "5429-pipeline-generator-buffers.ts": 2918, "5430-dyn-number-string-boxes.ts": 3059, "5435-crypto-hash-algorithms.ts": 2030, "5436-crypto-hash-names.js": 2098, @@ -1813,30 +1884,99 @@ "5462-locale-compare-probe-sweep.ts": 3400, "5463-locale-compare-combining-letters.ts": 2733, "5464-locale-compare-fractions.ts": 2833, + "5465-dyn-index-read-arrays.ts": 3603, + "5466-dyn-index-read-special.ts": 2823, "5470-strip-vt-osc.cjs": 2497, "5471-strip-vt-csi.cjs": 2651, "5472-strip-vt-random.cjs": 2375, "5473-strip-vt-unterminated.cjs": 3521, "5474-strip-vt-surrogates.ts": 2534, + "5510-assert-error-properties.js": 2744, + "5511-assert-error-class.ts": 3998, + "5512-assert-error-options.js": 3572, + "5513-assert-deep-equal-loose.js": 3537, + "5514-assert-deep-equal-typed.ts": 2922, + "5515-assert-deep-strict-typed-messages.ts": 2776, + "5516-assert-deep-equal-edges.js": 3465, + "5517-assert-partial-deep-equal.js": 4023, + "5518-assert-partial-deep-typed.ts": 3538, + "5519-assert-partial-deep-typed-edges.ts": 3741, + "5520-assert-record-literal-failures.ts": 3435, + "5520-dyn-property-read-receivers.ts": 3114, + "5521-assert-empty-message.js": 3945, + "5521-dyn-property-read-keys.ts": 2472, + "5522-assert-throws-failure-properties.ts": 3505, + "5523-assert-class-options.js": 3797, + "5524-assert-class-skip-prototype.ts": 2105, + "5530-buffer-pool.ts": 3788, + "5531-buffer-pool-first-slab.ts": 5451, + "5532-buffer-pool-first-slab-untyped.mjs": 37592, "5540-fs-callback-forms.cjs": 3233, "5541-fs-stats-fields.cjs": 3564, "5542-fs-promises-parity.cjs": 3940, "5543-fs-promises-typed.ts": 1655, "5544-fs-opendir.cjs": 3024, "5545-fs-options-encoding.cjs": 2994, + "5546-fs-read-stream.cjs": 3808, + "5547-fs-write-stream.cjs": 5798, "5548-fs-callback-arguments.cjs": 4041, "5549-fs-callback-option-checks.cjs": 3738, "5550-locale-compare-ascii-words.ts": 2509, "5551-array-search-references.ts": 3475, "5552-string-appends.ts": 2703, "5553-string-append-char-code.ts": 2870, + "5554-fs-streams-typed.ts": 4016, + "5555-fs-callback-turns.cjs": 3136, + "5555-http-server-close-keep-alive.ts": 6777, + "5556-http-client-agent-connection.ts": 3427, + "5557-http-outgoing-header-values.ts": 2893, + "5558-http-incoming-duplicate-headers.ts": 3230, + "5559-http-expect-header.ts": 3517, + "5560-http-response-framing.ts": 5004, + "5561-http-max-header-size.ts": 3610, + "5562-https-verify-errors.ts": 3646, + "5563-http-keep-alive-timeout.ts": 11895, + "5564-http-request-signal.ts": 4124, "5565-fs-dirent-untyped.mjs": 2600, "5566-fs-dirent-dyn-boundaries/main.mjs": 2607, + "5567-fs-cpsync.ts": 3736, + "5568-fs-cp-errors.js": 4246, + "5569-fs-cp-symlinks.ts": 2136, + "5570-fs-cp-async.ts": 3348, + "5571-fs-cp-deep.ts": 4817, "5585-record-results-inlined.ts": 2922, "5595-net-ref-unref.ts": 2593, + "5600-dyn-cycle-self.ts": 2318, + "5601-dyn-cycle-mutual.ts": 2455, + "5602-dyn-cycle-arrays.ts": 3115, + "5603-dyn-cycle-identity.ts": 3197, + "5604-dyn-cycle-checks.ts": 3210, + "5605-dyn-cycle-containers.ts": 2935, + "5606-dyn-cycle-capsules.ts": 2308, + "5610-readable-from-sets-and-views.ts": 3154, + "5611-readable-from-class-iterables.ts": 3984, + "5612-readable-from-iterables.cjs": 3740, + "5613-readable-from-iterable-timing.cjs": 4549, + "5614-readable-from-iterable-errors.cjs": 4240, + "5615-next-tick-callback-phases.ts": 3053, + "5616-next-tick-callback-phases.cjs": 3322, + "5617-pipeline-function-ends.ts": 2710, + "5618-pipeline-function-ends-errors.ts": 3693, + "5619-pipeline-function-ends.cjs": 3677, + "5620-timers-promises-values.ts": 2181, + "5621-util-format-float.ts": 2886, + "5622-events-once.mjs": 3520, + "5623-events-once.ts": 2509, + "5624-timers-promises-unawaited.ts": 2304, + "5625-promisify-exec.ts": 7463, + "5626-promisify-forms.ts": 2936, "5627-exec-callback.ts": 1900, + "5628-promisify-timers.ts": 3194, + "5629-promisify-stream.ts": 2517, + "5630-promisify-zlib.ts": 2503, "5631-promisify-crypto.ts": 2200, "5632-promisify-dns.ts": 1700, + "5633-zlib-typed-options.ts": 1800, "5644-net-socket-addresses.ts": 2220, "5645-net-socket-callbacks.ts": 2442, "5646-net-server-connections.ts": 2389, @@ -1844,7 +1984,84 @@ "5648-dns-lookup-forms.ts": 2992, "5649-dns-service-servers.ts": 2712, "5650-dns-lookup-errors-untyped.cjs": 2822, + "5651-tls-socket-protocol.ts": 2771, + "5652-http-client-trailer-overflow.ts": 3884, + "5653-net-socket-read-ahead.ts": 4524, + "5654-https-empty-servername.ts": 3451, + "5655-dgram-socket-options.ts": 3481, + "5656-dgram-option-states.ts": 3002, + "5657-net-reset-and-address.ts": 3415, + "5658-net-reset-dynamic.cjs": 2564, + "5659-net-blocklist-socketaddress.ts": 4443, + "5664-tls-client-identity.ts": 3727, + "5674-union-instanceof-array-arms.ts": 2159, + "5680-array-growth-after-copy.ts": 3791, "5681-array-pop-and-length.ts": 3754, + "5682-publish-wide-objects.ts": 3326, + "5683-date-local-fields-tz.ts": 3848, + "5700-cycle-release-retained.ts": 2156, + "5701-cycle-release-old-edges.ts": 2601, + "5702-publish-candidate-buffers.ts": 3469, + "5703-lifetime-region-debug.ts": 3115, + "5704-publish-refused-restores-counts.ts": 2979, + "5710-map-number-keys-hashed.ts": 3019, + "5711-map-number-write-after-read.ts": 2613, + "5712-map-number-overwrite-at-growth.ts": 2614, + "5714-cycle-large-candidate-buffers.ts": 3979, + "5715-json-parse-fitted-tables.ts": 3940, + "5717-set-prototype-of-primitives.ts": 3887, + "5719-int32-fields-dynamic-operators.ts": 3207, + "5720-intern-table-basics.ts": 3625, + "5721-intern-table-values.ts": 4014, + "5722-intern-table-workers.ts": 3210, + "5723-intern-table-shared-types.ts": 4601, + "5724-intern-table-array-keys.ts": 4072, + "5725-intern-table-callbacks.ts": 1996, + "5726-intern-table-key-limits.ts": 2644, + "5740-atomics-operation-semantics.ts": 4810, + "5741-atomics-contended-counters.ts": 2504, + "5742-atomics-lock-ordering.ts": 3000, + "5743-atomics-wait-notify.ts": 3638, + "5744-atomics-ring-buffer.ts": 3352, + "5745-atomics-semaphore-stress.ts": 3671, + "5746-shared-bulk-operations.ts": 2732, + "5747-fs-stat-throw-if-no-entry.ts": 2662, + "5748-fs-stat-throw-if-no-entry-js.cjs": 3085, + "5752-string-number-concat.ts": 3186, + "5760-crypto-hash-stream.ts": 3011, + "5761-crypto-hash-stream-states.ts": 3241, + "5762-crypto-hash-stream.cjs": 3286, + "5763-crypto-sha-blocks.ts": 3264, + "5764-class-structural-view-writes.ts": 4118, + "5764-crypto-hmac-edges.cjs": 2105, + "5765-class-structural-view-identity.ts": 3529, + "5766-class-structural-view-callbacks.ts": 3162, + "5767-class-opaque-accessor-view.ts": 2108, + "5770-guarded-union-narrows.ts": 2881, + "5771-guarded-downcasts.ts": 3555, + "5772-guarded-catch-narrows.ts": 2601, + "5773-stale-narrowing-inherited.ts": 2486, + "5776-power-of-two-remainder-integer-path.ts": 3263, + "5780-readline-history-options.ts": 2909, + "5781-readline-history-options.cjs": 3288, + "5782-readline-terminal-editing.ts": 2661, + "5783-readline-terminal-completion.cjs": 4948, + "5784-readline-terminal-promises.ts": 3042, + "5785-readline-terminal-keys.ts": 3376, + "5786-readline-terminal-history.cjs": 2745, + "5787-readline-cursor-helpers.cjs": 3241, + "5788-readline-promises-readline.ts": 3322, + "5789-readline-cursor-pos.cjs": 3206, + "5790-readline-keypress-events.ts": 3280, + "5791-dyn-constant-key-read-sites.ts": 2813, + "5791-readline-keypress-interfaces.cjs": 3397, + "5792-json-parse-repeated-layouts.ts": 3939, + "5792-readline-keypress-listeners.ts": 2151, + "5793-worker-message-channel.ts": 3274, + "5794-worker-message-port-close-order.ts": 3161, + "5795-worker-message-port-batches.ts": 1925, + "5796-worker-message-channel.cjs": 2678, + "5797-worker-message-port-transfer.ts": 3445, "5800-broadcast-channel.ts": 2899, "5801-broadcast-channel.cjs": 2727, "5802-broadcast-channel-workers.ts": 2809, @@ -1860,6 +2077,7 @@ "5812-dyn-box-async-fields.ts": 3522, "5813-mock-timers-module.ts": 3766, "5814-mock-timers-date.ts": 3760, + "5830-timers-promises-signal.ts": 3875, "5831-util-types-brands.ts": 3347, "5832-util-types-brands-dynamic.js": 3250, "5833-url-search-params-iterators.ts": 2678, @@ -1868,27 +2086,46 @@ "5836-url-search-params-tuple-pairs.ts": 2118, "5837-util-format-json-circular.ts": 3117, "5838-qs-stringify-bigint.ts": 2207, + "5839-timers-promises-scheduler.ts": 3195, + "5840-promise-with-resolvers-js.js": 2842, "5841-json-typed-dyn-members.ts": 5467, "5842-qs-parse-null-prototype.ts": 1774, "5843-qs-parse-null-prototype-js.mjs": 1559, "5844-date-unknown-values.ts": 3969, + "5845-groupby-null-prototype.ts": 2868, "5846-events-esm-default-max.ts": 2633, "5847-events-cjs-default-max.cjs": 2823, + "5849-class-view-observations.ts": 3073, + "5850-using-disposable-values.ts": 3391, + "5851-using-suppressed-error-values.ts": 2932, + "5852-using-disposable-values-js.js": 3132, + "5853-set-methods-map-arguments.ts": 3034, + "5854-array-from-async.ts": 4140, + "5855-array-from-async-js.js": 3324, + "5856-error-is-error.ts": 2937, "5857-regexp-unicode-sets.ts": 3350, "5858-disposable-stack.ts": 3050, "5859-disposable-stack-js.js": 3100, + "5860-url-ipv4-hosts.ts": 3101, "5870-worker-thread-names.ts": 3106, "5871-worker-untransferable.cjs": 2786, + "5872-readline-positional-create-interface.mjs": 3151, + "5873-readline-promises-namespace.cjs": 2023, "5874-worker-stdio-streams.ts": 2839, "5875-worker-stdout-pipe.cjs": 2740, "5879-worker-stdin.ts": 3158, + "5880-emitter-leak-warning-delivery.ts": 3418, "5881-inspect-weak-collections.ts": 2615, "5882-string-decoder-runtime-encoding.ts": 2541, "5883-os-set-priority.ts": 2380, "5884-inspect-layout-options.ts": 3108, + "5885-events-max-listener-helpers.ts": 2992, "5886-inspect-dates.ts": 2559, "5887-inspect-typed-arrays.ts": 3448, "5888-inspect-sorted.ts": 2998, + "5889-timers-scheduler-signal.ts": 2284, + "5890-while-string-length-counters.ts": 2516, + "5892-buffer-write-utf16le-runs.ts": 3107, "5900-float16-array.ts": 3000, "5901-float16-array-js.js": 2900, "5904-async-generator-completion.ts": 2950, @@ -1901,10 +2138,36 @@ "5913-prototype-type-error-messages.ts": 2484, "5914-dyn-in-class-prototype-members.ts": 3169, "5915-dyn-error-string-conversions.ts": 3200, + "5919-record-unions-shared-arms.ts": 3182, + "5920-promise-try-args.ts": 3028, + "5921-promise-try-args-js.js": 3179, "5922-matchall-groups-callbacks.ts": 3650, "5923-matchall-groups-js.js": 3970, + "5924-js-literal-identity.js": 2577, + "5925-class-view-identity-observers.ts": 2297, + "5926-class-view-switch-js.js": 2243, + "5927-switch-object-identity.ts": 2986, + "5928-groupby-iterables.ts": 3444, + "5929-groupby-js.js": 2997, + "5930-contract-int32-arrays.ts": 3715, + "5930-method-census-asserted-number-keys.ts": 3330, + "5930-record-union-pointers.ts": 3182, + "5931-asserted-reads-keep-number-parameters.ts": 2990, + "5931-record-union-unit-arms.ts": 2745, + "5932-generic-class-observed-methods.ts": 4272, + "5932-record-union-trees.ts": 2863, + "5933-class-fields-unknown-object.ts": 4189, + "5933-record-union-conversions.ts": 4140, + "5934-js-class-fields-property-order.js": 4613, + "5934-record-union-set-members.ts": 2265, + "5935-js-subclass-fields-and-element-calls.js": 2484, + "5936-checked-js-class-types.js": 3412, + "5937-override-narrowed-union-returns.ts": 3036, "5938-object-literal-computed-keys-keep-methods.ts": 3195, "5939-js-let-object-reassigned-wider.js": 1945, + "5940-callback-helper-specialization.ts": 3109, + "5941-callback-specialization-recursion.ts": 3112, + "5942-callback-specialization-throws.ts": 3221, "5943-fields-read-before-assignment.ts": 6856, "5944-js-fields-read-before-assignment.js": 4247, "5945-local-class-observed-methods.ts": 5063, @@ -1917,15 +2180,26 @@ "5956-erased-generic-members.ts": 6730, "5958-asserted-union-receiver.ts": 2430, "5959-date-to-json.ts": 3330, + "5960-frame-records-own-references.ts": 3563, "5960-generic-mapped-types.ts": 2540, + "5961-frame-spread-updates.ts": 3260, + "5962-frame-tuples.ts": 3208, "5970-net-connect-host-name.ts": 2400, "5971-buffer-chunks-to-string.ts": 2700, "5972-net-set-encoding-listeners.ts": 2550, + "5973-large-closure-environment.ts": 4499, "5974-typeof-module-exports.cjs": 2000, "5975-zlib-class-values.ts": 2800, + "5976-builtin-class-aliases.ts": 3100, + "5977-builtin-class-aliases.cjs": 2950, + "5978-builtin-class-alias-modules/main.ts": 2750, "5990-worker-uncloneable.mjs": 3283, "5991-worker-async-dispose.mjs": 5699, "5992-worker-thread-name-lifetime.mjs": 2687, + "5993-array-length-loop-resizing.ts": 3187, + "5994-array-length-sparse-loops.ts": 3292, + "5995-array-dense-loop-boundaries.ts": 2596, + "5996-rejections-between-macrotasks.cjs": 3034, "600-closures-basic.ts": 2349, "601-closures-loops.ts": 1788, "602-closures-identity-recursion.ts": 2372, @@ -1933,7 +2207,23 @@ "6021-asserted-literal-union-arm.ts": 3400, "603-closures-rc-stress.ts": 1832, "604-closures-for-of.ts": 2639, + "6041-error-cause-redeclared.ts": 2988, + "6042-regex-match-object-destructuring.ts": 3393, + "6043-string-index-narrowed-union.ts": 2983, "605-closures-forward-capture-tdz.ts": 3277, + "6100-record-get-or-create.ts": 3771, + "6101-record-get-or-create-unchecked/main.ts": 2107, + "6103-typeof-builtin-bindings.cjs": 2202, + "6104-typeof-builtin-bindings.mjs": 2031, + "6105-cjs-top-level-this/main.js": 1371, + "6106-esm-top-level-this.mjs": 1915, + "6110-cjs-speculative-wrapper-fences.cjs": 1980, + "6140-cjs-require-tdz-throws/main.js": 2731, + "6141-cjs-require-tdz-checked/main.js": 2856, + "6142-cjs-require-published-early/main.js": 3400, + "6143-cjs-require-json/main.js": 3245, + "6144-cjs-require-builtin-nested/main.js": 2181, + "6145-cjs-require-cycle/main.js": 3507, "700-classes-basic.ts": 2530, "701-classes-composition.ts": 2357, "702-classes-this-capture.ts": 2140, @@ -2041,6 +2331,7 @@ "array-checked-callback-results.mjs": 3053, "array-checked-constructor.js": 3001, "array-concat-widening.ts": 2947, + "array-deque-unshift.ts": 3233, "array-element-compound-updates.ts": 3325, "array-fill-unit-values.ts": 2281, "array-from-checked.js": 3228, @@ -2049,6 +2340,7 @@ "array-numeric-own-keys.ts": 2770, "array-optional-number-write-index.ts": 2584, "array-primitive-ordering.ts": 3806, + "array-queue-head-room.ts": 2847, "array-read-helper-effects.ts": 3223, "array-refined-storage.ts": 3132, "array-sort-dense-effects.ts": 2813, @@ -2151,6 +2443,8 @@ "class-symbol-accessors.ts": 2458, "class-symbol-cache.mjs": 2883, "class-value-constructor-exceptions.ts": 2222, + "closed-virtual-dispatch-modules/main.ts": 2682, + "closed-virtual-dispatch.ts": 3944, "closure-nullable-union-return.ts": 2472, "closure-union-locals.ts": 2465, "collection-array-cycles.ts": 2552, @@ -2220,7 +2514,11 @@ "fresh-array-union-layout.js": 2480, "frozen-checked-dictionaries.mjs": 2629, "fs-native-realpath.ts": 2351, + "fs-null-byte-paths-js.cjs": 2762, + "fs-null-byte-paths.ts": 3411, "fs-read-encodings.ts": 2505, + "fs-read-utf8-edges.ts": 3246, + "fs-read-utf8-handle-forms.ts": 3163, "fs-remove-readonly.ts": 2354, "fs-write-string-bytes-union.ts": 2286, "function-apply-value.mjs": 2558, @@ -2247,6 +2545,7 @@ "generic-pattern-bindings.ts": 2698, "generic-record-index.ts": 2753, "generic-visitor-void.ts": 2405, + "global-accumulator-loops.ts": 3053, "global-error-subclasses.ts": 1773, "global-runtime-probes.js": 2288, "global-stored-probes.js": 2726, @@ -2288,6 +2587,8 @@ "js-array-iterators.js": 2559, "js-arrow-bind.cjs": 3343, "js-buffer-callback-brand.cjs": 2262, + "js-builder-string-method-names/main.ts": 4550, + "js-callable-builder.js": 4500, "js-callable-proxy.js": 3516, "js-chained-native-results.mjs": 2641, "js-checked-array-iterator.js": 2670, @@ -2299,6 +2600,7 @@ "js-class-added-parameters.js": 2472, "js-class-checked-overrides.js": 2571, "js-class-computed-fields.js": 1830, + "js-class-constructor-returns/main.ts": 4350, "js-class-enumeration.js": 3296, "js-class-field-accessor-override.js": 2567, "js-class-fluent-overrides.js": 2822, @@ -2322,6 +2624,7 @@ "js-dormant-array-write.js": 1888, "js-dormant-overrides.js": 2569, "js-expando-records.js": 2454, + "js-function-prototype-value.js": 3650, "js-implicit-call-results.js": 3064, "js-inferred-callback-exports/main.js": 3830, "js-inferred-class-array-return.js": 2570, @@ -2337,6 +2640,7 @@ "js-nested-native-descriptors.js": 2412, "js-normalize-dispatch.js": 2793, "js-nullish-inherited-fields.mjs": 2710, + "js-number-parse-dynamic.js": 2700, "js-object-receiver-storage.js": 2613, "js-object-returning-constructors.mjs": 2186, "js-object-shared-arrays.js": 2465, @@ -2345,6 +2649,7 @@ "js-optional-class-fields.js": 2266, "js-optional-member-call.js": 1883, "js-primitive-union-addition.js": 2256, + "js-process-destructure.mjs": 4750, "js-promise-all-values.js": 3079, "js-promise-catch-handler.js": 1749, "js-promise-finally-order.js": 1913, @@ -2352,6 +2657,7 @@ "js-prototype-host-boundary.js": 2224, "js-reflect-receivers.js": 3119, "js-regex-record-captures.js": 2010, + "js-reshaped-object-literal.js": 5700, "js-scalar-iterables.js": 2477, "js-shared-method-dispatch.js": 2033, "js-specialized-field-method.js": 2335, @@ -2365,6 +2671,9 @@ "json-runtime-options.mjs": 1997, "json-stringify-value.js": 2563, "json-typed-parse-direct.ts": 2355, + "json-typed-parse-index-signatures.ts": 3686, + "json-typed-parse-optional-members.ts": 3044, + "json-typed-parse-union-members.ts": 3090, "literal-switch-dispatch.ts": 2500, "live-reference-shared-converters.ts": 2552, "llvm-read-receiver-lifetime.ts": 2343, @@ -2386,6 +2695,7 @@ "loop-reference-lifetimes.ts": 2542, "map-checked-boundary/main.ts": 2629, "map-checked-seed.js": 3013, + "map-get-set-fusion.ts": 3284, "map-number-keys-dense.ts": 2562, "map-read-lifetimes-exits.ts": 2227, "map-read-lifetimes.ts": 2725, @@ -2577,6 +2887,8 @@ "promise-resolve-void.ts": 2341, "prototype-method-overrides.mjs": 4870, "prototype-named-data-records.ts": 2513, + "rc-last-use-moves.ts": 2854, + "rc-private-array-writes.ts": 3076, "record-field-presence.ts": 3025, "record-key-order.ts": 2807, "record-optional-json-presence.ts": 2623, @@ -2592,6 +2904,8 @@ "regex-checked-casts.ts": 2350, "regex-checked-storage.js": 2669, "regex-exec-long-subject.ts": 4533, + "regex-exec-result-arrays.ts": 3204, + "regex-fs-text-surfaces.ts": 3720, "regex-input-lifetimes.ts": 2535, "regex-native-matcher.ts": 3683, "regex-optional-captures.ts": 2606, @@ -2616,12 +2930,20 @@ "set-iterator-seeds.ts": 2896, "set-optional-seeds.ts": 2121, "small-collection-transitions.ts": 2645, + "speculative-integer-globals.ts": 2544, + "speculative-integer-loops.ts": 2308, + "speculative-integer-reads.ts": 3059, "stack-depth-async-bodies.ts": 1910, "stack-overflow-async.ts": 2447, "stack-overflow-caught.ts": 2424, "stack-overflow-finally.ts": 2613, "stack-overflow-shapes.ts": 2534, "stack-overflow-uncaught.ts": 2317, + "stackless-async-concurrency.ts": 3322, + "stackless-async-context.ts": 1853, + "stackless-async-exceptions.ts": 3251, + "stackless-async-ordering.ts": 3551, + "stackless-async-unhandled.ts": 1779, "static-field-updates/main.ts": 2050, "static-method-receivers.ts": 2243, "stdio-values.mjs": 2578, @@ -2629,10 +2951,15 @@ "stdio-write-overrides.mjs": 2495, "stored-native-builtins.js": 2831, "stored-object-helpers.js": 2747, + "stream-dynamic-entry-kinds.cjs": 2778, + "stream-dynamic-entry-kinds.ts": 2624, + "stream-encoded-from-entries.cjs": 2924, + "stream-encoded-from-entries.ts": 3274, "string-char-code-integer-loops.ts": 2495, "string-code-points.ts": 1970, "string-concat-number-parts.ts": 2044, "string-construction-order.ts": 2502, + "string-field-append.ts": 2926, "string-lifetimes-boundaries.ts": 2481, "string-lifetimes-evaluation.ts": 2486, "string-lifetimes-iteration.ts": 2043, @@ -2659,6 +2986,7 @@ "text-codec-var.ts": 2182, "text-encode-into-boundaries.ts": 1971, "timers-clear-many.ts": 2704, + "truthy-object-logical.ts": 5950, "tuple-and-string-search.ts": 2663, "tuple-array-union.ts": 2706, "tuple-callback-visitation.ts": 3464, @@ -4053,6 +4381,8 @@ "4451-int32-field-storage.ts": 3668, "4452-int32-field-dynamic-writes.ts": 4531, "4453-int32-closed-locals.ts": 2786, + "4454-int32-field-reflection-reads.ts": 3494, + "4455-live-view-reads-without-stores.ts": 4871, "4471-unassigned-field-reads.ts": 3896, "4476-string-ascii-header-fact.ts": 3522, "4477-buffer-write-ascii-runs.ts": 3321, @@ -4221,6 +4551,8 @@ "5020-lifetime-region-large.ts": 4812, "5021-lifetime-region-opaque-holders.ts": 3957, "5022-lifetime-region-workers.ts": 38841, + "5023-lifetime-region-await.ts": 5986, + "5024-lifetime-region-await-forms.ts": 5123, "503-array-functions.ts": 2982, "504-array-rc-stress.ts": 3289, "5040-promise-all-race-turns.ts": 4439, @@ -4402,6 +4734,16 @@ "5248-event-target-subclass-warnings.ts": 4513, "5249-event-target-subclass-unconstructed.ts": 2679, "525-map-rc-stress.ts": 2764, + "5250-cipher-aes-gcm.ts": 8740, + "5251-cipher-aes-cbc.ts": 42449, + "5252-cipher-aes-ctr.ts": 26060, + "5253-cipher-chacha20-poly1305.ts": 8385, + "5254-cipher-errors.js": 28463, + "5255-cipher-encodings.cjs": 25164, + "5256-cipher-deprecation.cjs": 45696, + "5257-cipher-vectors.ts": 7138, + "5258-cipher-handles.js": 24870, + "5259-cipher-chunked.ts": 7052, "526-set-basics.ts": 2480, "5260-record-unknown-identity.ts": 3505, "5261-optional-field-identity.ts": 3914, @@ -4441,6 +4783,14 @@ "5319-map-iteration-values.ts": 4062, "532-record-arrays-rc-stress.ts": 2708, "5320-map-iterate-across-compaction.ts": 2736, + "5321-integer-closed-arithmetic-loops.ts": 3710, + "5321-json-parse-cycles.ts": 3383, + "5322-integer-variable-step-loops.ts": 3384, + "5322-json-stringify-parsed-trees.ts": 3838, + "5323-dyn-numeric-index-reads.ts": 3192, + "5323-int32-nonnegative-slot-facts.ts": 3774, + "5324-int32-initializer-locals.ts": 2663, + "5324-structured-clone-documents.ts": 4264, "5325-record-boundary-returns.ts": 3046, "5326-record-boundary-params.ts": 2989, "5327-record-boundary-identity.ts": 3529, @@ -4449,14 +4799,36 @@ "533-array-element-cycles.ts": 3118, "5330-record-boundary-fields.ts": 3048, "5331-record-boundary-methods.ts": 2560, + "5332-int32-narrowed-slot-facts.ts": 3588, + "5333-integer-guarded-while-counters.ts": 3896, "5333-record-boundary-nullable-argument.ts": 3100, "5334-fetch-mentions-without-calls.ts": 4032, + "5335-stream-operators-values.ts": 6378, "5335-top-level-await-pending-exit-code.ts": 2383, + "5336-stream-operators-promise-timing.ts": 4282, "5336-top-level-await-pending-exit-listener.ts": 2814, "5337-dc-subscriber-throw-next-tick.cjs": 3273, + "5337-stream-operators-concurrency.ts": 8858, "5338-js-tdz-function-element.js": 3733, + "5338-stream-operators-checks.cjs": 5284, "5339-array-buffer-constructor.js": 2884, + "5339-stream-for-await-hops.ts": 5704, "534-record-width-subtyping.ts": 3530, + "5340-readable-from-options.ts": 4031, + "5341-compound-any-operand.ts": 3905, + "5341-generator-callback-params.ts": 4564, + "5341-integer-recursion-boundaries.ts": 4375, + "5341-readable-from-dynamic.cjs": 5807, + "5342-compound-any-operand.cjs": 3381, + "5342-generator-scalar-channels.ts": 3638, + "5342-integer-recursion-stack-and-mutual.ts": 3513, + "5342-stream-compose.ts": 7562, + "5343-any-flow-compound.ts": 2631, + "5343-integer-recursion-array-index.ts": 4214, + "5343-iterator-result-records.ts": 2610, + "5343-stream-compose-edges.ts": 10326, + "5344-json-parse-undeclared-keys.ts": 5823, + "5344-stream-pipeline-timing.ts": 4092, "5345-object-values-for-of-unknown.ts": 4013, "5346-object-values-length-dynamic.ts": 3930, "5347-object-values-loop-control.ts": 3598, @@ -4465,6 +4837,12 @@ "535-object-statics.ts": 2862, "5350-object-values-wide-record.ts": 3272, "5351-object-values-methods-index.ts": 3637, + "5352-final-field-loop-reads.ts": 4118, + "5353-captured-const-values.ts": 3887, + "5354-captured-const-tdz.ts": 3465, + "5355-captured-function-declarations.ts": 4030, + "5356-captured-tdz-ordering.ts": 4031, + "5357-init-only-module-bindings.ts": 3717, "536-map-seed-array.ts": 3237, "5360-array-sort-comparator-calls.ts": 4265, "5361-array-sort-inconsistent-comparators.ts": 4181, @@ -4473,6 +4851,16 @@ "5364-array-sort-union-and-boolean-elements.ts": 14300, "5365-array-sort-dynamic-values.js": 2919, "537-map-iter-drains.ts": 3050, + "5370-zlib-stream-roundtrip.ts": 4546, + "5371-zlib-stream-chunking.cjs": 6071, + "5372-zlib-stream-errors.cjs": 6496, + "5373-zlib-stream-options.cjs": 6767, + "5374-zlib-stream-flush-params.ts": 5113, + "5375-zlib-stream-promises.ts": 6129, + "5376-zlib-constants.cjs": 3393, + "5377-zlib-stream-backpressure.cjs": 4992, + "5378-zlib-sync-options.cjs": 4120, + "5379-zlib-callback-options.ts": 4524, "538-map-set-for-of.ts": 3029, "5390-interface-loop-element-lifetimes.ts": 3566, "5391-interface-dispatch-hierarchies.ts": 3731, @@ -4494,7 +4882,18 @@ "5411-inspect-class-hierarchies.ts": 2423, "5412-inspect-dynamic-collections.ts": 3042, "5413-inspect-class-values.cjs": 3217, + "5415-zlib-dynamic-inputs.cjs": 4054, "542-union-element-arrays.ts": 2812, + "5420-readable-from-generators.ts": 5843, + "5421-readable-from-generators.cjs": 3752, + "5422-stream-async-push-wakeup.ts": 3679, + "5423-stream-end-and-eos-timing.ts": 4453, + "5424-stream-write-buffer-encoding.ts": 3204, + "5425-pipeline-generator-stage.ts": 3700, + "5426-pipeline-generator-errors.ts": 4597, + "5427-pipeline-generator-backpressure.ts": 4175, + "5428-pipeline-generator-stages.cjs": 9702, + "5429-pipeline-generator-buffers.ts": 6406, "5430-dyn-number-string-boxes.ts": 3930, "5435-crypto-hash-algorithms.ts": 3451, "5436-crypto-hash-names.js": 3185, @@ -4525,30 +4924,99 @@ "5462-locale-compare-probe-sweep.ts": 4164, "5463-locale-compare-combining-letters.ts": 3755, "5464-locale-compare-fractions.ts": 3312, + "5465-dyn-index-read-arrays.ts": 3798, + "5466-dyn-index-read-special.ts": 4384, "5470-strip-vt-osc.cjs": 2708, "5471-strip-vt-csi.cjs": 3038, "5472-strip-vt-random.cjs": 2850, "5473-strip-vt-unterminated.cjs": 4462, "5474-strip-vt-surrogates.ts": 3554, + "5510-assert-error-properties.js": 5349, + "5511-assert-error-class.ts": 4329, + "5512-assert-error-options.js": 3469, + "5513-assert-deep-equal-loose.js": 4138, + "5514-assert-deep-equal-typed.ts": 3792, + "5515-assert-deep-strict-typed-messages.ts": 3601, + "5516-assert-deep-equal-edges.js": 4686, + "5517-assert-partial-deep-equal.js": 2953, + "5518-assert-partial-deep-typed.ts": 4685, + "5519-assert-partial-deep-typed-edges.ts": 4346, + "5520-assert-record-literal-failures.ts": 4088, + "5520-dyn-property-read-receivers.ts": 4057, + "5521-assert-empty-message.js": 3734, + "5521-dyn-property-read-keys.ts": 3703, + "5522-assert-throws-failure-properties.ts": 4256, + "5523-assert-class-options.js": 4903, + "5524-assert-class-skip-prototype.ts": 4411, + "5530-buffer-pool.ts": 4413, + "5531-buffer-pool-first-slab.ts": 5557, + "5532-buffer-pool-first-slab-untyped.mjs": 220640, "5540-fs-callback-forms.cjs": 3079, "5541-fs-stats-fields.cjs": 4039, "5542-fs-promises-parity.cjs": 3312, "5543-fs-promises-typed.ts": 2944, "5544-fs-opendir.cjs": 3622, "5545-fs-options-encoding.cjs": 3497, + "5546-fs-read-stream.cjs": 5295, + "5547-fs-write-stream.cjs": 6075, "5548-fs-callback-arguments.cjs": 3905, "5549-fs-callback-option-checks.cjs": 3471, "5550-locale-compare-ascii-words.ts": 2824, "5551-array-search-references.ts": 3964, "5552-string-appends.ts": 3653, "5553-string-append-char-code.ts": 3292, + "5554-fs-streams-typed.ts": 4873, + "5555-fs-callback-turns.cjs": 3708, + "5555-http-server-close-keep-alive.ts": 15618, + "5556-http-client-agent-connection.ts": 4177, + "5557-http-outgoing-header-values.ts": 4083, + "5558-http-incoming-duplicate-headers.ts": 2888, + "5559-http-expect-header.ts": 14863, + "5560-http-response-framing.ts": 11391, + "5561-http-max-header-size.ts": 4316, + "5562-https-verify-errors.ts": 4777, + "5563-http-keep-alive-timeout.ts": 17783, + "5564-http-request-signal.ts": 10735, "5565-fs-dirent-untyped.mjs": 2898, "5566-fs-dirent-dyn-boundaries/main.mjs": 3333, + "5567-fs-cpsync.ts": 4945, + "5568-fs-cp-errors.js": 4099, + "5569-fs-cp-symlinks.ts": 3686, + "5570-fs-cp-async.ts": 4176, + "5571-fs-cp-deep.ts": 5306, "5585-record-results-inlined.ts": 3681, "5595-net-ref-unref.ts": 3036, + "5600-dyn-cycle-self.ts": 3365, + "5601-dyn-cycle-mutual.ts": 4239, + "5602-dyn-cycle-arrays.ts": 3453, + "5603-dyn-cycle-identity.ts": 3497, + "5604-dyn-cycle-checks.ts": 3761, + "5605-dyn-cycle-containers.ts": 4084, + "5606-dyn-cycle-capsules.ts": 3095, + "5610-readable-from-sets-and-views.ts": 5921, + "5611-readable-from-class-iterables.ts": 3857, + "5612-readable-from-iterables.cjs": 5191, + "5613-readable-from-iterable-timing.cjs": 5131, + "5614-readable-from-iterable-errors.cjs": 4553, + "5615-next-tick-callback-phases.ts": 4347, + "5616-next-tick-callback-phases.cjs": 3259, + "5617-pipeline-function-ends.ts": 60581, + "5618-pipeline-function-ends-errors.ts": 13092, + "5619-pipeline-function-ends.cjs": 14768, + "5620-timers-promises-values.ts": 3726, + "5621-util-format-float.ts": 3395, + "5622-events-once.mjs": 3355, + "5623-events-once.ts": 4676, + "5624-timers-promises-unawaited.ts": 4420, + "5625-promisify-exec.ts": 8488, + "5626-promisify-forms.ts": 3100, "5627-exec-callback.ts": 2500, + "5628-promisify-timers.ts": 3913, + "5629-promisify-stream.ts": 3357, + "5630-promisify-zlib.ts": 3392, "5631-promisify-crypto.ts": 2600, "5632-promisify-dns.ts": 2000, + "5633-zlib-typed-options.ts": 2200, "5644-net-socket-addresses.ts": 2393, "5645-net-socket-callbacks.ts": 2060, "5646-net-server-connections.ts": 2755, @@ -4556,7 +5024,84 @@ "5648-dns-lookup-forms.ts": 3416, "5649-dns-service-servers.ts": 4916, "5650-dns-lookup-errors-untyped.cjs": 3889, + "5651-tls-socket-protocol.ts": 30873, + "5652-http-client-trailer-overflow.ts": 3512, + "5653-net-socket-read-ahead.ts": 6212, + "5654-https-empty-servername.ts": 6322, + "5655-dgram-socket-options.ts": 5089, + "5656-dgram-option-states.ts": 4267, + "5657-net-reset-and-address.ts": 3588, + "5658-net-reset-dynamic.cjs": 4002, + "5659-net-blocklist-socketaddress.ts": 5090, + "5664-tls-client-identity.ts": 6340, + "5674-union-instanceof-array-arms.ts": 3943, + "5680-array-growth-after-copy.ts": 4581, "5681-array-pop-and-length.ts": 3216, + "5682-publish-wide-objects.ts": 3376, + "5683-date-local-fields-tz.ts": 3911, + "5700-cycle-release-retained.ts": 3414, + "5701-cycle-release-old-edges.ts": 4210, + "5702-publish-candidate-buffers.ts": 3848, + "5703-lifetime-region-debug.ts": 2875, + "5704-publish-refused-restores-counts.ts": 3759, + "5710-map-number-keys-hashed.ts": 3686, + "5711-map-number-write-after-read.ts": 3642, + "5712-map-number-overwrite-at-growth.ts": 4057, + "5714-cycle-large-candidate-buffers.ts": 4303, + "5715-json-parse-fitted-tables.ts": 4861, + "5717-set-prototype-of-primitives.ts": 3977, + "5719-int32-fields-dynamic-operators.ts": 4013, + "5720-intern-table-basics.ts": 5183, + "5721-intern-table-values.ts": 4244, + "5722-intern-table-workers.ts": 37693, + "5723-intern-table-shared-types.ts": 25115, + "5724-intern-table-array-keys.ts": 43939, + "5725-intern-table-callbacks.ts": 3924, + "5726-intern-table-key-limits.ts": 3414, + "5740-atomics-operation-semantics.ts": 4541, + "5741-atomics-contended-counters.ts": 5176, + "5742-atomics-lock-ordering.ts": 44911, + "5743-atomics-wait-notify.ts": 25944, + "5744-atomics-ring-buffer.ts": 3295, + "5745-atomics-semaphore-stress.ts": 39641, + "5746-shared-bulk-operations.ts": 3506, + "5747-fs-stat-throw-if-no-entry.ts": 3747, + "5748-fs-stat-throw-if-no-entry-js.cjs": 4239, + "5752-string-number-concat.ts": 2903, + "5760-crypto-hash-stream.ts": 3720, + "5761-crypto-hash-stream-states.ts": 3947, + "5762-crypto-hash-stream.cjs": 3953, + "5763-crypto-sha-blocks.ts": 4280, + "5764-class-structural-view-writes.ts": 4359, + "5764-crypto-hmac-edges.cjs": 3178, + "5765-class-structural-view-identity.ts": 4251, + "5766-class-structural-view-callbacks.ts": 3950, + "5767-class-opaque-accessor-view.ts": 3507, + "5770-guarded-union-narrows.ts": 3945, + "5771-guarded-downcasts.ts": 3443, + "5772-guarded-catch-narrows.ts": 3695, + "5773-stale-narrowing-inherited.ts": 3758, + "5776-power-of-two-remainder-integer-path.ts": 3584, + "5780-readline-history-options.ts": 3008, + "5781-readline-history-options.cjs": 5419, + "5782-readline-terminal-editing.ts": 5323, + "5783-readline-terminal-completion.cjs": 5914, + "5784-readline-terminal-promises.ts": 8841, + "5785-readline-terminal-keys.ts": 7210, + "5786-readline-terminal-history.cjs": 5018, + "5787-readline-cursor-helpers.cjs": 4120, + "5788-readline-promises-readline.ts": 3931, + "5789-readline-cursor-pos.cjs": 10833, + "5790-readline-keypress-events.ts": 5781, + "5791-dyn-constant-key-read-sites.ts": 3910, + "5791-readline-keypress-interfaces.cjs": 10593, + "5792-json-parse-repeated-layouts.ts": 4520, + "5792-readline-keypress-listeners.ts": 3779, + "5793-worker-message-channel.ts": 6071, + "5794-worker-message-port-close-order.ts": 4996, + "5795-worker-message-port-batches.ts": 2995, + "5796-worker-message-channel.cjs": 3812, + "5797-worker-message-port-transfer.ts": 3633, "5800-broadcast-channel.ts": 28374, "5801-broadcast-channel.cjs": 27251, "5802-broadcast-channel-workers.ts": 27251, @@ -4572,6 +5117,7 @@ "5812-dyn-box-async-fields.ts": 5278, "5813-mock-timers-module.ts": 4227, "5814-mock-timers-date.ts": 17708, + "5830-timers-promises-signal.ts": 4170, "5831-util-types-brands.ts": 4380, "5832-util-types-brands-dynamic.js": 3658, "5833-url-search-params-iterators.ts": 5339, @@ -4580,27 +5126,46 @@ "5836-url-search-params-tuple-pairs.ts": 3193, "5837-util-format-json-circular.ts": 3408, "5838-qs-stringify-bigint.ts": 3519, + "5839-timers-promises-scheduler.ts": 3897, + "5840-promise-with-resolvers-js.js": 3596, "5841-json-typed-dyn-members.ts": 3238, "5842-qs-parse-null-prototype.ts": 7077, "5843-qs-parse-null-prototype-js.mjs": 5395, "5844-date-unknown-values.ts": 4543, + "5845-groupby-null-prototype.ts": 4290, "5846-events-esm-default-max.ts": 3102, "5847-events-cjs-default-max.cjs": 3039, + "5849-class-view-observations.ts": 4659, + "5850-using-disposable-values.ts": 3072, + "5851-using-suppressed-error-values.ts": 3950, + "5852-using-disposable-values-js.js": 3905, + "5853-set-methods-map-arguments.ts": 3831, + "5854-array-from-async.ts": 5635, + "5855-array-from-async-js.js": 3433, + "5856-error-is-error.ts": 3928, "5857-regexp-unicode-sets.ts": 3900, "5858-disposable-stack.ts": 3750, "5859-disposable-stack-js.js": 3600, + "5860-url-ipv4-hosts.ts": 4018, "5870-worker-thread-names.ts": 31697, "5871-worker-untransferable.cjs": 27481, + "5872-readline-positional-create-interface.mjs": 6166, + "5873-readline-promises-namespace.cjs": 3811, "5874-worker-stdio-streams.ts": 27801, "5875-worker-stdout-pipe.cjs": 31391, "5879-worker-stdin.ts": 33232, + "5880-emitter-leak-warning-delivery.ts": 4319, "5881-inspect-weak-collections.ts": 2915, "5882-string-decoder-runtime-encoding.ts": 3854, "5883-os-set-priority.ts": 3417, "5884-inspect-layout-options.ts": 3596, + "5885-events-max-listener-helpers.ts": 3588, "5886-inspect-dates.ts": 2736, "5887-inspect-typed-arrays.ts": 2997, "5888-inspect-sorted.ts": 2861, + "5889-timers-scheduler-signal.ts": 8928, + "5890-while-string-length-counters.ts": 3822, + "5892-buffer-write-utf16le-runs.ts": 3465, "5900-float16-array.ts": 3750, "5901-float16-array-js.js": 3700, "5904-async-generator-completion.ts": 3800, @@ -4613,10 +5178,36 @@ "5913-prototype-type-error-messages.ts": 4068, "5914-dyn-in-class-prototype-members.ts": 4327, "5915-dyn-error-string-conversions.ts": 4247, + "5919-record-unions-shared-arms.ts": 4941, + "5920-promise-try-args.ts": 3195, + "5921-promise-try-args-js.js": 3082, "5922-matchall-groups-callbacks.ts": 5960, "5923-matchall-groups-js.js": 6650, + "5924-js-literal-identity.js": 3065, + "5925-class-view-identity-observers.ts": 2685, + "5926-class-view-switch-js.js": 3057, + "5927-switch-object-identity.ts": 2661, + "5928-groupby-iterables.ts": 4500, + "5929-groupby-js.js": 3482, + "5930-contract-int32-arrays.ts": 4745, + "5930-method-census-asserted-number-keys.ts": 4815, + "5930-record-union-pointers.ts": 4941, + "5931-asserted-reads-keep-number-parameters.ts": 3015, + "5931-record-union-unit-arms.ts": 4558, + "5932-generic-class-observed-methods.ts": 5678, + "5932-record-union-trees.ts": 4329, + "5933-class-fields-unknown-object.ts": 4834, + "5933-record-union-conversions.ts": 4357, + "5934-js-class-fields-property-order.js": 4883, + "5934-record-union-set-members.ts": 4003, + "5935-js-subclass-fields-and-element-calls.js": 4470, + "5936-checked-js-class-types.js": 4183, + "5937-override-narrowed-union-returns.ts": 3827, "5938-object-literal-computed-keys-keep-methods.ts": 7194, "5939-js-let-object-reassigned-wider.js": 4436, + "5940-callback-helper-specialization.ts": 4548, + "5941-callback-specialization-recursion.ts": 4448, + "5942-callback-specialization-throws.ts": 3443, "5943-fields-read-before-assignment.ts": 72293, "5944-js-fields-read-before-assignment.js": 40310, "5945-local-class-observed-methods.ts": 6212, @@ -4629,15 +5220,26 @@ "5956-erased-generic-members.ts": 8750, "5958-asserted-union-receiver.ts": 3160, "5959-date-to-json.ts": 4330, + "5960-frame-records-own-references.ts": 3535, "5960-generic-mapped-types.ts": 3300, + "5961-frame-spread-updates.ts": 3732, + "5962-frame-tuples.ts": 3372, "5970-net-connect-host-name.ts": 2800, "5971-buffer-chunks-to-string.ts": 3100, "5972-net-set-encoding-listeners.ts": 3000, + "5973-large-closure-environment.ts": 5145, "5974-typeof-module-exports.cjs": 2500, "5975-zlib-class-values.ts": 4200, + "5976-builtin-class-aliases.ts": 5450, + "5977-builtin-class-aliases.cjs": 4300, + "5978-builtin-class-alias-modules/main.ts": 4450, "5990-worker-uncloneable.mjs": 22432, "5991-worker-async-dispose.mjs": 25768, "5992-worker-thread-name-lifetime.mjs": 28932, + "5993-array-length-loop-resizing.ts": 4127, + "5994-array-length-sparse-loops.ts": 2815, + "5995-array-dense-loop-boundaries.ts": 4171, + "5996-rejections-between-macrotasks.cjs": 22172, "600-closures-basic.ts": 2534, "601-closures-loops.ts": 2297, "602-closures-identity-recursion.ts": 2427, @@ -4645,7 +5247,23 @@ "6021-asserted-literal-union-arm.ts": 7200, "603-closures-rc-stress.ts": 2589, "604-closures-for-of.ts": 3514, + "6041-error-cause-redeclared.ts": 4631, + "6042-regex-match-object-destructuring.ts": 3091, + "6043-string-index-narrowed-union.ts": 3627, "605-closures-forward-capture-tdz.ts": 4638, + "6100-record-get-or-create.ts": 3173, + "6101-record-get-or-create-unchecked/main.ts": 3228, + "6103-typeof-builtin-bindings.cjs": 1796, + "6104-typeof-builtin-bindings.mjs": 1715, + "6105-cjs-top-level-this/main.js": 3213, + "6106-esm-top-level-this.mjs": 1672, + "6110-cjs-speculative-wrapper-fences.cjs": 2500, + "6140-cjs-require-tdz-throws/main.js": 2989, + "6141-cjs-require-tdz-checked/main.js": 4062, + "6142-cjs-require-published-early/main.js": 5156, + "6143-cjs-require-json/main.js": 3050, + "6144-cjs-require-builtin-nested/main.js": 3329, + "6145-cjs-require-cycle/main.js": 4761, "700-classes-basic.ts": 2674, "701-classes-composition.ts": 2606, "702-classes-this-capture.ts": 2403, @@ -4753,6 +5371,7 @@ "array-checked-callback-results.mjs": 3402, "array-checked-constructor.js": 3477, "array-concat-widening.ts": 3858, + "array-deque-unshift.ts": 4273, "array-element-compound-updates.ts": 3591, "array-fill-unit-values.ts": 2572, "array-from-checked.js": 3097, @@ -4761,6 +5380,7 @@ "array-numeric-own-keys.ts": 3632, "array-optional-number-write-index.ts": 3380, "array-primitive-ordering.ts": 3925, + "array-queue-head-room.ts": 5069, "array-read-helper-effects.ts": 3425, "array-refined-storage.ts": 3909, "array-sort-dense-effects.ts": 2917, @@ -4863,6 +5483,8 @@ "class-symbol-accessors.ts": 2479, "class-symbol-cache.mjs": 3721, "class-value-constructor-exceptions.ts": 3210, + "closed-virtual-dispatch-modules/main.ts": 3375, + "closed-virtual-dispatch.ts": 4025, "closure-nullable-union-return.ts": 2727, "closure-union-locals.ts": 2878, "collection-array-cycles.ts": 3240, @@ -4932,7 +5554,11 @@ "fresh-array-union-layout.js": 2679, "frozen-checked-dictionaries.mjs": 3220, "fs-native-realpath.ts": 2477, + "fs-null-byte-paths-js.cjs": 3914, + "fs-null-byte-paths.ts": 5084, "fs-read-encodings.ts": 3111, + "fs-read-utf8-edges.ts": 4161, + "fs-read-utf8-handle-forms.ts": 3562, "fs-remove-readonly.ts": 2708, "fs-write-string-bytes-union.ts": 2772, "function-apply-value.mjs": 2976, @@ -4959,6 +5585,7 @@ "generic-pattern-bindings.ts": 3518, "generic-record-index.ts": 3239, "generic-visitor-void.ts": 3402, + "global-accumulator-loops.ts": 3531, "global-error-subclasses.ts": 2754, "global-runtime-probes.js": 2804, "global-stored-probes.js": 3113, @@ -5000,6 +5627,8 @@ "js-array-iterators.js": 3215, "js-arrow-bind.cjs": 3596, "js-buffer-callback-brand.cjs": 3526, + "js-builder-string-method-names/main.ts": 5400, + "js-callable-builder.js": 10100, "js-callable-proxy.js": 3865, "js-chained-native-results.mjs": 3076, "js-checked-array-iterator.js": 2790, @@ -5011,6 +5640,7 @@ "js-class-added-parameters.js": 2522, "js-class-checked-overrides.js": 2542, "js-class-computed-fields.js": 2281, + "js-class-constructor-returns/main.ts": 5400, "js-class-enumeration.js": 3084, "js-class-field-accessor-override.js": 3865, "js-class-fluent-overrides.js": 3368, @@ -5034,6 +5664,7 @@ "js-dormant-array-write.js": 2014, "js-dormant-overrides.js": 3783, "js-expando-records.js": 2526, + "js-function-prototype-value.js": 5100, "js-implicit-call-results.js": 2892, "js-inferred-callback-exports/main.js": 3776, "js-inferred-class-array-return.js": 3477, @@ -5049,6 +5680,7 @@ "js-nested-native-descriptors.js": 2725, "js-normalize-dispatch.js": 3264, "js-nullish-inherited-fields.mjs": 3080, + "js-number-parse-dynamic.js": 4350, "js-object-receiver-storage.js": 3515, "js-object-returning-constructors.mjs": 3015, "js-object-shared-arrays.js": 2539, @@ -5057,6 +5689,7 @@ "js-optional-class-fields.js": 3252, "js-optional-member-call.js": 3095, "js-primitive-union-addition.js": 2779, + "js-process-destructure.mjs": 4600, "js-promise-all-values.js": 3443, "js-promise-catch-handler.js": 2034, "js-promise-finally-order.js": 2084, @@ -5064,6 +5697,7 @@ "js-prototype-host-boundary.js": 4779, "js-reflect-receivers.js": 3486, "js-regex-record-captures.js": 2214, + "js-reshaped-object-literal.js": 7750, "js-scalar-iterables.js": 3719, "js-shared-method-dispatch.js": 2114, "js-specialized-field-method.js": 3011, @@ -5077,6 +5711,9 @@ "json-runtime-options.mjs": 2794, "json-stringify-value.js": 2833, "json-typed-parse-direct.ts": 3580, + "json-typed-parse-index-signatures.ts": 3707, + "json-typed-parse-optional-members.ts": 4645, + "json-typed-parse-union-members.ts": 2884, "literal-switch-dispatch.ts": 2951, "live-reference-shared-converters.ts": 19276, "llvm-read-receiver-lifetime.ts": 3803, @@ -5098,6 +5735,7 @@ "loop-reference-lifetimes.ts": 3339, "map-checked-boundary/main.ts": 2843, "map-checked-seed.js": 2966, + "map-get-set-fusion.ts": 3638, "map-number-keys-dense.ts": 2773, "map-read-lifetimes-exits.ts": 2761, "map-read-lifetimes.ts": 3024, @@ -5289,6 +5927,8 @@ "promise-resolve-void.ts": 2846, "prototype-method-overrides.mjs": 4119, "prototype-named-data-records.ts": 3179, + "rc-last-use-moves.ts": 4127, + "rc-private-array-writes.ts": 4075, "record-field-presence.ts": 3698, "record-key-order.ts": 2706, "record-optional-json-presence.ts": 3216, @@ -5304,6 +5944,8 @@ "regex-checked-casts.ts": 2405, "regex-checked-storage.js": 3560, "regex-exec-long-subject.ts": 5584, + "regex-exec-result-arrays.ts": 4036, + "regex-fs-text-surfaces.ts": 4013, "regex-input-lifetimes.ts": 3241, "regex-native-matcher.ts": 3652, "regex-optional-captures.ts": 2740, @@ -5328,12 +5970,20 @@ "set-iterator-seeds.ts": 3269, "set-optional-seeds.ts": 2164, "small-collection-transitions.ts": 3586, + "speculative-integer-globals.ts": 2999, + "speculative-integer-loops.ts": 4097, + "speculative-integer-reads.ts": 3960, "stack-depth-async-bodies.ts": 2727, "stack-overflow-async.ts": 2942, "stack-overflow-caught.ts": 3056, "stack-overflow-finally.ts": 3694, "stack-overflow-shapes.ts": 3565, "stack-overflow-uncaught.ts": 2532, + "stackless-async-concurrency.ts": 5553, + "stackless-async-context.ts": 3712, + "stackless-async-exceptions.ts": 4059, + "stackless-async-ordering.ts": 4230, + "stackless-async-unhandled.ts": 3409, "static-field-updates/main.ts": 2914, "static-method-receivers.ts": 2350, "stdio-values.mjs": 2761, @@ -5341,10 +5991,15 @@ "stdio-write-overrides.mjs": 3205, "stored-native-builtins.js": 3341, "stored-object-helpers.js": 2959, + "stream-dynamic-entry-kinds.cjs": 8165, + "stream-dynamic-entry-kinds.ts": 2619, + "stream-encoded-from-entries.cjs": 4394, + "stream-encoded-from-entries.ts": 4716, "string-char-code-integer-loops.ts": 2575, "string-code-points.ts": 2460, "string-concat-number-parts.ts": 2155, "string-construction-order.ts": 3307, + "string-field-append.ts": 3228, "string-lifetimes-boundaries.ts": 3304, "string-lifetimes-evaluation.ts": 2835, "string-lifetimes-iteration.ts": 2037, @@ -5371,6 +6026,7 @@ "text-codec-var.ts": 3480, "text-encode-into-boundaries.ts": 2224, "timers-clear-many.ts": 3073, + "truthy-object-logical.ts": 4450, "tuple-and-string-search.ts": 2595, "tuple-array-union.ts": 3305, "tuple-callback-visitation.ts": 3801, diff --git a/tests/harness/coverage.test.ts b/tests/harness/coverage.test.ts index 06f0e75a8..420651418 100644 --- a/tests/harness/coverage.test.ts +++ b/tests/harness/coverage.test.ts @@ -3,7 +3,13 @@ import { globSync, mkdtempSync, readFileSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { expect, test } from "vitest"; -import { analyze, coverageEnvelope, coveragePasses, renderCoverage } from "@scriptc/compiler"; +import { + analyze, + coverageEnvelope, + coveragePasses, + renderCoverage, + renderExplainDynamic, +} from "@scriptc/compiler"; import { shardSelect, shardSuffix } from "./shard.js"; const repoRoot = join(import.meta.dirname, "../.."); @@ -230,6 +236,105 @@ test("CLI: build --print=diagnostics reports failures and divergence warnings as } }); +test("coverage counts the statements of dynamic-tier functions", async () => { + expect(analyze(join(repoRoot, "tests/corpus/400-fib.ts")).coverage.stats.statementsDynamic).toBe( + 0, + ); + const { coverage, sourceTexts } = analyze(fixture("tier-demotions.ts")); + expect(coverage.stats.statementsDynamic).toBe(1); + expect(coverage.tiers).toBeUndefined(); + expect(coverageEnvelope(coverage, { compilerVersion: "test", sourceTexts }).stats).toMatchObject({ + statementsTotal: 5, + statementsStatic: 3, + statementsDynamic: 1, + statementsFailed: 2, + }); + await expect(report(fixture("tier-demotions.ts"))).toMatchFileSnapshot( + "__snapshots__/coverage-tier-demotions.txt", + ); +}); + +test("--explain-dynamic attributes each refusal to the unit the dynamic tier would take", async () => { + const { coverage, sourceTexts } = analyze(fixture("tier-demotions.ts"), { explainDynamic: true }); + const tiers = coverage.tiers!; + expect(tiers.mode).toBe("auto"); + expect(tiers.statementsDynamic).toBe(1); + expect(tiers.dynamicFunctions.map((fn) => fn.name)).toContain("readName"); + const requests = [...tiers.requests].sort((a, b) => a.loc.start - b.loc.start); + expect( + requests.map((r) => [r.code, r.seed, r.scope, r.unit.kind, r.unit.name, r.enclosing.name]), + ).toEqual([ + ["SC1090", "reflection", "reached", "class", "Point", ""], + ["SC2012", "operation", "reached", "binding", "n", "precise"], + ]); + expect(tiers.units.map((u) => [u.kind, u.name, u.requests, u.sites])).toEqual([ + ["class", "Point", 1, 0], + ["function", "", 0, 1], + ["function", "precise", 0, 1], + ["binding", "n", 1, 0], + ]); + const envelope = coverageEnvelope(coverage, { compilerVersion: "test", sourceTexts }); + expect(envelope.tiers?.requests[0]).toMatchObject({ unit: { kind: expect.any(String) } }); + const relative = JSON.parse(JSON.stringify(tiers).replaceAll(repoRoot + "/", "")) as typeof tiers; + await expect(renderExplainDynamic(relative)).toMatchFileSnapshot( + "__snapshots__/coverage-explain-dynamic.txt", + ); + expect(renderExplainDynamic(relative, { query: "Point" })).toContain("class Point"); + expect(renderExplainDynamic(relative, { query: "Point" })).not.toContain("binding n"); + expect(renderExplainDynamic(relative, { query: "nothing" })).toContain( + "no demoted unit matches 'nothing'", + ); +}); + +test("CLI: --tier and --explain-dynamic", () => { + const scriptcCli = join(repoRoot, "packages/cli/src/main.ts"); + const outDir = mkdtempSync(join(tmpdir(), "scriptc-tier-")); + const run = (...args: string[]) => + spawnSync(process.execPath, ["--import", "tsx", scriptcCli, ...args], { + cwd: repoRoot, + encoding: "utf8", + env: { ...process.env, SCRIPTC_NO_CACHE: "1" }, + }); + try { + const explained = run( + "coverage", + fixture("tier-demotions.ts"), + "--explain-dynamic", + "--tier=static-only", + ); + expect(explained.status).toBe(0); + expect(explained.stdout).toContain("in dynamic tier 1"); + expect(explained.stdout).toContain("--tier=static-only"); + expect(explained.stdout).toContain("binding n in precise"); + const one = run("coverage", fixture("tier-demotions.ts"), "--explain-dynamic=Point"); + expect(one.stdout).toContain("class Point"); + expect(one.stdout).not.toContain("binding n in precise"); + const bad = run("coverage", fixture("tier-demotions.ts"), "--tier=dynamic-only"); + expect(bad.status).toBe(1); + expect(bad.stderr).toContain('unknown --tier value "dynamic-only"'); + const misplaced = run("build", fixture("tier-demotions.ts"), "--explain-dynamic"); + expect(misplaced.status).toBe(1); + expect(misplaced.stderr).toContain("--explain-dynamic is a coverage option"); + // Until demotion lands, static-only compiles exactly like the default. + const ir = (name: string, ...args: string[]) => { + const out = join(outDir, name); + const built = run( + "build", + join(repoRoot, "tests/corpus/400-fib.ts"), + "--emit=ir", + "-o", + out, + ...args, + ); + expect(built.status).toBe(0); + return readFileSync(out, "utf8"); + }; + expect(ir("static-only.ir.json", "--tier=static-only")).toBe(ir("auto.ir.json")); + } finally { + rmSync(outDir, { recursive: true, force: true }); + } +}); + test("fully static program reports 100%", () => { const out = report(join(repoRoot, "tests/corpus/400-fib.ts")); expect(out).toContain("(100%)"); diff --git a/tests/harness/errors.test.ts b/tests/harness/errors.test.ts index 28daa88d3..51f04d310 100644 --- a/tests/harness/errors.test.ts +++ b/tests/harness/errors.test.ts @@ -960,3 +960,56 @@ for (const input of [ciphertext, Buffer.from("forged data"), Buffer.alloc(0)]) { expect(r.stderr).toBe(""); }); }); + +// Function expressions the frontend lowers speculatively (the receiver of +// `.call`/`.apply`, a computed class base) keep their refused statements as +// runtime fences. They used to vanish: the capture buffer held the refusal, +// the candidate was kept, and the statement was simply missing from it. +describe("refused statements in speculatively lowered JavaScript functions", () => { + test.each([ + [ + "speculative-call-this", + `(function () { + var root = this; + console.log("before"); + console.log(root === module.exports); + console.log("after"); +}).call(this); +`, + "before\n", + 4, + ], + [ + "speculative-apply", + `(function () { + console.log("before"); + console.log(typeof module.exports); +}).apply(null, []); +`, + "before\n", + 3, + ], + [ + "speculative-class-base", + `class Base {} +class Derived extends (function () { + console.log("before"); + console.log(typeof module.exports); + return Base; +})() {} +console.log(typeof Derived); +`, + "before\n", + 4, + ], + ] as const)("%s", async (name, source, stdout, line) => { + const r = await compileAndRun(name, source, "cjs"); + expect(r.exitCode).toBe(1); + expect(r.stdout).toBe(stdout); + expect(r.stderr).toMatch( + new RegExp( + `^Uncaught Error: 'module\\.exports' is part of the standard library types but has no scriptc lowering yet \\[SC2020 at .*${name}\\.cjs:${line}\\]\\n$`, + ), + ); + }); +}); diff --git a/tests/harness/npm-static.test.ts b/tests/harness/npm-static.test.ts index c5021fd06..ba716076c 100644 --- a/tests/harness/npm-static.test.ts +++ b/tests/harness/npm-static.test.ts @@ -821,10 +821,71 @@ describe(`npm-static pilots${sanitize ? " (sanitized)" : ""}`, () => { package: "ms", status: "refused", detail: "it ships no own .d.ts declaration surface", + hint: "to compile it anyway, name it with --npm-static=ms (or --npm-static=ms... to include its dependencies)", }, ]); }, 120_000); + // Types-first resolution answers @types/twinned, but the refusal names the + // package that has the code and the flag that compiles it. + test("an @types-typed package is refused under its runtime name with the opt-in remedy", () => { + const { coverage } = analyze(join(pilotRoot, "twinned-cli.ts")); + expect(coverage.npmStatic).toEqual([ + { + package: "twinned", + status: "refused", + detail: + "its declared types come from the separate @types/twinned package, not the package itself", + hint: "to compile it anyway, name it with --npm-static=twinned (or --npm-static=twinned... to include its dependencies)", + }, + ]); + const diagnostic = coverage.diagnostics.find((d) => d.code === "SC2013"); + expect(diagnostic?.message).toContain("'twinned'"); + expect(diagnostic?.hint).toContain("--npm-static=twinned"); + }, 120_000); + + test("a named package's untyped dependency points at the transitive opt-in", () => { + const { coverage } = analyze(join(pilotRoot, "twinned-cli.ts"), { npmStatic: ["twinned"] }); + expect(coverage.npmStatic).toContainEqual({ + package: "ms", + status: "refused", + detail: "it ships no own .d.ts declaration surface", + hint: "to compile it anyway, name it with --npm-static=ms, or use --npm-static=twinned... to include all of twinned's dependencies", + }); + }, 120_000); + + test("--npm-static=pkg... compiles the package's dependency closure", async () => { + const entry = join(pilotRoot, "twinned-cli.ts"); + const { coverage } = analyze(entry, { npmStatic: ["twinned..."] }); + expect(coverage.npmStatic).toEqual([ + { package: "twinned", status: "static" }, + { package: "ms", status: "static" }, + ]); + expect(coverage.preflightFailed).toBe(false); + const binary = await buildStatic(entry, ["twinned..."]); + const [nodeRes, nativeRes] = await Promise.all([ + runBinary("node", [entry]), + runBinary(binary, []), + ]); + expect(nativeRes.stdout.toString("utf8")).toBe(nodeRes.stdout.toString("utf8")); + expect(nativeRes.exitCode).toBe(nodeRes.exitCode); + }, 180_000); + + test("type-only imports select no package", async () => { + const entry = join(pilotRoot, "type-only-cli.ts"); + const { coverage } = analyze(entry); + expect(coverage.npmStatic ?? []).toEqual([]); + expect(coverage.preflightFailed).toBe(false); + expect(coverage.diagnostics).toEqual([]); + const binary = await buildStatic(entry, []); + const [nodeRes, nativeRes] = await Promise.all([ + runBinary("node", [entry]), + runBinary(binary, []), + ]); + expect(nativeRes.stdout.toString("utf8")).toBe(nodeRes.stdout.toString("utf8")); + expect(nativeRes.exitCode).toBe(nodeRes.exitCode); + }, 180_000); + test("ms supports invalid duration strings without runtime fences", () => { const { coverage } = analyze(join(pilotRoot, "ms-invalid-cli.ts"), { npmStatic: ["ms"] }); expect(coverage.npmStatic).toEqual([{ package: "ms", status: "static" }]); @@ -974,6 +1035,7 @@ describe(`npm-static pilots${sanitize ? " (sanitized)" : ""}`, () => { package: "typegapped", status: "refused", detail: "it ships no own .d.ts declaration surface", + hint: "to compile it anyway, name it with --npm-static=typegapped (or --npm-static=typegapped... to include its dependencies)", }, { package: "escape-string-regexp", status: "static" }, ]); diff --git a/tests/harness/self-hosting-range-cases.ts b/tests/harness/self-hosting-range-cases.ts index 05d6a1fe6..ed2b24ca9 100644 --- a/tests/harness/self-hosting-range-cases.ts +++ b/tests/harness/self-hosting-range-cases.ts @@ -375,7 +375,8 @@ export function integerRangeCases(): RangeCase[] { ], }, ], - [proof(cursor, 0, 80), proof(decoded, 0, 4294967295), proof(head, 0, 124)], + // Passes enter at 0, 24, 48, 72 and 96 (at most 99), so the failing test reads at most 123. + [proof(cursor, 0, 80), proof(decoded, 0, 4294967295), proof(head, 0, 123)], (func) => { for (const id of ["cursor", "decoded"]) func.locals.push({ id, name: id, type: F64, mutable: true }); diff --git a/tests/harness/zero-cost-gate.test.ts b/tests/harness/zero-cost-gate.test.ts new file mode 100644 index 000000000..0b08da748 --- /dev/null +++ b/tests/harness/zero-cost-gate.test.ts @@ -0,0 +1,160 @@ +import { spawnSync } from "node:child_process"; +import { expect, test } from "vitest"; +import { + cachegrindCounts, + compareArtifact, + compareCounts, + compareExecutables, + dynamicTierSymbols, + nmOutput, + programVerdict, +} from "../../scripts/zero-cost-compare.mjs"; + +const ir = (functions: object[], globals: object[]) => + Buffer.from(JSON.stringify({ irVersion: 16, functions, globals, sourceFile: "/p/main.ts" })); + +const llvm = (body: string, extra = "") => + Buffer.from( + [ + "; Generated by scriptc (LLVM backend) from /p/main.ts. Do not edit.", + '@.str = private constant [3 x i8] c"hi\\00"', + "define internal i64 @add(i64 %a, i64 %b) {", + ` ${body}`, + "}", + 'define void @"main.$1"() { ret void }', + extra, + "declare void @scr_print(ptr)", + "", + ].join("\n"), + ); + +test("the zero-cost gate names the first differing IR and LLVM unit", () => { + const f = { name: "f", params: [], body: [{ kind: "return" }] }; + const g = { name: "g", params: [], body: [] }; + const base = ir([f, g, f], [{ name: "a", type: "i64" }]); + expect(compareArtifact("ir", base, Buffer.from(base))).toEqual({ identical: true }); + + const changed = compareArtifact( + "ir", + base, + ir([f, { ...g, params: [1] }, f], [{ name: "a", type: "i64" }]), + ); + expect(changed).toMatchObject({ + identical: false, + first: "changed g", + changed: ["g"], + added: [], + module: null, + }); + + const header = compareArtifact("ir", base, ir([f, g, f], [{ name: "a", type: "f64" }])); + expect(header).toMatchObject({ first: "module globals[0:a]", changed: [] }); + // Repeated names are matched by occurrence, so an insertion shows as added. + expect(compareArtifact("ir", base, ir([f, g, f, f], [{ name: "a", type: "i64" }]))).toMatchObject( + { + first: "added f#3", + }, + ); + + const ll = llvm("%r = add i64 %a, %b\n ret i64 %r"); + expect(compareArtifact("llvm", ll, Buffer.from(ll))).toEqual({ identical: true }); + expect(compareArtifact("llvm", ll, llvm("%r = sub i64 %a, %b\n ret i64 %r"))).toMatchObject({ + first: "changed add", + changed: ["add"], + functions: { baseline: 2, candidate: 2 }, + }); + expect( + compareArtifact( + "llvm", + ll, + llvm("%r = add i64 %a, %b\n ret i64 %r", "define void @new() { ret void }"), + ), + ).toMatchObject({ first: "added new", added: ["new"] }); + const global = compareArtifact("llvm", ll, Buffer.from(ll.toString().replace("hi", "ho"))); + expect(global).toMatchObject({ first: "module line 2", changed: [] }); + expect(global.module.candidate).toContain("ho"); +}); + +test("the zero-cost gate compares executables and instruction counts per function", () => { + const nm = + "0000000000001000 0000000000000010 T main\n0000000000001010 0000000000000020 t scr_add\n"; + const resized = compareExecutables( + Buffer.from("abcd"), + Buffer.from("abXd"), + nm, + nm.replace("0000000000000020 t", "0000000000000030 t"), + ); + expect(resized).toMatchObject({ identical: false, first: "resized scr_add", offset: 2 }); + + const output = (main: number, prefault: number, loader: number) => + [ + "events: Ir", + "fl=???", + "fn=main", + `0 ${main}`, + "fn=scr_sa_prefault", + `0 ${prefault}`, + "fn=_dl_lookup_symbol_x", + `0 ${loader}`, + "fn=scr_add", + "0 7", + "0 3", + ].join("\n"); + const own = new Set(["main", "scr_add", "scr_sa_prefault"]); + const base = cachegrindCounts(output(100, 50, 9000), own); + expect([...base]).toEqual([ + ["main", 100], + ["scr_add", 10], + ]); + expect(compareCounts(base, cachegrindCounts(output(100, 75, 9300), own)).identical).toBe(true); + expect(compareCounts(base, cachegrindCounts(output(104, 50, 9000), own))).toMatchObject({ + identical: false, + total: { baseline: 110, candidate: 114 }, + deltas: [{ name: "main", delta: 4 }], + }); +}); + +test("the zero-cost gate throws when nm does not list a binary's symbols", () => { + const ok = { status: 0, signal: null, stdout: "0000000000001000 T main\n", stderr: "" }; + expect(nmOutput("prog", ok)).toBe(ok.stdout); + const missing = spawnSync("scriptc-missing-nm", ["prog"], { encoding: "utf8" }); + expect(() => nmOutput("prog", missing)).toThrow(/^nm prog: .*ENOENT/); + expect(() => + nmOutput("prog", { status: null, signal: "SIGKILL", stdout: "", stderr: "" }), + ).toThrow("nm prog: killed by SIGKILL"); + expect(() => + nmOutput("prog", { status: 1, signal: null, stdout: "", stderr: "nm: prog: no symbols\n" }), + ).toThrow("nm prog: exited with status 1: nm: prog: no symbols"); +}); + +test("the zero-cost gate recognizes dynamic-tier runtime symbols", () => { + expect( + dynamicTierSymbols([ + "main", + "scr_dyn_release", + "scr_dyn_obj_drop_keys", + "scr_dyn_this_push_dyn", + "scr_js_load_ic", + "scr_dyn_obj_read", + "scr_dyn_obj_read", + ]), + ).toEqual(["scr_dyn_obj_read", "scr_js_load_ic"]); +}); + +test("the zero-cost gate fails differing programs unless they match the signature", () => { + const ok = { ok: true }; + const same = { ir: { identical: true }, exe: { identical: true } }; + const differs = { ir: { identical: true }, exe: { identical: false } }; + const verdict = ( + artifacts: object, + allowed = false, + baseline: object = ok, + candidate: object = ok, + ) => programVerdict({ baseline, candidate, artifacts, allowed }); + expect(verdict(same)).toEqual({ verdict: "identical", fails: false }); + expect(verdict(differs)).toEqual({ verdict: "differs", fails: true }); + expect(verdict(differs, true)).toEqual({ verdict: "allowed", fails: false }); + const failed = { ok: false, error: "SC1090" }; + expect(verdict({}, false, failed, failed)).toEqual({ verdict: "both-fail", fails: false }); + expect(verdict({}, true, ok, failed)).toEqual({ verdict: "build-error", fails: true }); +});