Merge the Node island runtime branch and stamp the node packs with 0.4.3

This commit is contained in:
Chris Tate
2026-10-23 15:32:55 -05:00
228 changed files with 9198 additions and 919 deletions
+1 -1
View File
@@ -532,7 +532,7 @@ jobs:
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.linux == 'true' }}
name: test (tsgo and tsc-ts give the same IR)
runs-on: ubuntu-24.04
timeout-minutes: 30
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
+22 -2
View File
@@ -6,6 +6,28 @@ All notable changes to scriptc will be documented in this file.
<!-- release:start -->
## 0.4.3
### Highlights
- **Faster checking and smaller builds with tsc-ts 0.0.9:** scriptc checks with tsc-ts 0.0.9, and builds reuse the bundled declaration files' diagnostics instead of checking them again. Small IR builds are 10.6 to 16.0% faster and small executable builds 5.5 to 7.6% faster, with identical output. A native build of tsc-ts with this release checks 0.4 to 6.6% faster than with 0.4.2 and peaks 4 to 16% lower.
- **Integer arrays and loops:**
- Number arrays that only hold int32 values use 4-byte slots in locals, across direct calls and in module globals, and loop-carried locals written from their elements stay int32. A workload that walks arrays of cursors runs 54.5% faster.
- Loops with two counters closing in on each other (`for (let i = lo, j = hi; i < j; i++, j--)`) and element copies through `a[i]!` run without per-element checks, and two-counter `while` loops run on integer counters.
- Loops over typed arrays and charAt scans are versioned on their bounds, and ASCII strings are scanned as bytes.
- **Less work for untyped and dynamic code:**
- String-keyed record reads under `??` and `!` no longer allocate: a dictionary-walking workload makes no union boxes instead of 363,359 and runs 12 to 15% faster.
- Checked-dynamic method calls dispatch on method ids, observed class methods call natively while their slot is unchanged, and calls on bag-tracked class instances skip building their views.
- **Node.js parity:**
- `crypto.getCipherInfo`, AES-CCM, AES key wrap, the legacy `createCipheriv` modes, `checkPrime`/`generatePrime`, and Node's argument checks for `generateKeyPair`.
- Streams: `construct`, `writev`, `decodeStrings` and `defaultEncoding`, including `writev` on `Transform` and `PassThrough`.
- `FinalizationRegistry`, `new.target`, `String.prototype.at` on untyped values, `Error.stackTraceLimit`, `spawnSync`'s `env` option, and class constructors called through `call`/`apply` throw Node's TypeError.
- Records enumerate in the key order their object literal wrote, and objects converting to a union with an empty-record arm keep every field.
- **npm packages:** zod 3 compiles natively and zod 4's core runs, minified packages such as marked compile, chalk 5 and hot-shots 8 build from built-in shims, and `@hono/node-server` 2.x is built in. User code types npm package declarations at a checked, identity-preserving boundary.
- **Builds:** the cached build is reused after whitespace and comment edits, function bodies are emitted on forked workers, and the LLVM emitter reads its analyses from one recorded walk of each module.
<!-- release:end -->
## 0.4.2
### Highlights
@@ -159,8 +181,6 @@ All notable changes to scriptc will be documented in this file.
- **The same program compiles to the same executable wherever it sits on disk** and whatever order its functions lower in, and edits outside a declaration do not rename the helpers and closures inside it.
- **Libraries that use untyped byte arrays** link.
<!-- release:end -->
## 0.4.1
### Highlights
@@ -0,0 +1,6 @@
{
"entry": "workloads/index-cursors.ts",
"args": [
"20000"
]
}
@@ -0,0 +1,25 @@
// Arrays of indices walked by loop-carried cursors: each step reads the
// next index from the array and relinks the visited entries, so the
// cursor and every stored value are small integers read back from the
// array itself.
const N = 4096;
function relink(rounds: number): number {
const next: number[] = [];
for (let i = 0; i < N; i++) next.push(i + 1 < N ? i + 1 : 0);
let at = next[0]!;
let total = 0;
for (let r = 0; r < rounds; r++) {
for (let s = 0; s < N; s++) {
const t = next[at]!;
next[at] = next[t]!;
next[t] = at;
at = t;
total = (total + at) | 0;
}
}
return total;
}
const rounds = Number(process.argv[2] ?? "1");
console.log("relink", relink(rounds));
@@ -27,7 +27,8 @@
A `Duplex.from` body function may take the `{ signal }` argument, which is aborted with the error the Duplex is destroyed with.
`stream.compose` accepts streams and async generator functions in any position, an async function as the last stage, and an array, a string, or a generator as the first stage; one stage is `Duplex.from` of it.
A web `ReadableStream` is read as an async iterable by `Readable.from` and as the first stage of `compose` or `pipeline`, and through its reader by `Readable.fromWeb` (with `objectMode`, `highWaterMark`, `encoding`, and `signal` in an options object literal) and `Duplex.from`.
A web stream in a later compose stage, `Readable.toWeb`, and web `WritableStream` and `TransformStream` interop are unsupported.
`new TransformStream()` without arguments builds the identity transform: its readable side is a web `ReadableStream`, its writable side supports `getWriter`, `abort`, `close`, and `locked`, and the writer supports `write`, `close`, `abort`, `releaseLock`, `ready`, `closed`, and `desiredSize`, with Node's backpressure (a write completes when the readable side reads its chunk).
A web stream in a later compose stage, `Readable.toWeb`, a `TransformStream` with a transformer or queuing strategies, the `WritableStream` constructor, `pipeTo`, and `pipeThrough` are unsupported.
The `ERR_INVALID_ARG_VALUE` message for a stream that is no longer readable or writable names the stream's class instead of printing the stream.
`readable.compose(stage, { signal })` is `stream.compose(readable, stage)` with the signal added; the options are an object literal whose `signal` is a variable or a property read.
`pipeline` from `node:stream/promises` accepts a trailing `{ signal, end }` object, with or without function stages, and `finished` accepts `readable`, `writable`, `error`, `cleanup`, and `signal` options, with boolean literals for the flags.
+2 -1
View File
@@ -10,9 +10,10 @@
"build": "fumadocs-mdx && next build",
"start": "next start",
"typecheck": "fumadocs-mdx && next typegen && tsc --noEmit",
"test:unit": "node --test test/compatibility-matrix.test.mjs",
"test:routes": "node scripts/test-routes.mjs",
"compat:check": "pnpm --dir .. node-compat:check",
"check": "pnpm compat:check && pnpm typecheck && pnpm build && pnpm test:routes"
"check": "pnpm compat:check && pnpm typecheck && pnpm test:unit && pnpm build && pnpm test:routes"
},
"dependencies": {
"@vercel/geistdocs": "2.7.8",
+157 -198
View File
@@ -4,52 +4,28 @@
// empty after an edit.
// @refresh reset
import { useEffect, useMemo, useState } from "react";
import { memo, startTransition, useCallback, useDeferredValue, useEffect, useId, useMemo, useState, type FocusEvent, type KeyboardEvent, type ReactNode } from "react";
import {
buildTree,
expandableIds,
filterTree,
flattenTree,
nodeCount,
rowMatcher,
searchTexts,
statusCounts,
statuses,
type NodeStability,
type Row,
type Snapshot,
type Status,
type Verification,
} from "@/lib/compatibility-tree";
// Set by next.config.mjs from the committed artifact.
const artifactVersion = process.env.COMPATIBILITY_ARTIFACT_VERSION;
const rowCount = Number(process.env.COMPATIBILITY_ROW_COUNT);
type Status =
| "supported"
| "partial"
| "refused"
| "not-implemented"
| "by-design"
| "unreviewed"
| "not-applicable";
type VerificationBasis =
| "test-backed"
| "explicit-refusal"
| "verified-absence"
| "declared-gap"
| "registry-gap"
| "architectural-policy"
| "unreviewed"
| "not-applicable"
| "derived";
type Verification = { label: string; detail: string };
type Tier = { status: Status; detail: string; verification: VerificationBasis };
type NodeStability = { index: string; level: number; text: string; inherited: boolean } | null;
type Row = {
id: string;
chapter: string;
kind: string;
name: string;
signature: string;
label: string;
apiSymbol: string;
depth: number;
scope: "api" | "documentation" | "metadata" | "configuration";
anchor: string;
nodeStability: NodeStability;
static: Tier;
};
type Chapter = { slug: string; title: string; entries: number; apiEntries: number };
type Snapshot = { nodeVersion: string; verificationBases: Record<VerificationBasis, Verification>; chapters: Chapter[]; rows: Row[] };
type TreeNode = { row: Row; children: TreeNode[] };
type VisibleNode = { node: TreeNode; level: number };
const labels: Record<Status, string> = {
supported: "Supported",
partial: "Partial",
@@ -70,7 +46,6 @@ const styles: Record<Status, string> = {
"not-applicable": "border-gray-alpha-400 bg-background-200 text-gray-700",
};
const statuses = ["supported", "partial", "not-implemented", "refused", "by-design", "unreviewed", "not-applicable"] as const;
const stabilityStyles: Record<number, string> = {
0: "border-red-500/40 bg-red-100 text-red-900 dark:bg-red-200",
1: "border-amber-500/50 bg-amber-100 text-amber-900 dark:bg-amber-200",
@@ -78,114 +53,83 @@ const stabilityStyles: Record<number, string> = {
3: "border-gray-alpha-500 bg-gray-alpha-100 text-gray-900",
};
function StatusBadge({ status, detail, verification }: { status: Status; detail?: string; verification?: Verification }) {
const tooltip = verification ? `${detail ?? ""}${detail ? "\n\n" : ""}Verification: ${verification.label}. ${verification.detail}` : detail;
/** A badge's tooltip, mounted only while the badge is hovered or has
* keyboard focus (the :focus-visible rule the CSS-only tooltip used), and
* named by aria-describedby while open. Hover and focus are tracked apart,
* so moving the pointer away keeps a keyboard-focused badge's tooltip.
* Escape dismisses it until the next hover or focus. */
function BadgeTooltip({ tooltipClass, children, badge }: { tooltipClass: string; children: ReactNode; badge: ReactNode }) {
const [hovered, setHovered] = useState(false);
const [focused, setFocused] = useState(false);
const open = hovered || focused;
const id = useId();
const enter = useCallback(() => setHovered(true), []);
const leave = useCallback(() => setHovered(false), []);
const focus = useCallback((event: FocusEvent<HTMLSpanElement>) => {
if (event.currentTarget.matches(":focus-visible")) setFocused(true);
}, []);
const blur = useCallback(() => setFocused(false), []);
const key = useCallback((event: KeyboardEvent<HTMLSpanElement>) => {
if (event.key !== "Escape") return;
setHovered(false);
setFocused(false);
}, []);
return (
<span className="group relative inline-flex" tabIndex={tooltip ? 0 : undefined}>
<span
className={`inline-flex whitespace-nowrap rounded-full border px-2 py-0.5 text-[11px] font-medium ${styles[status]}`}
title={tooltip}
>
{labels[status]}
</span>
{tooltip ? (
<span role="tooltip" className="pointer-events-none invisible absolute left-0 top-full z-30 mt-2 w-64 rounded-md border border-gray-alpha-500 bg-background-100 p-2 text-left text-xs font-normal leading-5 text-gray-1000 opacity-0 shadow-popover transition-opacity group-hover:visible group-hover:opacity-100 group-focus-visible:visible group-focus-visible:opacity-100">
{detail ? <span className="block">{detail}</span> : null}
{verification ? <span className={detail ? "mt-2 block border-t border-gray-alpha-400 pt-2" : "block"}><strong>Verification:</strong> {verification.label}. {verification.detail}</span> : null}
<span
className="relative inline-flex"
tabIndex={0}
aria-describedby={open ? id : undefined}
onMouseEnter={enter}
onMouseLeave={leave}
onFocus={focus}
onBlur={blur}
onKeyDown={key}
>
{badge}
{open ? (
<span id={id} role="tooltip" className={`pointer-events-none absolute top-full z-30 mt-2 rounded-md border border-gray-alpha-500 bg-background-100 p-2 text-left text-xs font-normal leading-5 text-gray-1000 opacity-100 shadow-popover transition-opacity starting:opacity-0 ${tooltipClass}`}>
{children}
</span>
) : null}
</span>
);
}
function StabilityBadge({ stability }: { stability: NodeStability }) {
export const StatusBadge = memo(function StatusBadge({ status, detail, verification }: { status: Status; detail?: string; verification?: Verification }) {
const tooltip = verification ? `${detail ?? ""}${detail ? "\n\n" : ""}Verification: ${verification.label}. ${verification.detail}` : detail;
const badge = (
<span
className={`inline-flex whitespace-nowrap rounded-full border px-2 py-0.5 text-[11px] font-medium ${styles[status]}`}
title={tooltip}
>
{labels[status]}
</span>
);
if (!tooltip) return <span className="relative inline-flex">{badge}</span>;
return (
<BadgeTooltip tooltipClass="left-0 w-64" badge={badge}>
{detail ? <span className="block">{detail}</span> : null}
{verification ? <span className={detail ? "mt-2 block border-t border-gray-alpha-400 pt-2" : "block"}><strong>Verification:</strong> {verification.label}. {verification.detail}</span> : null}
</BadgeTooltip>
);
});
export const StabilityBadge = memo(function StabilityBadge({ stability }: { stability: NodeStability }) {
if (stability === null) return <span className="text-xs text-gray-700" title="Node.js does not publish a stability index for this entry.">—</span>;
const detail = `Node.js Stability ${stability.index}: ${stability.text}${stability.inherited ? " (inherited from the nearest parent section)" : ""}`;
return (
<span className="group relative inline-flex" tabIndex={0}>
<span className={`inline-flex min-w-7 justify-center rounded-full border px-2 py-0.5 text-[11px] font-medium ${stabilityStyles[stability.level] ?? stabilityStyles[3]}`} title={detail}>
{stability.index}
</span>
<span role="tooltip" className="pointer-events-none invisible absolute left-1/2 top-full z-30 mt-2 w-72 -translate-x-1/2 rounded-md border border-gray-alpha-500 bg-background-100 p-2 text-left text-xs font-normal leading-5 text-gray-1000 opacity-0 shadow-popover transition-opacity group-hover:visible group-hover:opacity-100 group-focus-visible:visible group-focus-visible:opacity-100">
{detail}
</span>
</span>
);
}
function statusCounts(rows: Row[], tier: "static") {
return rows.reduce<Record<Status, number>>(
(counts, row) => {
if (row.scope !== "api" || row.depth === 0) return counts;
counts[row[tier].status] += 1;
return counts;
},
{
supported: 0,
partial: 0,
refused: 0,
"not-implemented": 0,
"by-design": 0,
unreviewed: 0,
"not-applicable": 0,
},
);
}
/** Node's all.json is preorder with an explicit heading depth. */
function buildTree(rows: Row[]): TreeNode[] {
const roots: TreeNode[] = [];
const stack: TreeNode[] = [];
for (const row of rows) {
const node: TreeNode = { row, children: [] };
while (stack.length > row.depth) stack.pop();
const parent = stack[row.depth - 1];
if (row.depth === 0 || parent === undefined) roots.push(node);
else parent.children.push(node);
stack[row.depth] = node;
stack.length = row.depth + 1;
}
return roots;
}
function filterTree(nodes: TreeNode[], matches: (row: Row) => boolean): TreeNode[] {
const filtered: TreeNode[] = [];
for (const node of nodes) {
const children = filterTree(node.children, matches);
if (matches(node.row) || children.length > 0) filtered.push({ row: node.row, children });
}
return filtered;
}
function flattenTree(nodes: TreeNode[], expanded: Set<string>, forceExpanded: boolean): VisibleNode[] {
const visible: VisibleNode[] = [];
const visit = (items: TreeNode[], level: number) => {
for (const node of items) {
visible.push({ node, level });
if (node.children.length > 0 && (forceExpanded || expanded.has(node.row.id))) {
visit(node.children, level + 1);
<BadgeTooltip
tooltipClass="left-1/2 w-72 -translate-x-1/2"
badge={
<span className={`inline-flex min-w-7 justify-center rounded-full border px-2 py-0.5 text-[11px] font-medium ${stabilityStyles[stability.level] ?? stabilityStyles[3]}`} title={detail}>
{stability.index}
</span>
}
}
};
visit(nodes, 1);
return visible;
}
function nodeCount(nodes: TreeNode[]): number {
return nodes.reduce((count, node) => count + 1 + nodeCount(node.children), 0);
}
function expandableIds(nodes: TreeNode[]): Set<string> {
const ids = new Set<string>();
const visit = (items: TreeNode[]) => {
for (const node of items) {
if (node.children.length > 0) ids.add(node.row.id);
visit(node.children);
}
};
visit(nodes);
return ids;
}
>
{detail}
</BadgeTooltip>
);
});
function Chevron({ open }: { open: boolean }) {
return (
@@ -195,6 +139,47 @@ function Chevron({ open }: { open: boolean }) {
);
}
type MatrixRowProps = {
row: Row;
level: number;
hasChildren: boolean;
open: boolean;
filtering: boolean;
displayName: string;
href: string;
verification: Verification | undefined;
onToggle: (id: string) => void;
};
/** One table row. Memoized on its own values, so a filter or expansion
* change re-renders only the rows whose values changed. */
const MatrixRow = memo(function MatrixRow({ row, level, hasChildren, open, filtering, displayName, href, verification, onToggle }: MatrixRowProps) {
const isRoot = level === 1;
return (
<tr aria-level={level} aria-expanded={hasChildren ? open : undefined} className={`align-top hover:bg-gray-alpha-100/60 ${isRoot ? "bg-background-200/60" : ""}`}>
<td>
<div className="flex items-start gap-1.5" style={{ paddingLeft: `${(level - 1) * 18}px` }}>
{hasChildren ? (
<button type="button" onClick={() => onToggle(row.id)} disabled={filtering} aria-label={`${open ? "Collapse" : "Expand"} ${displayName}`} className="mt-0.5 flex h-5 w-5 shrink-0 items-center justify-center rounded text-gray-800 hover:bg-gray-alpha-200 hover:text-gray-1000 disabled:cursor-default disabled:hover:bg-transparent">
<Chevron open={open} />
</button>
) : <span className="h-5 w-5 shrink-0" />}
<div className="min-w-0">
<a href={href} target="_blank" rel="noopener noreferrer" className={`${isRoot ? "font-sans text-sm font-medium" : "font-mono text-xs"} text-gray-1000 underline decoration-gray-alpha-500 underline-offset-2 hover:decoration-gray-1000`}>
{displayName}
</a>
<div className="mt-1 text-[11px] text-gray-800">
{row.kind}{row.apiSymbol && row.apiSymbol !== row.signature && !isRoot ? ` · ${row.apiSymbol}` : ""}
</div>
</div>
</div>
</td>
<td className="text-center"><StabilityBadge stability={row.nodeStability} /></td>
<td><StatusBadge status={row.static.status} detail={row.static.detail} verification={verification} /></td>
</tr>
);
});
export function NodeCompatibilityMatrix() {
const [snapshot, setSnapshot] = useState<Snapshot | null>(null);
const [loadError, setLoadError] = useState(false);
@@ -202,15 +187,16 @@ export function NodeCompatibilityMatrix() {
const [staticFilter, setStaticFilter] = useState<"all" | Status>("all");
const [expanded, setExpanded] = useState<Set<string>>(() => new Set());
const rows = snapshot?.rows ?? [];
const chapters = snapshot?.chapters ?? [];
const rows = useMemo(() => snapshot?.rows ?? [], [snapshot]);
const chapters = useMemo(() => snapshot?.chapters ?? [], [snapshot]);
const verificationBases = snapshot?.verificationBases;
const apiRowCount = useMemo(() => rows.filter((row) => row.scope === "api").length, [rows]);
const chapterTitles = useMemo(() => new Map(chapters.map((item) => [item.slug, item.title])), [chapters]);
const base = `https://nodejs.org/docs/v${snapshot?.nodeVersion ?? "24.22.0"}/api/`;
const staticCounts = useMemo(() => statusCounts(rows, "static"), [rows]);
const staticCounts = useMemo(() => statusCounts(rows), [rows]);
const visibleStatuses = useMemo(() => statuses.filter((status) => staticCounts[status] > 0), [staticCounts]);
const tree = useMemo(() => buildTree(rows), [rows]);
const texts = useMemo(() => searchTexts(rows, chapterTitles), [rows, chapterTitles]);
useEffect(() => {
const controller = new AbortController();
@@ -231,36 +217,23 @@ export function NodeCompatibilityMatrix() {
return () => controller.abort();
}, []);
const needle = query.trim().toLocaleLowerCase();
const hasResultFilter = needle !== "" || staticFilter !== "all";
const rowMatches = (row: Row) => {
if (staticFilter !== "all" && row.static.status !== staticFilter) return false;
if (!needle) return true;
return [chapterTitles.get(row.chapter), row.kind, row.name, row.signature, row.apiSymbol, row.static.detail]
.join(" ")
.toLocaleLowerCase()
.includes(needle);
};
// The search field and the status select update at once; the table
// follows the deferred values, so input stays responsive while the rows
// re-render.
const deferredQuery = useDeferredValue(query);
const deferredStatic = useDeferredValue(staticFilter);
const needle = deferredQuery.trim().toLocaleLowerCase();
const hasResultFilter = needle !== "" || deferredStatic !== "all";
const resultTree = useMemo(
() => hasResultFilter ? filterTree(tree, rowMatches) : tree,
// rowMatches closes over these filter values.
// eslint-disable-next-line react-hooks/exhaustive-deps
[hasResultFilter, needle, staticFilter, tree],
const result = useMemo(
() => hasResultFilter ? filterTree(tree, rowMatcher(texts, needle, deferredStatic)) : { nodes: tree, matched: nodeCount(tree) },
[hasResultFilter, needle, deferredStatic, texts, tree],
);
const visible = useMemo(
() => flattenTree(resultTree, expanded, hasResultFilter),
[expanded, hasResultFilter, resultTree],
);
const matchedCount = useMemo(
() => hasResultFilter ? tree.reduce((count, node) => {
const visit = (item: TreeNode): number => (rowMatches(item.row) ? 1 : 0) + item.children.reduce((sum, child) => sum + visit(child), 0);
return count + visit(node);
}, 0) : nodeCount(tree),
// rowMatches closes over these filter values.
// eslint-disable-next-line react-hooks/exhaustive-deps
[hasResultFilter, needle, staticFilter, tree],
() => flattenTree(result.nodes, expanded, hasResultFilter),
[expanded, hasResultFilter, result],
);
const matchedCount = result.matched;
const changeQuery = (value: string) => setQuery(value);
const resetFilters = () => {
@@ -268,12 +241,12 @@ export function NodeCompatibilityMatrix() {
setStaticFilter("all");
setExpanded(new Set());
};
const toggle = (id: string) => setExpanded((current) => {
const toggle = useCallback((id: string) => setExpanded((current) => {
const next = new Set(current);
if (next.has(id)) next.delete(id);
else next.add(id);
return next;
});
}), []);
useEffect(() => {
if (!snapshot) return;
@@ -316,8 +289,8 @@ export function NodeCompatibilityMatrix() {
<span aria-live="polite">{visible.length.toLocaleString()} visible · {matchedCount.toLocaleString()} {hasResultFilter ? "matching" : "API"} rows</span>
{!hasResultFilter ? (
<span className="flex gap-1">
<button type="button" onClick={() => setExpanded(expandableIds(tree))} className="rounded px-1.5 py-1 text-gray-900 hover:bg-gray-alpha-100 hover:text-gray-1000">Expand all</button>
<button type="button" onClick={() => setExpanded(new Set())} className="rounded px-1.5 py-1 text-gray-900 hover:bg-gray-alpha-100 hover:text-gray-1000">Collapse all</button>
<button type="button" onClick={() => startTransition(() => setExpanded(expandableIds(tree)))} className="rounded px-1.5 py-1 text-gray-900 hover:bg-gray-alpha-100 hover:text-gray-1000">Expand all</button>
<button type="button" onClick={() => startTransition(() => setExpanded(new Set()))} className="rounded px-1.5 py-1 text-gray-900 hover:bg-gray-alpha-100 hover:text-gray-1000">Collapse all</button>
</span>
) : null}
</div>
@@ -335,33 +308,19 @@ export function NodeCompatibilityMatrix() {
<tbody>
{visible.map(({ node, level }) => {
const row = node.row;
const hasChildren = node.children.length > 0;
const open = hasResultFilter || expanded.has(row.id);
const href = `${base}${row.chapter}.html${row.anchor ? `#${row.anchor}` : ""}`;
const isRoot = level === 1;
const displayName = isRoot ? chapterTitles.get(row.chapter) ?? row.label : row.label;
return (
<tr key={row.id} aria-level={level} aria-expanded={hasChildren ? open : undefined} className={`align-top hover:bg-gray-alpha-100/60 ${isRoot ? "bg-background-200/60" : ""}`}>
<td>
<div className="flex items-start gap-1.5" style={{ paddingLeft: `${(level - 1) * 18}px` }}>
{hasChildren ? (
<button type="button" onClick={() => toggle(row.id)} disabled={hasResultFilter} aria-label={`${open ? "Collapse" : "Expand"} ${displayName}`} className="mt-0.5 flex h-5 w-5 shrink-0 items-center justify-center rounded text-gray-800 hover:bg-gray-alpha-200 hover:text-gray-1000 disabled:cursor-default disabled:hover:bg-transparent">
<Chevron open={open} />
</button>
) : <span className="h-5 w-5 shrink-0" />}
<div className="min-w-0">
<a href={href} target="_blank" rel="noopener noreferrer" className={`${isRoot ? "font-sans text-sm font-medium" : "font-mono text-xs"} text-gray-1000 underline decoration-gray-alpha-500 underline-offset-2 hover:decoration-gray-1000`}>
{displayName}
</a>
<div className="mt-1 text-[11px] text-gray-800">
{row.kind}{row.apiSymbol && row.apiSymbol !== row.signature && !isRoot ? ` · ${row.apiSymbol}` : ""}
</div>
</div>
</div>
</td>
<td className="text-center"><StabilityBadge stability={row.nodeStability} /></td>
<td><StatusBadge status={row.static.status} detail={row.static.detail} verification={verificationBases?.[row.static.verification]} /></td>
</tr>
<MatrixRow
key={row.id}
row={row}
level={level}
hasChildren={node.children.length > 0}
open={hasResultFilter || expanded.has(row.id)}
filtering={hasResultFilter}
displayName={level === 1 ? chapterTitles.get(row.chapter) ?? row.label : row.label}
href={`${base}${row.chapter}.html${row.anchor ? `#${row.anchor}` : ""}`}
verification={verificationBases?.[row.static.verification]}
onToggle={toggle}
/>
);
})}
</tbody>
@@ -35971,9 +35971,9 @@
"inherited": true
},
"static": {
"status": "not-implemented",
"detail": "No static compiler lowering is registered for this API yet.",
"verification": "registry-gap"
"status": "partial",
"detail": "Implemented for the documented scriptc module-loader subset.",
"verification": "test-backed"
}
},
{
@@ -81657,9 +81657,9 @@
"inherited": true
},
"static": {
"status": "not-implemented",
"detail": "No static compiler lowering is registered for this API yet.",
"verification": "registry-gap"
"status": "partial",
"detail": "Implemented for the documented static async-iteration subset.",
"verification": "test-backed"
}
},
{
+157
View File
@@ -0,0 +1,157 @@
// The compatibility matrix's data model: the ledger rows as a tree, and the
// search and filter passes over it. Kept free of React so the passes can be
// tested directly.
export type Status =
| "supported"
| "partial"
| "refused"
| "not-implemented"
| "by-design"
| "unreviewed"
| "not-applicable";
export type VerificationBasis =
| "test-backed"
| "explicit-refusal"
| "verified-absence"
| "declared-gap"
| "registry-gap"
| "architectural-policy"
| "unreviewed"
| "not-applicable"
| "derived";
export type Verification = { label: string; detail: string };
export type Tier = { status: Status; detail: string; verification: VerificationBasis };
export type NodeStability = { index: string; level: number; text: string; inherited: boolean } | null;
export type Row = {
id: string;
chapter: string;
kind: string;
name: string;
signature: string;
label: string;
apiSymbol: string;
depth: number;
scope: "api" | "documentation" | "metadata" | "configuration";
anchor: string;
nodeStability: NodeStability;
static: Tier;
};
export type Chapter = { slug: string; title: string; entries: number; apiEntries: number };
export type Snapshot = {
nodeVersion: string;
verificationBases: Record<VerificationBasis, Verification>;
chapters: Chapter[];
rows: Row[];
};
export type TreeNode = { row: Row; children: TreeNode[] };
export type VisibleNode = { node: TreeNode; level: number };
export const statuses = ["supported", "partial", "not-implemented", "refused", "by-design", "unreviewed", "not-applicable"] as const;
export function statusCounts(rows: Row[]): Record<Status, number> {
return rows.reduce<Record<Status, number>>(
(counts, row) => {
if (row.scope !== "api" || row.depth === 0) return counts;
counts[row.static.status] += 1;
return counts;
},
{
supported: 0,
partial: 0,
refused: 0,
"not-implemented": 0,
"by-design": 0,
unreviewed: 0,
"not-applicable": 0,
},
);
}
/** Node's all.json is preorder with an explicit heading depth. */
export function buildTree(rows: Row[]): TreeNode[] {
const roots: TreeNode[] = [];
const stack: TreeNode[] = [];
for (const row of rows) {
const node: TreeNode = { row, children: [] };
while (stack.length > row.depth) stack.pop();
const parent = stack[row.depth - 1];
if (row.depth === 0 || parent === undefined) roots.push(node);
else parent.children.push(node);
stack[row.depth] = node;
stack.length = row.depth + 1;
}
return roots;
}
/** The lowercase text a search matches against, built once per row: the
* chapter title, kind, name, signature, API symbol and support detail. */
export function searchTexts(rows: Row[], chapterTitles: ReadonlyMap<string, string>): Map<string, string> {
return new Map(
rows.map((row) => [
row.id,
[chapterTitles.get(row.chapter), row.kind, row.name, row.signature, row.apiSymbol, row.static.detail]
.join(" ")
.toLocaleLowerCase(),
]),
);
}
/** The rows a status filter and a lowercase search needle keep. */
export function rowMatcher(
texts: ReadonlyMap<string, string>,
needle: string,
status: "all" | Status,
): (row: Row) => boolean {
return (row) => {
if (status !== "all" && row.static.status !== status) return false;
return needle === "" || (texts.get(row.id) ?? "").includes(needle);
};
}
/** The matching rows with their ancestors, and the number of matching
* rows, in one pass. */
export function filterTree(nodes: TreeNode[], matches: (row: Row) => boolean): { nodes: TreeNode[]; matched: number } {
let matched = 0;
const visit = (items: TreeNode[]): TreeNode[] => {
const filtered: TreeNode[] = [];
for (const node of items) {
const children = visit(node.children);
const own = matches(node.row);
if (own) matched += 1;
if (own || children.length > 0) filtered.push({ row: node.row, children });
}
return filtered;
};
return { nodes: visit(nodes), matched };
}
export function flattenTree(nodes: TreeNode[], expanded: ReadonlySet<string>, forceExpanded: boolean): VisibleNode[] {
const visible: VisibleNode[] = [];
const visit = (items: TreeNode[], level: number) => {
for (const node of items) {
visible.push({ node, level });
if (node.children.length > 0 && (forceExpanded || expanded.has(node.row.id))) {
visit(node.children, level + 1);
}
}
};
visit(nodes, 1);
return visible;
}
export function nodeCount(nodes: TreeNode[]): number {
return nodes.reduce((count, node) => count + 1 + nodeCount(node.children), 0);
}
export function expandableIds(nodes: TreeNode[]): Set<string> {
const ids = new Set<string>();
const visit = (items: TreeNode[]) => {
for (const node of items) {
if (node.children.length > 0) ids.add(node.row.id);
visit(node.children);
}
};
visit(nodes);
return ids;
}
+115
View File
@@ -0,0 +1,115 @@
import assert from "node:assert/strict";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { test } from "node:test";
import { fileURLToPath, pathToFileURL } from "node:url";
import { createElement } from "react";
import { renderToStaticMarkup } from "react-dom/server";
import ts from "typescript";
import {
buildTree,
expandableIds,
filterTree,
flattenTree,
nodeCount,
rowMatcher,
searchTexts,
statusCounts,
} from "../src/lib/compatibility-tree.ts";
const row = (id, depth, label, status, detail = "") => ({
id,
chapter: "zlib",
kind: depth === 0 ? "module" : "method",
name: label,
signature: label,
label,
apiSymbol: label,
depth,
scope: "api",
anchor: "",
nodeStability: null,
static: { status, detail, verification: "test-backed" },
});
const rows = [
row("root", 0, "zlib", "partial"),
row("a", 1, "zlib.gzipSync()", "supported", "Lowered with Node's options"),
row("b", 1, "zlib.Gzip", "partial"),
row("b1", 2, "gzip.flush()", "supported"),
row("c", 1, "zlib.zstdCompress()", "not-implemented"),
];
const titles = new Map([["zlib", "Zlib"]]);
test("the tree follows the rows' heading depths", () => {
const tree = buildTree(rows);
assert.equal(tree.length, 1);
assert.deepEqual(tree[0].children.map((node) => node.row.id), ["a", "b", "c"]);
assert.deepEqual(tree[0].children[1].children.map((node) => node.row.id), ["b1"]);
assert.equal(nodeCount(tree), 5);
assert.deepEqual([...expandableIds(tree)].sort(), ["b", "root"]);
});
test("filtering keeps matching rows with their ancestors and counts the matches once", () => {
const tree = buildTree(rows);
const texts = searchTexts(rows, titles);
const byText = filterTree(tree, rowMatcher(texts, "flush", "all"));
assert.equal(byText.matched, 1);
assert.deepEqual(flattenTree(byText.nodes, new Set(), true).map(({ node, level }) => `${node.row.id}@${level}`), ["root@1", "b@2", "b1@3"]);
const byDetail = filterTree(tree, rowMatcher(texts, "node's options", "all"));
assert.deepEqual(flattenTree(byDetail.nodes, new Set(), true).map(({ node }) => node.row.id), ["root", "a"]);
const byStatus = filterTree(tree, rowMatcher(texts, "", "supported"));
assert.equal(byStatus.matched, 2);
const both = filterTree(tree, rowMatcher(texts, "gzip", "partial"));
assert.equal(both.matched, 1);
assert.equal(filterTree(tree, rowMatcher(texts, "brotli", "all")).nodes.length, 0);
});
test("collapsed rows hide their children until expanded", () => {
const tree = buildTree(rows);
assert.deepEqual(flattenTree(tree, new Set(), false).map(({ node }) => node.row.id), ["root"]);
assert.deepEqual(flattenTree(tree, new Set(["root"]), false).map(({ node }) => node.row.id), ["root", "a", "b", "c"]);
assert.deepEqual(flattenTree(tree, new Set(["root", "b"]), false).map(({ node }) => node.row.id), ["root", "a", "b", "b1", "c"]);
});
test("status counts cover API rows below the chapters", () => {
const counts = statusCounts(rows);
assert.equal(counts.supported, 2);
assert.equal(counts.partial, 1);
assert.equal(counts["not-implemented"], 1);
});
/** The matrix component, compiled for Node from its TSX source. */
async function loadMatrix() {
const docs = new URL("../", import.meta.url);
const source = await readFile(new URL("src/components/node-compatibility-matrix.tsx", docs), "utf8");
const { outputText } = ts.transpileModule(source, {
compilerOptions: { jsx: ts.JsxEmit.ReactJSX, module: ts.ModuleKind.ESNext, target: ts.ScriptTarget.ES2022 },
});
const lib = pathToFileURL(fileURLToPath(new URL("src/lib/compatibility-tree.ts", docs))).href;
const dir = new URL("node_modules/.cache/compatibility-matrix-test/", docs);
await mkdir(dir, { recursive: true });
const file = new URL("node-compatibility-matrix.mjs", dir);
await writeFile(file, outputText.replace('"@/lib/compatibility-tree"', JSON.stringify(lib)));
return import(file.href);
}
test("badges render without their tooltips until hovered or focused", async () => {
const { StatusBadge, StabilityBadge } = await loadMatrix();
const status = renderToStaticMarkup(
createElement(StatusBadge, {
status: "partial",
detail: "Lowered with Node's options",
verification: { label: "Test-backed", detail: "Corpus programs cover it." },
}),
);
assert.ok(status.includes('tabindex="0"'), status);
assert.ok(status.includes('title="Lowered with Node&#x27;s options'), status);
assert.ok(!status.includes('role="tooltip"'), status);
assert.ok(!status.includes("aria-describedby"), status);
const stability = renderToStaticMarkup(
createElement(StabilityBadge, { stability: { index: "2", level: 2, text: "Stable", inherited: false } }),
);
assert.ok(stability.includes('tabindex="0"'), stability);
assert.ok(!stability.includes('role="tooltip"'), stability);
const legend = renderToStaticMarkup(createElement(StatusBadge, { status: "supported" }));
assert.ok(!legend.includes("tabindex"), legend);
});
@@ -52194,6 +52194,7 @@
"tests/corpus/6294-webcrypto-js-values.mjs",
"tests/corpus/6295-jose-hs256/main.mjs",
"tests/corpus/6331-jose-root-entry/main.ts",
"tests/corpus/7743-transform-stream-identity.ts",
"tests/corpus/text-codec-values/main.ts",
"tests/corpus/webassembly-api.mjs",
"tests/corpus/webassembly-dylink/main.mjs",
@@ -53443,8 +53444,11 @@
"inherited": true
},
"static": {
"status": "not-implemented",
"evidence": "compiler-unmatched:globals"
"status": "partial",
"evidence": "compiler-feature:globals.TransformStream",
"tests": [
"tests/corpus/7743-transform-stream-identity.ts"
]
},
"anchorSource": "exact"
},
@@ -111483,6 +111487,7 @@
"tests/corpus/2630-stream-consumers-errors.ts",
"tests/corpus/2862-web-readable-async-iteration.ts",
"tests/corpus/4875-stream-consumers-array-buffer.ts",
"tests/corpus/7743-transform-stream-identity.ts",
"tests/corpus/stream-web-readable.ts",
"tests/corpus/stream-web-require.cjs",
"tests/harness/fetch-conformance.test.ts"
@@ -112957,8 +112962,11 @@
"inherited": true
},
"static": {
"status": "not-implemented",
"evidence": "compiler-unmatched:webstreams"
"status": "partial",
"evidence": "compiler-feature:webstreams.TransformStream",
"tests": [
"tests/corpus/7743-transform-stream-identity.ts"
]
},
"anchorSource": "exact"
},
+3 -1
View File
@@ -407,7 +407,9 @@
{ "chapter": "zlib", "signatures": ["`zlib.constants`"], "status": "partial", "evidence": ["tests/corpus/5376-zlib-constants.cjs", "tests/corpus/5374-zlib-stream-flush-params.ts"] },
{ "chapter": "dns", "symbols": ["dns.lookup", "dns.lookupService", "dns.getServers", "dns.getDefaultResultOrder", "dns.setDefaultResultOrder"], "status": "partial", "evidence": ["tests/corpus/5648-dns-lookup-forms.ts", "tests/corpus/5649-dns-service-servers.ts", "tests/corpus/5650-dns-lookup-errors-untyped.cjs"] },
{ "chapter": "dns", "symbols": ["dnsPromises.lookup", "dnsPromises.lookupService", "dnsPromises.getDefaultResultOrder"], "status": "partial", "evidence": ["tests/corpus/dns-promises-lookup.ts", "tests/corpus/dns-promises-untyped.cjs"] },
{ "chapter": "webstreams", "symbols": ["ReadableStream"], "status": "partial", "evidence": ["tests/corpus/stream-web-readable.ts", "tests/corpus/stream-web-require.cjs", "tests/corpus/2862-web-readable-async-iteration.ts"] }
{ "chapter": "webstreams", "symbols": ["ReadableStream"], "status": "partial", "evidence": ["tests/corpus/stream-web-readable.ts", "tests/corpus/stream-web-require.cjs", "tests/corpus/2862-web-readable-async-iteration.ts"] },
{ "chapter": "webstreams", "symbols": ["TransformStream"], "status": "partial", "evidence": ["tests/corpus/7743-transform-stream-identity.ts"] },
{ "chapter": "globals", "signatures": ["Class: `TransformStream`"], "status": "partial", "evidence": ["tests/corpus/7743-transform-stream-identity.ts"] }
],
"dedicated": [
{ "symbols": ["assert.doesNotThrow", "assert.doesNotReject"], "status": "partial", "evidence": ["tests/corpus/node24-no-error-assertions.ts", "tests/corpus/node24-no-error-assertions.cjs", "tests/corpus/node24-no-error-assertions-coercion.js", "tests/fixtures/node-types/no-error-assertions.ts", "tests/harness/project-config.test.ts"] },
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-darwin-arm64",
"version": "0.4.2",
"version": "0.4.3",
"description": "Native scriptc compiler for darwin-arm64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-darwin-x64",
"version": "0.4.2",
"version": "0.4.3",
"description": "Native scriptc compiler for darwin-x64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-arm64-gnu",
"version": "0.4.2",
"version": "0.4.3",
"description": "Native scriptc compiler for linux-arm64-gnu",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-arm64-musl",
"version": "0.4.2",
"version": "0.4.3",
"description": "Native scriptc compiler for linux-arm64-musl",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-x64-gnu",
"version": "0.4.2",
"version": "0.4.3",
"description": "Native scriptc compiler for linux-x64-gnu",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-x64-musl",
"version": "0.4.2",
"version": "0.4.3",
"description": "Native scriptc compiler for linux-x64-musl",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-win32-x64-msvc",
"version": "0.4.2",
"version": "0.4.3",
"description": "Native scriptc compiler for win32-x64-msvc",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "scriptc",
"version": "0.4.2",
"version": "0.4.3",
"description": "Compile ordinary TypeScript and JavaScript to small, fast native executables and WebAssembly modules that run without Node.js",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+28
View File
@@ -1106,6 +1106,34 @@ declare var ReadableStream: {
from(iterable: string): ReadableStream<string>;
readonly prototype: ReadableStream<unknown>;
};
/* The writable side of a TransformStream and its writer. The
* WritableStream constructor has no native lowering; writable streams come
* from `new TransformStream()`. */
interface WritableStreamDefaultWriter<W = unknown> {
readonly closed: Promise<void>;
readonly desiredSize: number | null;
readonly ready: Promise<void>;
abort(reason?: unknown): Promise<void>;
close(): Promise<void>;
releaseLock(): void;
write(chunk?: W): Promise<void>;
}
interface WritableStream<W = unknown> {
readonly locked: boolean;
abort(reason?: unknown): Promise<void>;
close(): Promise<void>;
getWriter(): WritableStreamDefaultWriter<W>;
}
/* The identity transform: chunks written to `writable` are read from
* `readable`, and a write completes when its chunk is read. */
interface TransformStream<I = unknown, O = unknown> {
readonly readable: ReadableStream<O>;
readonly writable: WritableStream<I>;
}
declare var TransformStream: {
new <I = unknown, O = I>(): TransformStream<I, O>;
readonly prototype: TransformStream<unknown, unknown>;
};
/* fetch, typed as the native JSON + readable-stream slice: one URL, an
* optional init with the members CLI requests carry, and a Response
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/compiler",
"version": "0.4.2",
"version": "0.4.3",
"description": "The scriptc compiler — TypeScript/JavaScript frontend, typed IR, LLVM backend",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
@@ -40,7 +40,7 @@
},
"dependencies": {
"@scriptc/runtime": "workspace:*",
"tsc-ts": "0.0.8",
"tsc-ts": "0.0.9",
"typescript": "7.0.2",
"typescript5": "npm:typescript@5.9.3"
},
+65 -12
View File
@@ -459,18 +459,72 @@ export type FreeObservers = "weak" | "weak-or-source";
* type's disposals), and the block — `scr_cyc_free` for collector-headered
* objects, `scr_rt_free` otherwise. Labels use the `sf.` prefix; the lines
* end in an open block.
*
* With `keep` (an alloca of ptr holding NULL; inline allocation paths only),
* a block that would go onto the calling thread's free list is stored there
* instead, for a teardown segment (emitSegmentFree); a block the system or
* another arena owns is still freed.
*/
export function emitObjectFree(
host: AllocHost,
cyc: boolean,
observers: FreeObservers = "weak",
keep: string | null = null,
): string[] {
const lines = threadLocalState(host, "sf", freeLines(host, cyc, observers));
const lines = threadLocalState(host, "sf", freeLines(host, cyc, observers, keep));
if (host.objectAudit === false) return lines.filter((line) => line !== FREE_NOTE);
host.declare(`declare void @scr_obj_free_note()`);
return lines;
}
/** Lines that put the blocks `first` to `last` (linked through their first
* words; one block when they are equal) of the class at reservation offset
* `off` on the calling thread's free list. */
function freeListPushLines(prefix: string, off: string, first: string, last: string): string[] {
return [
` %${prefix}.shp = getelementptr inbounds i8, ptr @scr_sa, i64 ${SA_SHIFT}`,
` %${prefix}.sh32 = load i32, ptr %${prefix}.shp`,
` %${prefix}.sh = zext i32 %${prefix}.sh32 to i64`,
` %${prefix}.c = lshr i64 ${off}, %${prefix}.sh`,
` %${prefix}.c8 = shl i64 %${prefix}.c, 3`,
` %${prefix}.fo = add i64 %${prefix}.c8, ${SA_FREE}`,
` %${prefix}.fp = getelementptr inbounds i8, ptr @scr_sa, i64 %${prefix}.fo`,
` %${prefix}.head = load ptr, ptr %${prefix}.fp`,
` store ptr %${prefix}.head, ptr ${last}`,
` store ptr ${first}, ptr %${prefix}.fp`,
];
}
/**
* Lines that free a teardown segment: the blocks `first` to `last` (ptr
* values, both NULL for an empty segment), linked through their first words,
* which emitObjectFree kept. Every block of a segment is in the calling
* thread's arena and of one size class, so the segment goes onto that
* class's free list in one step, and the next allocations of the class take
* its blocks in segment order. Labels use `prefix`; the lines end in the
* open block `${prefix}.done`.
*/
export function emitSegmentFree(
host: AllocHost,
prefix: string,
first: string,
last: string,
): string[] {
declareState(host);
const lines = [
` %${prefix}.none = icmp eq ptr ${first}, null`,
` br i1 %${prefix}.none, label %${prefix}.done, label %${prefix}.push`,
`${prefix}.push:`,
` %${prefix}.base = load i64, ptr @scr_sa`,
` %${prefix}.bi = ptrtoint ptr ${first} to i64`,
` %${prefix}.off = sub i64 %${prefix}.bi, %${prefix}.base`,
...freeListPushLines(prefix, `%${prefix}.off`, first, last),
` br label %${prefix}.done`,
`${prefix}.done:`,
];
return threadLocalState(host, prefix, lines);
}
/** Lines that leave in `%<prefix>.obs` whether the program can observe a
* disposal of this type (folds to a constant). */
function observedLines(prefix: string, observers: FreeObservers): string[] {
@@ -482,8 +536,14 @@ function observedLines(prefix: string, observers: FreeObservers): string[] {
];
}
function freeLines(host: AllocHost, cyc: boolean, observers: FreeObservers): string[] {
function freeLines(
host: AllocHost,
cyc: boolean,
observers: FreeObservers,
keep: string | null,
): string[] {
if (host.inlineAlloc !== true) {
if (keep !== null) throw new Error("a kept block needs the inline allocation paths");
if (cyc) {
host.declare(`declare void @scr_cyc_free(ptr)`);
return [` call void @scr_obj_free_note()`, ` call void @scr_cyc_free(ptr %o)`];
@@ -548,16 +608,9 @@ function freeLines(host: AllocHost, cyc: boolean, observers: FreeObservers): str
` %sf.inx = call i1 @llvm.expect.i1(i1 %sf.in, i1 true)`,
` br i1 %sf.inx, label %sf.push, label %sf.sys`,
`sf.push:`,
` %sf.shp = getelementptr inbounds i8, ptr @scr_sa, i64 ${SA_SHIFT}`,
` %sf.sh32 = load i32, ptr %sf.shp`,
` %sf.sh = zext i32 %sf.sh32 to i64`,
` %sf.c = lshr i64 %sf.off, %sf.sh`,
` %sf.c8 = shl i64 %sf.c, 3`,
` %sf.fo = add i64 %sf.c8, ${SA_FREE}`,
` %sf.fp = getelementptr inbounds i8, ptr @scr_sa, i64 %sf.fo`,
` %sf.head = load ptr, ptr %sf.fp`,
` store ptr %sf.head, ptr ${blk}`,
` store ptr ${blk}, ptr %sf.fp`,
...(keep !== null
? [` store ptr ${blk}, ptr ${keep}`]
: freeListPushLines("sf", "%sf.off", blk, blk)),
` br label %sf.done`,
`sf.sys: ; outside the reservation, or the allocator is compiled out`,
` call void @scr_obj_free_note()`,
@@ -7,9 +7,15 @@ import type { LlvmEmitterContext } from "./expr-context.js";
* buffer. Neither overlaps the other, a native field, array storage or a
* module global's slot, so element stores leave header loads and global
* receiver slots loop-invariant. Untagged accesses and calls still alias
* every category. */
* every category.
*
* Header loads are also `!invariant.load`: no store or call between the
* construction of a view and its release changes them. Without it, a data
* pointer load ahead of a bounds branch is sunk into the in-bounds block,
* and LICM keeps a conditionally executed load inside the loop. */
export function bytesHeaderAlias(host: LlvmEmitterContext): string {
return host.memoryAliasAttachment("bytes:header");
const tag = host.memoryAliasAttachment("bytes:header");
return tag === "" ? "" : `${tag}, !invariant.load !{}`;
}
export function bytesElementAlias(host: LlvmEmitterContext): string {
@@ -11,7 +11,7 @@ import {
type IrType,
} from "../../ir/ir.js";
import { analyzeIntegerRanges } from "../../ir/integer-ranges.js";
import { matchIntegerCountedForLoop } from "../../ir/integer-loops.js";
import { matchIntegerCountedForLoop, matchIntegerPairedForLoop } from "../../ir/integer-loops.js";
import { findInitializedByteLoopBindings } from "./byte-windows.js";
import { matchBytesLoopVersion, type BytesLoopVersion } from "./bytes-loop-versions.js";
import { emitLlvmModule } from "./emitter.js";
@@ -249,6 +249,11 @@ test("emitted loops version the body and tag typed-array memory", () => {
const elements = /store double [^\n]*!tbaa !(\d+)/.exec(body)![1];
const header = /load ptr, ptr [^\n]*!tbaa !(\d+)/.exec(body)![1];
expect(elements).not.toBe(header);
// Header loads are invariant for the view's lifetime; element accesses
// are not.
expect(body).toMatch(/load ptr, ptr [^\n]*, !tbaa !\d+, !invariant\.load !\{\}$/m);
expect(body).toMatch(/load i64, ptr [^\n]*, !tbaa !\d+, !invariant\.load !\{\}$/m);
expect(body).not.toMatch(/(load double|store double)[^\n]*invariant\.load/);
});
test("loops to a typed array's length version the other typed arrays they index", () => {
@@ -618,3 +623,78 @@ test("while loop counters below a string's length version its reads", () => {
expect(version.bounded.map((c) => c.slack)).toEqual([0]);
expect(version.fixed).toEqual([]);
});
test("a two-counter loop versions the typed arrays it indexes at either counter", () => {
// for (let i = 0, j = k; i < j; i++, j--) { const t = out[i]; out[i] = out[j]; out[j] = t; }
const loop: IrStmt & { kind: "for" } = {
kind: "for",
init: {
kind: "block",
body: [
{ kind: "varDecl", localId: "i", init: num(0), loc },
{ kind: "varDecl", localId: "j", init: ref("k"), loc },
],
loc,
},
cond: { kind: "bin", op: "<", left: ref("i"), right: ref("j"), type: BOOL, loc },
update: {
kind: "block",
body: [
{ kind: "assign", localId: "i", value: bin("+", ref("i"), num(1)), loc },
{ kind: "assign", localId: "j", value: bin("-", ref("j"), num(1)), loc },
],
loc,
},
body: [
{ kind: "varDecl", localId: "t", init: get(ref("out", f64s), ref("i")), loc },
set(ref("out", f64s), ref("i"), get(ref("out", f64s), ref("j"))),
set(ref("out", f64s), ref("j"), ref("t")),
// Another index stays checked.
set(ref("src", f64s), bin("+", ref("i"), num(1)), num(0)),
],
loc,
};
const kernel = fn([loop], [local("t", F64)]);
const ranges = analyzeIntegerRanges(kernel);
const locals = new Map(kernel.locals.map((l) => [l.id, l]));
const paired = matchIntegerPairedForLoop(loop, locals, ranges)!;
expect(paired).not.toBeNull();
const version = matchBytesLoopVersion(
loop,
null,
{
locals,
invariantGlobals: new Set(),
captured: new Set(),
ranges,
initialized: findInitializedByteLoopBindings(kernel).get(loop) ?? new Set(),
proven: new Set(),
workers: false,
owned: new Set(),
scalarStrings: new Set(),
},
paired,
)!;
expect(version.paired?.loop).toBe(paired);
expect(version.paired?.receivers.map((r) => version.receivers[r]!.ref)).toEqual([
ref("out", f64s),
]);
expect(version.bounds.size).toBe(4);
const mod: IrModule = {
irVersion: 17,
sourceFile: loc.file,
entry: "__main",
functions: [
{ name: "__main", params: [], returnType: VOID, locals: [], body: [], loc },
kernel,
],
};
const body = /^define internal [^\n]*@sc_bf_kernel[^]*?^}/m.exec(emitLlvmModule(mod))![0];
const guard = body.slice(0, body.search(/^bytes\.version\.(fast|slow)\d+:$/m));
expect(guard).toMatch(/icmp sge i64 %\S+, 0/);
expect(guard).toMatch(/icmp slt i64 %\S+, %\S+/);
const fast = /^bytes\.version\.fast\d+:$[^]*?^bytes\.version\.done\d+:$/m.exec(body)![0];
// The fast copy keeps only the `src[i + 1]` check, and no second guard.
expect(fast.match(/bytes\.index\.invalid\d+:/g)).toHaveLength(1);
expect(fast).not.toContain("counted.fast");
});
@@ -1,11 +1,15 @@
import { F64, type IrExpr, type IrLocal, type IrStmt } from "../../ir/ir.js";
import type { IntegerRange, IntegerRanges } from "../../ir/integer-ranges.js";
import type { IntegerCountedForLoop } from "../../ir/integer-loops.js";
import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js";
import type { LlvmEmitterContext } from "./expr-context.js";
import { byteNumberAccess } from "../../ir/byte-numbers.js";
import { everyExprChild, everyStmtChild } from "../../ir/traverse.js";
import { exactInteger, widenInteger } from "./integer-values.js";
import { emitCountedLoopLimit, emitCountedLoopSafety } from "./counted-loops.js";
import {
emitCountedLoopLimit,
emitCountedLoopSafety,
emitExactIntegerTest,
} from "./counted-loops.js";
import { emitBytesHeaderLength, emitBytesSharedStorage } from "./expr-bytes.js";
import { f64Lit } from "./common.js";
import { emitStringAsciiFlag, emitStringByteLength } from "./string-reads.js";
@@ -79,6 +83,10 @@ export interface BytesLoopVersion {
guarded: ReadonlySet<string>;
/** The counted loop whose iteration space the affine checks cover. */
counted: IntegerCountedForLoop | null;
/** A two-counter loop (`for (let i = lo, j = hi; i < j; i++, j--)`) and
* the receivers it indexes at either counter. Both counters stay between
* the two starts, so the guard requires `lo >= 0` and `hi < length`. */
paired: { loop: IntegerPairedForLoop; receivers: number[] } | null;
/** The guard also requires every counter value within [-bound, bound]. */
counterBound: number | null;
/** Integer ranges of affine index operands that hold in the unchecked
@@ -153,6 +161,7 @@ export function matchBytesLoopVersion(
loop: IrStmt & { kind: "for" | "while" },
counted: IntegerCountedForLoop | null,
facts: BytesLoopFacts,
paired: IntegerPairedForLoop | null = null,
): BytesLoopVersion | null {
const written = new Set<string>();
const temps = new Map<string, IrExpr>();
@@ -438,6 +447,9 @@ export function matchBytesLoopVersion(
string,
{ check: BoundedCheck; nodes: { node: Node; access: WhileAccess }[] }
>();
const pairedCounters =
paired !== null && counted === null ? new Set([paired.low.localId, paired.high.localId]) : null;
const pairedNodes = new Map<number, Node[]>();
const access = (node: Node, array: IrExpr, index: IrExpr, width: number): void => {
const ref = root(array);
if (!ref) return;
@@ -448,6 +460,20 @@ export function matchBytesLoopVersion(
else owned.set(receiver, [node]);
}
if (facts.proven.has(node)) return;
// An element at either counter of a two-counter loop.
if (
pairedCounters !== null &&
width === 1 &&
ref.type.kind === "bytes" &&
index.kind === "varRef" &&
pairedCounters.has(index.localId)
) {
const receiver = receiverOf(ref);
const nodes = pairedNodes.get(receiver);
if (nodes) nodes.push(node);
else pairedNodes.set(receiver, [node]);
return;
}
const shape = counter !== null ? affine(index) : null;
if (shape && shape.counter !== 0) {
const operands = new Set<IrExpr>();
@@ -543,11 +569,16 @@ export function matchBytesLoopVersion(
Math.max(0, MAX_CHECKS - fixedKept.length - affineKept.length),
);
const ownedKept = [...owned.entries()].slice(0, MAX_CHECKS);
const pairedKept = [...pairedNodes.entries()].slice(
0,
Math.max(0, MAX_CHECKS - fixedKept.length - affineKept.length - boundedKept.length),
);
if (
fixedKept.length === 0 &&
affineKept.length === 0 &&
boundedKept.length === 0 &&
ownedKept.length === 0
ownedKept.length === 0 &&
pairedKept.length === 0
)
return null;
const bounds = new Set<Node>();
@@ -616,6 +647,10 @@ export function matchBytesLoopVersion(
if (access.index !== access.ref) narrow(access.index, { min: 0, max: SAFE });
}
}
for (const [receiver, nodes] of pairedKept) {
used.add(receiver);
for (const node of nodes) bounds.add(node);
}
const counterChecks: CounterCheck[] = [];
for (const [id, min] of counterMins) {
const info = whileAccesses!.counters.get(id)!;
@@ -658,6 +693,10 @@ export function matchBytesLoopVersion(
counters: counterChecks,
guarded,
counted: affineKept.length > 0 ? counted : null,
paired:
paired !== null && pairedKept.length > 0
? { loop: paired, receivers: pairedKept.map(([receiver]) => renumber.get(receiver)!) }
: null,
counterBound: needsBound ? bound : null,
ranges,
};
@@ -1143,6 +1182,25 @@ export function emitBytesLoopVersionGuard(
conditions.push(both);
}
}
if (version.paired !== null) {
// Both counters stay within [lo, hi] (exact integers below 2^53): every
// access at either one is in bounds when `lo >= 0` and `hi < length`.
const { loop } = version.paired;
const [low, high] = [loop.low.start, loop.high.start].map((expr) => {
const tested = emitExactIntegerTest(host, host.emitExpr(expr).name, -SAFE, SAFE);
conditions.push(tested.ok);
return tested.integer;
});
const nonNegative = B.tmp();
B.line(`${nonNegative} = icmp sge i64 ${low}, 0`);
conditions.push(nonNegative);
for (const receiver of version.paired.receivers) {
// Lengths are at most 2^53 elements: a signed comparison is exact.
const below = B.tmp();
B.line(`${below} = icmp slt i64 ${high}, ${lengths[receiver]}`);
conditions.push(below);
}
}
let valid = "true";
for (const cond of conditions) {
if (valid === "true") valid = cond;
@@ -1156,39 +1214,6 @@ export function emitBytesLoopVersionGuard(
else B.condBr(valid, fast, slow);
}
/** Tests that a double is an exact integer within [min, max] (not -0; NaN
* fails the ordered comparisons) and gives its i64. The integer is frozen,
* so it is a fixed value even when the test fails. */
function emitExactIntegerTest(
host: LlvmEmitterContext,
value: string,
min: number,
max: number,
): { ok: string; integer: string } {
const B = host.B;
const low = B.tmp(),
high = B.tmp(),
inRange = B.tmp(),
raw = B.tmp(),
integer = B.tmp(),
back = B.tmp(),
bits = B.tmp(),
backBits = B.tmp(),
exact = B.tmp(),
ok = B.tmp();
B.line(`${low} = fcmp oge double ${value}, ${f64Lit(min)}`);
B.line(`${high} = fcmp ole double ${value}, ${f64Lit(max)}`);
B.line(`${inRange} = and i1 ${low}, ${high}`);
B.line(`${raw} = fptosi double ${value} to i64`);
B.line(`${integer} = freeze i64 ${raw}`);
B.line(`${back} = sitofp i64 ${integer} to double`);
B.line(`${bits} = bitcast double ${value} to i64`);
B.line(`${backBits} = bitcast double ${back} to i64`);
B.line(`${exact} = icmp eq i64 ${bits}, ${backBits}`);
B.line(`${ok} = and i1 ${inRange}, ${exact}`);
return { ok, integer };
}
function scaleCounter(host: LlvmEmitterContext, value: string, scale: number): string {
if (scale === 1) return value;
const scaled = host.B.tmp();
@@ -77,6 +77,65 @@ test("leaf layouts keep direct teardown while recursive layouts share the depth
}
});
test("teardowns of types with children of their own type free their blocks as one segment", () => {
const linked = { kind: "object" as const, className: "Linked" };
const mod: IrModule = {
irVersion: 17,
sourceFile: loc.file,
entry: "main",
functions: [{ name: "main", params: [], returnType: VOID, locals: [], body: [], loc }],
classes: [
{
name: "Linked",
fields: [
{ name: "next", type: linked },
{ name: "other", type: linked },
],
methods: [],
loc,
},
],
records: [{ id: "branch", fields: [{ name: "next", type: linked }] }],
};
const define = (llvm: string, name: string): string =>
new RegExp(`^define internal [^@]*@${name}\\([^]*?^}`, "m").exec(llvm)?.[0] ?? "";
const linkedDestroy = `${mangleClassRelease("Linked")}_destroy`;
const branchDestroy = `${mangleRecordRelease("branch")}_destroy`;
for (const workers of [false, true]) {
const llvm = emitLlvmModule({ ...mod, workers }, { pointerBits: 64 });
// Linked's teardown keeps its block and its children's in a segment: it
// releases its own block first and places it by address against its
// first child, a child torn down directly appends to the segment, one
// past the direct chain appends through the runtime, and the chain's
// first teardown frees the segment onto the free list in one step.
const seg = define(llvm, `${linkedDestroy}_s`);
expect(seg.startsWith("define internal ptr"), "segment form").toBe(true);
expect(seg).toMatch(
/store ptr %(?:sf\.blk|o), ptr %sg\.own[^]*call ptr @sc_release_Linked_destroy_s\(/,
);
// The children's releases have a copy for each place of the own block.
for (const copy of ["ta", "tb"])
expect(seg).toContain(
`call ptr @scr_rc_destroy_seg(ptr %v0, ptr @${linkedDestroy}_sc, ptr @${linkedDestroy}, ptr %${copy}0.bt)`,
);
expect(seg).not.toContain("call void @scr_rc_destroy(");
expect(seg).toContain("icmp ult ptr %sg.o, %sg.r1");
const first = define(llvm, `${linkedDestroy}_d`);
expect(first).toContain(`call ptr @${linkedDestroy}_s(ptr %o, i32 %d, ptr %head)`);
expect(first).toContain("store ptr %first, ptr %sgf.fp");
expect(first.includes("llvm.threadlocal.address"), "thread-local state").toBe(workers);
expect(define(llvm, linkedDestroy)).toContain(`call void @${linkedDestroy}_d(ptr %o, i32 0)`);
// A record whose child is another type frees its own block as before.
expect(define(llvm, `${branchDestroy}_s`)).toBe("");
expect(define(llvm, branchDestroy)).toContain(`call void @${mangleClassRelease("Linked")}(`);
}
// Without the inline allocation paths (32-bit targets) a teardown frees
// through the runtime, block by block.
const llvm32 = emitLlvmModule(mod, { pointerBits: 32 });
expect(define(llvm32, `${linkedDestroy}_s`)).toBe("");
expect(define(llvm32, `${linkedDestroy}_d`)).toContain(`call void @${linkedDestroy}_d(`);
});
test("releases split into an inlinable fast path and an out-of-line remainder", () => {
// An array of the class itself makes the hierarchy cycle-capable.
const kids = { kind: "array" as const, elem: { kind: "object" as const, className: "Node" } };
+58 -15
View File
@@ -43,6 +43,9 @@ import {
destroyDefinitions,
directChildReleaseLines,
directDestroyTarget,
segmentPrologueLines,
segmentTeardownLines,
segmentedTeardown,
usedDirectTargets,
type DirectDestroy,
type RegionHooks,
@@ -1082,25 +1085,62 @@ export function emitClassShapes(
bounded &&
(directTargets.has(`object:${cls.name}`) ||
refFields.some((f) => directDestroyTarget(host, f.type, directTargets) !== null));
// The teardown's own symbol, and whether it keeps a segment (shapes.ts
// segmentedTeardown).
const destroy = `${meta.hierarchy ? mangleClassReleaseDirect(cls.name) : mangleClassRelease(cls.name)}_destroy`;
const segmented =
chained &&
!meta.compact &&
runtimePrefix === null &&
!isStreamRooted &&
!meta.threadSlot &&
segmentedTeardown(
host,
destroy,
refFields.map((f) => f.type),
directTargets,
);
// The field-releasing teardown body shared by both release shapes
// (the public one on standalone classes, the DIRECT one on hierarchy
// members). Runs at rc == 0.
const teardown = (lines: string[]): void => {
refFields.forEach((f, t) => {
lines.push(
` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${f.index}`,
` %v${t} = load ptr, ptr %f${t}`,
);
const release = (prefix: string, f: (typeof refFields)[number], t: number): string[] => {
const direct = bounded ? directDestroyTarget(host, f.type, directTargets) : null;
if (direct)
lines.push(
...directChildReleaseLines(host, `td${t}`, `%v${t}`, direct, llvmCommentText(f.name)),
);
else
lines.push(
if (direct === null)
return [
` call void ${releaseSym(host, f.type)}(ptr %v${t}) ; ${llvmCommentText(f.name)}`,
);
});
];
const own = segmented && direct.destroy === destroy;
return directChildReleaseLines(
host,
`${prefix}${t}`,
`%v${t}`,
direct,
llvmCommentText(f.name),
own,
);
};
const loads = refFields.map((f, t) => [
` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${f.index}`,
` %v${t} = load ptr, ptr %f${t}`,
]);
if (segmented) {
// A segmented teardown loads its fields, releases its own block, and
// then releases the fields (shapes.ts segmentTeardownLines).
const ownType = refFields.flatMap((f, t) =>
directDestroyTarget(host, f.type, directTargets)?.destroy === destroy ? [`%v${t}`] : [],
);
lines.push(
...segmentPrologueLines(),
...loads.flat(),
...emitObjectFree(host, traced, "weak", "%sg.own"),
...segmentTeardownLines(ownType, (prefix) =>
refFields.flatMap((f, t) => release(prefix, f, t)),
),
);
return;
}
refFields.forEach((f, t) => lines.push(...loads[t]!, ...release("td", f, t)));
if (runtimePrefix) lines.push(...regCall("td", runtimePrefix.drop, ""));
if (isStreamRooted) lines.push(...stCall("tds", "scr_stream_st_release", ""));
if (meta.threadSlot) lines.push(...slotForgetLines(host, struct, meta, "tdw"));
@@ -1182,10 +1222,12 @@ export function emitClassShapes(
];
if (traced) reld.push(...cycleDeadLines(host, "teardown"));
if (bounded) {
const destroy = `${mangleClassReleaseDirect(cls.name)}_destroy`;
const destroyBody: string[] = [];
teardown(destroyBody);
defs.push(...destroyDefinitions(destroy, destroyBody, chained), ``);
defs.push(
...destroyDefinitions(destroy, destroyBody, chained, segmented ? host : null),
``,
);
host.declare(`declare void @scr_rc_destroy(ptr, ptr)`);
reld.push(` call void @scr_rc_destroy(ptr %o, ptr @${destroy})`);
} else teardown(reld);
@@ -1207,6 +1249,7 @@ export function emitClassShapes(
bounded,
lifetime,
chained,
segmented,
),
``,
);
@@ -29,6 +29,9 @@ export const SCR_ARR_FIELD = {
* (SCR_ARR_STATES in scr_array.h). */
export const SCR_ARR_STATES = 16;
/** The `spare` flag of a frozen %ScrArr (SCR_ARR_FROZEN in scr_array.h). */
export const SCR_ARR_FROZEN = 32;
/** Loads the i32 `len` or `cap` of a %ScrArr at `ptr` into `dest` as a
* size-type count. */
export function loadArrCount(
@@ -151,3 +151,36 @@ function emitExactStart(host: LlvmEmitterContext, expr: IrExpr): string {
B.line(`${ok} = and i1 ${inRange}, ${exact}`);
return ok;
}
/** Tests that a double is an exact integer within [min, max] (not -0; NaN
* fails the ordered comparisons) and gives its i64. The integer is frozen,
* so it is a fixed value even when the test fails. */
export function emitExactIntegerTest(
host: LlvmEmitterContext,
value: string,
min: number,
max: number,
): { ok: string; integer: string } {
const B = host.B;
const low = B.tmp(),
high = B.tmp(),
inRange = B.tmp(),
raw = B.tmp(),
integer = B.tmp(),
back = B.tmp(),
bits = B.tmp(),
backBits = B.tmp(),
exact = B.tmp(),
ok = B.tmp();
B.line(`${low} = fcmp oge double ${value}, ${f64Lit(min)}`);
B.line(`${high} = fcmp ole double ${value}, ${f64Lit(max)}`);
B.line(`${inRange} = and i1 ${low}, ${high}`);
B.line(`${raw} = fptosi double ${value} to i64`);
B.line(`${integer} = freeze i64 ${raw}`);
B.line(`${back} = sitofp i64 ${integer} to double`);
B.line(`${bits} = bitcast double ${value} to i64`);
B.line(`${backBits} = bitcast double ${back} to i64`);
B.line(`${exact} = icmp eq i64 ${bits}, ${backBits}`);
B.line(`${ok} = and i1 ${inRange}, ${exact}`);
return { ok, integer };
}
@@ -505,8 +505,12 @@ export function emitDenseArrayState(
indexExpr: IrExpr | undefined,
kind: "state" | "has",
width: number,
/** The index is a dense array loop counter its entry test proved below
* the hole-free prefix (provenDenseIndex): the slot is a present value. */
proven = false,
): string {
const B = host.B;
if (proven) return kind === "state" ? f64Lit(SCR_ARR_VALUE) : "true";
const fn = kind === "state" ? "scr_arr_state" : "scr_arr_has";
const ty = kind === "state" ? "double" : "i1";
host.declare(`declare ${kind === "state" ? "double" : "zeroext i1"} @${fn}(ptr, double)`);
@@ -1,9 +1,10 @@
import { expect, test } from "vitest";
import { BOOL, F64, arrayOf, type IrExpr, type IrLocal, type IrStmt } from "../../ir/ir.js";
import type { IntegerCountedForLoop } from "../../ir/integer-loops.js";
import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js";
import {
countedDenseShape,
matchDenseArrayLoop,
pairedDenseShape,
type DenseLoopShape,
} from "./dense-array-loops.js";
@@ -42,6 +43,8 @@ const locals = new Map<string, IrLocal>([
["a", { id: "a", name: "a", type: ARR, mutable: false }],
["b", { id: "b", name: "b", type: ARR, mutable: true }],
["c", { id: "c", name: "c", type: ARR, mutable: false }],
["j", { id: "j", name: "j", type: F64, mutable: true }],
["t", { id: "t", name: "t", type: F64, mutable: false }],
]);
/** for (let i = 0; i < a.length; i++) body */
@@ -193,3 +196,100 @@ test("a loop to another invariant limit covers each array whose length it keeps"
expect([...matchLimited(resizing, new Set(["b"]))!.arrays.keys()]).toEqual(["a"]);
expect(matchLimited(resizing)).toBeNull();
});
test("a two-counter loop covers the arrays indexed at either counter", () => {
// for (let i = 1, j = k - 1; i < j; i++, j--) { t = a[i]; a[i] = a[j]; a[j] = t; }
const paired: IntegerPairedForLoop = {
low: { localId: "i", start: num(1), step: 1, startGuarded: false },
high: { localId: "j", start: ref("k"), step: -1, startGuarded: true },
inclusive: false,
guarded: true,
range: { min: -Number.MAX_SAFE_INTEGER, max: Number.MAX_SAFE_INTEGER },
};
const pairedShape = pairedDenseShape(paired);
const matchPaired = (body: IrStmt[]) =>
matchDenseArrayLoop(loop(body), pairedShape, locals, new Set(), new Set(["a", "b"]));
const swap: IrStmt[] = [
{ kind: "varDecl", localId: "t", init: read("a"), loc },
write("a", read("a", ref("j"))),
write("a", ref("t"), ref("j")),
];
const dense = matchPaired(swap);
expect([...dense!.arrays.keys()]).toEqual(["a"]);
expect(dense!.pair).toEqual({ high: "j", lowStart: num(1), highStart: ref("k") });
expect(dense!.range).toEqual({ min: 0, max: 2147483646 });
// Stores at any other index may extend the array.
expect(matchPaired([...swap, write("a", num(0), ref("k"))])).toBeNull();
});
test("a present-undefined store is covered only where it stores a covered read's missing arm", () => {
// const v = b[i] (as number | undefined);
// if (v is undefined) c[i] = undefined; else c[i] = v is number ? v : throw;
const unionType = { kind: "union" as const, unionId: "u" };
const v: IrExpr = { kind: "varRef", localId: "v", type: unionType, loc };
const isTag = (tag: number): IrExpr => ({
kind: "unionIsTag",
unionId: "u",
tag,
negated: false,
value: v,
type: BOOL,
loc,
});
const reader = (array: string, index = ref("i")): IrExpr => ({
kind: "call",
callee: "%arr.idxOr.f64",
args: [arr(array), index],
type: unionType,
loc,
});
const undefinedStore: IrStmt = { kind: "arraySetUndefined", arr: arr("c"), index: ref("i"), loc };
const guarded = (array: string): IrStmt[] => [
{ kind: "varDecl", localId: "v", init: reader(array), loc },
{
kind: "if",
cond: isTag(1),
then: [undefinedStore],
else_: [
write("c", {
kind: "ternary",
cond: isTag(0),
then: { kind: "unionNarrow", unionId: "u", tag: 0, value: v, type: F64, loc },
else_: {
kind: "libCall",
fn: "error.nodeThrow",
args: [num(1), { kind: "strLit", value: "", type: { kind: "string" }, loc }],
type: F64,
loc,
},
type: F64,
loc,
}),
],
loc,
},
];
const withV = new Map([
...locals,
["v", { id: "v", name: "v", type: unionType, mutable: false }],
]);
const matchStores = (body: IrStmt[], initialized: string[]) =>
matchDenseArrayLoop(loop(body), shape, withV, new Set(), new Set(initialized));
const dense = matchStores(guarded("b"), ["a", "b", "c"]);
expect([...dense!.arrays.keys()].sort()).toEqual(["b", "c"]);
// A read of an array the entry test does not cover may miss in the fast
// copy, and its undefined store would run there.
expect(matchStores(guarded("b"), ["a", "c"])).toBeNull();
// A present undefined stored on its own leaves out its array and every
// array the other name may hold.
const alone: IrStmt[] = [{ kind: "exprStmt", expr: read("b"), loc }, undefinedStore];
expect(matchStores(alone, ["a", "b", "c"])).toBeNull();
const separate = matchDenseArrayLoop(
loop(alone),
shape,
withV,
new Set(["c"]),
new Set(["a", "b", "c"]),
);
expect([...separate!.arrays.keys()]).toEqual(["b"]);
});
@@ -20,16 +20,23 @@
* A counted loop with another invariant limit, such as `j < n` after
* `const n = a.length` or `j < 5`, is covered the same way: the test bounds
* the loop's exclusive limit by 2^31 - 1 and by each indexed array's
* `filled`, and the loop must provably never change any of their lengths. */
* `filled`, and the loop must provably never change any of their lengths.
*
* A two-counter loop closing in on itself, such as
* `for (let i = 1, j = k - 1; i < j; i++, j--)`, indexes only between its
* low counter's start and its high counter's: the low counter only rises
* and stays below the high one, which only falls. Its test bounds the low
* start below by 0 and the high start by 2^31 - 2 and below each indexed
* array's `filled`, both starts being exact integers. */
import type { IrExpr, IrLocal, IrStmt } from "../../ir/ir.js";
import type { IntegerRange } from "../../ir/integer-ranges.js";
import { preservesArrayLength } from "../../ir/array-length-stability.js";
import { everyExprChild, everyStmtChild } from "../../ir/traverse.js";
import type { IntegerCountedForLoop } from "../../ir/integer-loops.js";
import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js";
import type { LlvmEmitterContext } from "./expr-context.js";
import { loadArrCount, SCR_ARR_FIELD } from "./common.js";
import { emitCountedLoopLimit } from "./counted-loops.js";
import { emitCountedLoopLimit, emitExactIntegerTest } from "./counted-loops.js";
export const INT32_MAX = 2147483647;
@@ -47,6 +54,17 @@ export interface DenseArrayLoop {
arrays: ReadonlyMap<string, IrExpr & { kind: "varRef" }>;
/** The counter's range on every pass of the fast copy. */
range: IntegerRange;
/** A two-counter loop's high counter and both starts, when the counter
* is its low one (`length` and `limit` are then null). */
pair: DensePair | null;
}
/** The high counter of a two-counter dense loop, and the starts its entry
* test bounds. */
export interface DensePair {
high: string;
lowStart: IrExpr;
highStart: IrExpr;
}
/** The fast copy duplicates the loop; keep the duplicated code small. */
@@ -62,6 +80,8 @@ export interface DenseLoopShape {
/** The counter's proven range on every pass of the ordinary loop; the
* fast copy keeps its minimum, which must be at least 0. */
range: IntegerRange;
/** A two-counter loop's high counter and starts. */
pair?: DensePair;
}
/** `length.length - c` (c an integer literal, 0 when absent). */
@@ -107,6 +127,124 @@ export function countedDenseShape(
return { counter: counted.localId, length: start.receiver, range: counted.range };
}
/** The shape of a two-counter loop whose low counter rises and high
* counter falls: both stay between the two starts, which the entry test
* bounds, so the fast copy's counters lie in [0, 2^31 - 2]. */
export function pairedDenseShape(loop: IntegerPairedForLoop): DenseLoopShape {
return {
counter: loop.low.localId,
length: null,
range: { min: 0, max: INT32_MAX - 1 },
pair: { high: loop.high.localId, lowStart: loop.low.start, highStart: loop.high.start },
};
}
interface ElementRead {
arr: IrExpr;
index: IrExpr;
}
/** The array and index of a call of an element-reader helper
* (`%arr.idxOr.*`, ir/int32-arrays.ts isElementReader): `a[i]` as
* `number | undefined`. */
function elementReaderCall(e: IrExpr): ElementRead | null {
if (e.kind !== "call" || !e.callee.startsWith("%arr.idxOr.") || e.args.length !== 2) return null;
return { arr: e.args[0]!, index: e.args[1]! };
}
/** The array and index of an element read as `T | undefined`: an
* element-reader call, or its body inlined for an int32 array
* (`arrayState(a, i) === 1 ? wrap(a[i]) : wrap(undefined)`). */
function elementRead(e: IrExpr): ElementRead | null {
const call = elementReaderCall(e);
if (call) return call;
if (e.kind !== "ternary") return null;
const cond = e.cond;
if (cond.kind !== "bin" || cond.op !== "===") return null;
const state = cond.left,
present = cond.right,
wrapped = e.then,
missing = e.else_;
if (state.kind !== "arrayState" || present.kind !== "numLit" || present.value !== 1) return null;
if (wrapped.kind !== "unionWrap" || missing.kind !== "unionWrap") return null;
const value = wrapped.value;
if (value.kind !== "arrayGet" || missing.value.kind !== "unitLit") return null;
const same = (x: IrExpr, y: IrExpr): boolean =>
x.kind === "varRef" && y.kind === "varRef" && x.localId === y.localId;
return same(state.arr, value.arr) && same(state.index, value.index)
? { arr: state.arr, index: state.index }
: null;
}
/** The present-undefined stores of `a[i] = b[j]!` as lowering spells it,
* with the element read each one stores the missing arm of:
*
* const v = %arr.idxOr(b, j);
* if (v is undefined) a[i] = undefined; else a[i] = v is number ? v : throw;
*
* The read's local is declared once in the same list as the `if`, and only
* read after it. */
function missingElementStores(body: IrStmt[]): Map<IrStmt, ElementRead> {
const result = new Map<IrStmt, ElementRead>();
const writes = new Map<string, number>();
const count = (id: string): void => void writes.set(id, (writes.get(id) ?? 0) + 1);
const expr = (e: IrExpr): boolean => {
if (e.kind === "assignExpr" || e.kind === "incDec") count(e.localId);
return everyExprChild(e, expr, stmt);
};
const stmt = (s: IrStmt): boolean => {
if (s.kind === "varDecl" || s.kind === "assign") count(s.localId);
return everyStmtChild(s, expr, stmt);
};
body.every(stmt);
const list = (stmts: IrStmt[]): void => {
const reads = new Map<string, ElementRead>();
for (const s of stmts) {
if (s.kind === "varDecl" && s.init && writes.get(s.localId) === 1) {
const read = elementRead(s.init);
if (read) reads.set(s.localId, read);
}
if (s.kind === "if") {
const store = missingArmStore(s, reads);
if (store) result.set(store.stmt, store.read);
}
// Nested loops keep a loop out of the fast copy (matchDenseArrayLoop),
// and other statement lists leave their stores unmatched.
if (s.kind === "block") list(s.body);
if (s.kind === "if") {
list(s.then);
if (s.else_) list(s.else_);
}
}
};
list(body);
return result;
}
/** The present-undefined store of `if (v is undefined) a[i] = undefined;
* else a[i] = v is number ? v : throw;`, with the read `v` holds. */
function missingArmStore(
s: IrStmt & { kind: "if" },
reads: ReadonlyMap<string, ElementRead>,
): { stmt: IrStmt; read: ElementRead } | null {
const cond = s.cond;
if (cond.kind !== "unionIsTag" || cond.negated || cond.value.kind !== "varRef") return null;
const local = cond.value.localId;
const read = reads.get(local);
if (!read || s.then.length !== 1 || s.else_?.length !== 1) return null;
const store = s.then[0]!,
other = s.else_[0]!;
if (store.kind !== "arraySetUndefined" || other.kind !== "arraySet") return null;
const value = other.value;
if (value.kind !== "ternary") return null;
const test = value.cond,
narrowed = value.then;
if (test.kind !== "unionIsTag" || test.negated || test.tag === cond.tag) return null;
if (test.value.kind !== "varRef" || test.value.localId !== local) return null;
if (narrowed.kind !== "unionNarrow" || narrowed.tag !== test.tag) return null;
return narrowed.type.kind === "f64" ? { stmt: store, read } : null;
}
/** Match the fast copy of a loop of the given shape. `initialized` holds the
* bindings definitely assigned when the loop starts; `unaliased` the local
* arrays no other binding holds. */
@@ -118,6 +256,8 @@ export function matchDenseArrayLoop(
initialized: ReadonlySet<string>,
): DenseArrayLoop | null {
const { counter, length } = shape;
const high = shape.pair?.high ?? null;
const isCounter = (id: string): boolean => id === counter || id === high;
if (shape.range.min < 0 || !loop.cond) return null;
let nodes = 0,
nested = false;
@@ -126,13 +266,21 @@ export function matchDenseArrayLoop(
// element write.
const disturbed = new Set<string>();
const access = (arr: IrExpr, index: IrExpr): void => {
if (arr.kind === "varRef" && index.kind === "varRef" && index.localId === counter)
if (arr.kind === "varRef" && index.kind === "varRef" && isCounter(index.localId))
candidates.set(arr.localId, arr);
};
const missing = missingElementStores(loop.body);
// The element reads behind the guarded present-undefined stores.
const guards: ElementRead[] = [];
// The arrays of the other present-undefined stores. The fast copy runs
// them, and the slot each one writes leaves the hole-free prefix.
const undefinedStores: IrExpr[] = [];
const expr = (e: IrExpr): boolean => {
if (++nodes > MAX_LOOP_NODES) return false;
if (e.kind === "assignExpr" || e.kind === "incDec") disturbed.add(e.localId);
if (e.kind === "arrayGet") access(e.arr, e.index);
const reader = elementReaderCall(e);
if (reader) access(reader.arr, reader.index);
if (e.kind === "arrIntrinsic") {
if (e.method === "getNumber" && e.args[0]) access(e.receiver, e.args[0]);
else if (e.method !== "length" && e.receiver.kind === "varRef")
@@ -147,9 +295,21 @@ export function matchDenseArrayLoop(
if (s.kind === "assign" || s.kind === "varDecl") disturbed.add(s.localId);
if (s.kind === "arraySet") {
access(s.arr, s.index);
if (s.arr.kind === "varRef" && (s.index.kind !== "varRef" || s.index.localId !== counter))
if (s.arr.kind === "varRef" && (s.index.kind !== "varRef" || !isCounter(s.index.localId)))
disturbed.add(s.arr.localId);
}
if (s.kind === "arraySetUndefined") {
// A present undefined at a counter only where it stores a missing
// element read (`a[i] = b[j]!`): the fast copy reads a present value
// there, so the store never runs in it.
const read = missing.get(s);
if (read === undefined || s.index.kind !== "varRef" || !isCounter(s.index.localId)) {
undefinedStores.push(s.arr);
} else {
access(s.arr, s.index);
guards.push(read);
}
}
return everyStmtChild(s, expr, stmt);
};
if (!loop.body.every(stmt) || !expr(loop.cond) || nested) return null;
@@ -164,6 +324,15 @@ export function matchDenseArrayLoop(
)
)
return null;
// A store through another name, such as `const b = a` or a field, lands
// in `a` unless one of the two bindings is a local array no other binding
// holds.
const mayStoreUndefined = (id: string): boolean =>
undefinedStores.some((arr) =>
arr.kind === "varRef"
? arr.localId === id || (!unaliased.has(id) && !unaliased.has(arr.localId))
: !unaliased.has(id),
);
const arrays = new Map<string, IrExpr & { kind: "varRef" }>();
for (const [id, ref] of candidates) {
const local = locals.get(id);
@@ -173,6 +342,7 @@ export function matchDenseArrayLoop(
local.boxed ||
local.tdz ||
disturbed.has(id) ||
mayStoreUndefined(id) ||
(id !== length?.localId && !initialized.has(id))
)
continue;
@@ -180,18 +350,33 @@ export function matchDenseArrayLoop(
// once: the loop must not change any of their lengths.
if (
length === null &&
!preservesArrayLength(nodesOfLoop, { receiver: id, inRange: counter }, locals, unaliased)
!preservesArrayLength(
nodesOfLoop,
{ receiver: id, inRange: counter, inRangeHigh: high },
locals,
unaliased,
)
)
continue;
arrays.set(id, ref);
}
if (arrays.size === 0) return null;
// Each guarded store's read must be one the fast copy proves present.
for (const { arr, index } of guards)
if (
arr.kind !== "varRef" ||
!arrays.has(arr.localId) ||
index.kind !== "varRef" ||
!isCounter(index.localId)
)
return null;
return {
counter,
length,
limit: length === null ? (shape.limit ?? null) : null,
arrays,
range: { min: shape.range.min, max: Math.min(shape.range.max, INT32_MAX - 1) },
pair: shape.pair ?? null,
};
}
@@ -207,6 +392,7 @@ export function emitDenseLoopGuard(host: LlvmEmitterContext, loop: DenseArrayLoo
host.markMemoryPointer(ptr, "array:header");
return ptr;
};
if (loop.pair !== null) return emitPairedDenseGuard(host, loop, loop.pair, header);
if (loop.length === null) {
// A limit at or below the start runs no pass, and passes the test.
const limit = emitCountedLoopLimit(host, loop.limit!);
@@ -254,3 +440,36 @@ export function emitDenseLoopGuard(host: LlvmEmitterContext, loop: DenseArrayLoo
}
return valid;
}
/** A two-counter loop's entry test: both starts are exact integers, the
* low one at least 0 and the high one at most 2^31 - 2 and below every
* indexed array's hole-free prefix bound. The starts are stable values
* (matchIntegerPairedForLoop), so evaluating them here again is unobservable. */
function emitPairedDenseGuard(
host: LlvmEmitterContext,
loop: DenseArrayLoop,
pair: DensePair,
header: (array: string, field: number) => string,
): string {
const B = host.B;
const start = (expr: IrExpr, min: number, max: number) =>
emitExactIntegerTest(host, host.emitExpr(expr).name, min, max);
const low = start(pair.lowStart, 0, INT32_MAX - 1);
const high = start(pair.highStart, -1, INT32_MAX - 1);
let valid = B.tmp();
B.line(`${valid} = and i1 ${low.ok}, ${high.ok}`);
for (const ref of loop.arrays.values()) {
const array = host.emitStableReceiver(ref, []);
const filledPtr = header(array.name, SCR_ARR_FIELD.filled),
filled = B.tmp(),
wide = B.tmp(),
covered = B.tmp(),
both = B.tmp();
B.line(`${filled} = load i32, ptr ${filledPtr}${host.fieldAliasAttachment(filledPtr)}`);
B.line(`${wide} = zext i32 ${filled} to i64`);
B.line(`${covered} = icmp slt i64 ${high.integer}, ${wide}`);
B.line(`${both} = and i1 ${valid}, ${covered}`);
valid = both;
}
return valid;
}
@@ -52,13 +52,19 @@ import {
matchIntegerPairedForLoop,
} from "../../ir/integer-loops.js";
import { emitByteWindowGuard, matchByteWindow } from "./byte-windows.js";
import { countedDenseShape, emitDenseLoopGuard, matchDenseArrayLoop } from "./dense-array-loops.js";
import {
countedDenseShape,
emitDenseLoopGuard,
matchDenseArrayLoop,
pairedDenseShape,
} from "./dense-array-loops.js";
import {
emitCountedLoopLimit,
emitCountedLoopSafety,
emitPairedLoopSafety,
} from "./counted-loops.js";
import { emitBytesLoopVersionGuard, type BytesLoopVersion } from "./bytes-loop-versions.js";
import { bytesHeaderAlias } from "./bytes-alias.js";
import { mangleLocal } from "../mangle.js";
import { LlvmUnsupportedError } from "./unsupported.js";
import type { LlValue } from "./expr-context.js";
@@ -966,7 +972,9 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void {
}
: countedLoop && !countedLoop.guarded && countedLoop.followers.length === 0
? countedDenseShape(s, countedLoop)
: null;
: pairedLoop
? pairedDenseShape(pairedLoop)
: null;
const dense =
denseShape && !window
? matchDenseArrayLoop(
@@ -1043,15 +1051,17 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void {
LOOPS.of(emitter).unaliasedArrays,
)
: countedLoop;
const version = versionable ? emitter.matchBytesLoopVersion(s, versionCounted) : null;
const version = versionable
? emitter.matchBytesLoopVersion(s, versionCounted, versionCounted ? null : pairedLoop)
: null;
if (version) {
emitVersionedLoop(emitter, s, version);
break;
}
let countedJoin: string | null = null;
if (
(countedLoop?.guarded && !LOOPS.of(emitter).provenCountedGuards.has(s)) ||
pairedLoop?.guarded
(countedLoop?.guarded || pairedLoop?.guarded) &&
!LOOPS.of(emitter).provenCountedGuards.has(s)
) {
const safe = countedLoop
? emitCountedLoopSafety(emitter, countedLoop)
@@ -1144,11 +1154,15 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void {
B.line(`${start} = fptosi double ${value.name} to i64`);
B.line(`store i64 ${start}, ptr ${slot}`);
pairedSlots.push(slot);
const dense = LOOPS.of(emitter).denseLoop;
emitter.integerLoopBindings.set(counter.localId, {
slot,
type: "i64",
signed: true,
range: pairedLoop.range,
range:
dense?.pair && dense.counter === pairedLoop.low.localId
? dense.range
: pairedLoop.range,
});
}
} else if (s.init) {
@@ -1218,10 +1232,7 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void {
}
B.line(`${inBounds} = icmp ult i32 ${narrow}, ${len}`);
} else {
emitter.markMemoryPointer(lenPtr, "bytes:header");
B.line(
`${len} = load ${emitter.sizeType}, ptr ${lenPtr}${emitter.fieldAliasAttachment(lenPtr)}`,
);
B.line(`${len} = load ${emitter.sizeType}, ptr ${lenPtr}${bytesHeaderAlias(emitter)}`);
B.line(`${inBounds} = icmp ult ${emitter.sizeType} ${index}, ${len}`);
}
B.condBr(inBounds, lb, exit);
@@ -2085,7 +2096,8 @@ function emitVersionedLoop(
if (version.owned.size > 0) emitter.bytesOwned = new Set([...originalOwned, ...version.owned]);
if (version.ranges.size > 0)
emitter.integerRanges = new Map([...originalRanges, ...version.ranges]);
if (version.counted?.guarded) LOOPS.of(emitter).provenCountedGuards.add(s);
if (version.counted?.guarded || version.paired?.loop.guarded)
LOOPS.of(emitter).provenCountedGuards.add(s);
emitter.emitStmt(s);
B.br(done);
} finally {
@@ -499,6 +499,27 @@ test("class brand probes compare the program's key constant before the key text"
const named = work(probe("dyn.typedRefIs"), dyn);
expect(named).toContain("; ->v.typed_ref.type_key");
expect(named).toContain("@scr_dyn_typed_ref_is_key");
// Another key pointer is rejected on its length and last eight bytes
// ("Example" plus the prefix's ':') before the out-of-line comparison.
expect(named).toContain("; ->v.typed_ref.type_key_len");
expect(named).toContain("and i32 %t");
expect(named).toMatch(/icmp eq i32 %t\d+, 14\b/);
const tail = Buffer.from("object:Example").readBigInt64LE(6);
expect(named).toContain(`icmp eq i64 %t`);
expect(named).toContain(`, ${tail}`);
const shortKey: IrExpr = { kind: "strLit", value: "o:E", type: STRING, loc: receiverLoc };
const short = work(
{
kind: "libCall",
fn: "dyn.typedRefIs",
args: [ref(dyn), shortKey],
type: BOOL,
loc: receiverLoc,
},
dyn,
);
expect(short).toMatch(/icmp eq i32 %t\d+, 3\b/);
expect(short).not.toContain("load i64");
// The key-pointer test answers false for capsules boxed elsewhere.
const exact = work(probe("dyn.typedRefKeyIs"), dyn);
expect(exact).toContain("; ->v.typed_ref.type_key");
+95 -28
View File
@@ -60,7 +60,7 @@ import { captureFreeFunctions, observesStacks } from "../stack-observations.js";
import { findInitializedByteLoopBindings } from "./byte-windows.js";
import { matchBytesLoopVersion } from "./bytes-loop-versions.js";
import { globalSlotAlias } from "./bytes-alias.js";
import type { IntegerCountedForLoop } from "../../ir/integer-loops.js";
import type { IntegerCountedForLoop, IntegerPairedForLoop } from "../../ir/integer-loops.js";
import {
materializeSplitPiece,
type SplitSpan,
@@ -68,7 +68,7 @@ import {
type StringSplit,
} from "./split-loops.js";
import { scalarizeNumericRecords } from "../../ir/scalar-records.js";
import { addRecordResultEntries } from "./record-results.js";
import { addRecordResultEntries, resultEntryName } from "./record-results.js";
import { specializeCallbackCalls } from "../../ir/callback-specialization.js";
import {
type ChunkResult,
@@ -82,6 +82,7 @@ import { versionLoopBoundParameters } from "../../ir/parameter-versions.js";
import {
allocatedLiteral,
fillAllocationLength,
int32ArrayKey,
selectInt32Arrays,
} from "../../ir/int32-arrays.js";
import { everyExpr, everyExprChild, everyStmtChild, everyStmtList } from "../../ir/traverse.js";
@@ -185,7 +186,7 @@ import {
} from "./string-slices.js";
import { LazyCaptures } from "./lazy-captures.js";
import { StackCaptures } from "./stack-captures.js";
import { emitFieldAbsentTest, f64Lit, SCR_ARR_FIELD } from "./common.js";
import { emitFieldAbsentTest, f64Lit, SCR_ARR_FIELD, SCR_ARR_FROZEN } from "./common.js";
import { ffiExtendsNarrowIntegers } from "../targets.js";
import {
emitLiteralExpr,
@@ -513,6 +514,15 @@ function provenInt32Arrays(mod: IrModule): ReadonlySet<string> {
return new Set([...slots.int32Arrays, ...slots.checkedInt32Arrays]);
}
/** The functions whose calls the backend may emit as calls of their
* result-slot entries (record-results.ts). */
function resultEntryTargets(mod: IrModule): ReadonlySet<string> {
const names = new Set(mod.functions.map((fn) => fn.name));
const targets = new Set<string>();
for (const name of names) if (names.has(resultEntryName(name))) targets.add(name);
return targets;
}
/** The emitter for `input` after the module passes every emission runs. */
function moduleEmitter(input: IrModule, options: LlvmTargetOptions): LlEmitter {
// Facts are keyed by node identity: emit the same module whether it was
@@ -548,6 +558,7 @@ function moduleEmitter(input: IrModule, options: LlvmTargetOptions): LlEmitter {
),
),
provenInt32Arrays,
resultEntryTargets,
)
: mod,
options,
@@ -784,7 +795,7 @@ export class LlEmitter {
everyStmtList(fn.body, {
stmt: (s) => {
if (s.kind !== "varDecl" && s.kind !== "assign") return true;
const key = `${fn.name}\0${s.localId}`;
const key = int32ArrayKey(fn.name, s.localId);
const checked = checkedInt32Arrays.has(key);
if (!checked && !int32Arrays.has(key)) return true;
const value = s.kind === "varDecl" ? s.init : s.value;
@@ -802,15 +813,16 @@ export class LlEmitter {
});
}
/** Whether `arr` is one of the current function's int32 arrays
* (ir/int32-arrays.ts, kept by the int32 slot analysis). */
/** Whether `arr` reads one of the int32 arrays (ir/int32-arrays.ts, kept
* by the int32 slot analysis): a local or parameter of the current
* function, or a module global. */
int32Array(arr: IrExpr): boolean {
if (arr.kind !== "varRef") return false;
if (this.int32KindArray(arr)) return true;
if (this.int32Slots.int32Arrays.size === 0) return false;
return (
this.int32ArrayTemps.has(arr.localId) ||
this.int32Slots.int32Arrays.has(`${this.currentFunctionName}\0${arr.localId}`)
this.int32Slots.int32Arrays.has(int32ArrayKey(this.currentFunctionName, arr.localId))
);
}
@@ -977,8 +989,8 @@ export class LlEmitter {
}
/** Whether an element access at `index` of `arr` lies in the hole-free
* prefix: the counter of a dense array loop's fast copy indexing one of
* the arrays its entry test covered. */
* prefix: a counter of a dense array loop's fast copy indexing one of the
* arrays its entry test covered. */
provenDenseIndex(arr: IrExpr, index: IrExpr): boolean {
const loop = LOOPS.of(this).denseLoop;
return (
@@ -986,8 +998,8 @@ export class LlEmitter {
arr.kind === "varRef" &&
loop.arrays.has(arr.localId) &&
index.kind === "varRef" &&
index.localId === loop.counter &&
this.integerLoopBindings.has(loop.counter)
(index.localId === loop.counter || index.localId === loop.pair?.high) &&
this.integerLoopBindings.has(index.localId)
);
}
@@ -4256,6 +4268,7 @@ export class LlEmitter {
matchBytesLoopVersion(
s: IrStmt & { kind: "for" | "while" },
counted: IntegerCountedForLoop | null,
paired: IntegerPairedForLoop | null = null,
): ReturnType<typeof matchBytesLoopVersion> {
const fn = LOOPS.of(this).bytesLoopFunction;
if (!fn) return null;
@@ -4265,17 +4278,22 @@ export class LlEmitter {
const invariantGlobals = new Set(this.stableGlobals);
for (const global of this.initializerBindings.get(fn.name) ?? [])
invariantGlobals.add(global.id);
return matchBytesLoopVersion(s, counted, {
locals: this.currentLocals,
invariantGlobals,
captured: this.captureIds,
ranges: this.integerRanges,
initialized,
proven: this.bytesBounds,
workers: this.mod.workers === true,
owned: this.bytesOwned,
scalarStrings: new Set(this.stringSlices.keys()),
});
return matchBytesLoopVersion(
s,
counted,
{
locals: this.currentLocals,
invariantGlobals,
captured: this.captureIds,
ranges: this.integerRanges,
initialized,
proven: this.bytesBounds,
workers: this.mod.workers === true,
owned: this.bytesOwned,
scalarStrings: new Set(this.stringSlices.keys()),
},
paired,
);
}
/** The alias tag of a whole-slot load of a binding (module globals only). */
@@ -6284,6 +6302,41 @@ export class LlEmitter {
return ok;
}
/** Branches on "the array `arr` is frozen" (SCR_ARR_FROZEN in its storage
* flags, scr_array.h), or published too when `published`; like
* emitPublishedTest, leaves the builder in the cold block and answers the
* continuation label. */
private emitFrozenArrayTest(arr: string, published: boolean): string {
const B = this.B;
const sparep = B.tmp(),
spare = B.tmp(),
bit = B.tmp();
let frozen = B.tmp();
B.line(
`${sparep} = getelementptr inbounds %ScrArr, ptr ${arr}, i32 0, i32 ${SCR_ARR_FIELD.spare}`,
);
B.line(`${spare} = load i16, ptr ${sparep}`);
B.line(`${bit} = and i16 ${spare}, ${SCR_ARR_FROZEN}`);
B.line(`${frozen} = icmp ne i16 ${bit}, 0`);
if (published) {
const rc = B.tmp(),
immortal = B.tmp(),
either = B.tmp();
B.line(`${rc} = load ${this.sizeType}, ptr ${arr}`);
B.line(`${immortal} = icmp eq ${this.sizeType} ${rc}, -1`);
B.line(`${either} = or i1 ${frozen}, ${immortal}`);
frozen = either;
}
this.declare(`declare i1 @llvm.expect.i1(i1, i1)`);
const expected = B.tmp();
B.line(`${expected} = call i1 @llvm.expect.i1(i1 ${frozen}, i1 false)`);
const cold = B.newLabel("arr.frozen"),
ok = B.newLabel("arr.ok");
B.condBr(expected, cold, ok);
B.startBlock(cold);
return ok;
}
private publishedNamesSym: string | null = null;
/** The class-name table the hierarchy field guards index by preorder. */
@@ -6348,15 +6401,29 @@ export class LlEmitter {
op: number,
index = f64Lit(0),
count = f64Lit(0),
third: string | null = null,
): void {
if (this.publishedTypes?.containerGuarded(arrType) !== true) return;
const published = this.publishedTypes;
if (published?.containerGuarded(arrType) !== true) return;
const B = this.B;
const ok = this.emitPublishedTest(arr);
this.declare(`declare zeroext i1 @scr_throw_published_array(i32, ptr, double, double)`);
const guards = published.arrayGuards(arrType);
const ok = guards.frozen
? this.emitFrozenArrayTest(arr, guards.published)
: this.emitPublishedTest(arr);
const threw = B.tmp();
B.line(
`${threw} = call zeroext i1 @scr_throw_published_array(i32 ${op}, ptr ${arr}, double ${index}, double ${count})`,
);
if (third === null) {
this.declare(`declare zeroext i1 @scr_throw_published_array(i32, ptr, double, double)`);
B.line(
`${threw} = call zeroext i1 @scr_throw_published_array(i32 ${op}, ptr ${arr}, double ${index}, double ${count})`,
);
} else {
this.declare(
`declare zeroext i1 @scr_throw_published_array_range(i32, ptr, double, double, double)`,
);
B.line(
`${threw} = call zeroext i1 @scr_throw_published_array_range(i32 ${op}, ptr ${arr}, double ${index}, double ${count}, double ${third})`,
);
}
const unwind = B.newLabel("pub.throw");
B.condBr(threw, unwind, ok);
B.startBlock(unwind);
@@ -44,6 +44,7 @@ import { LlvmUnsupportedError } from "./unsupported.js";
import type { LlvmEmitterContext, ExprOf, LlValue } from "./expr-context.js";
import { f64Lit, ffiNativeTypeLl, ffiNativeParamLl, ffiNativeReturnLl } from "./common.js";
import { canStackUnion, emitStackUnion } from "./stack-unions.js";
import { emitStackRecordKeyGet, scalarRecordKeyGet } from "./expr-records.js";
import { emitCallArrayRead } from "./local-array-reads.js";
import { emitScalarOptional, isScalarOptionalSource } from "./local-union-storage.js";
import { emitStackMapRead } from "./map-read-lifetimes.js";
@@ -166,6 +167,9 @@ export function emitCallExpr(
return { name: host.emitOwnedNullableStack(a).box, type: a.type };
// An inline union field read copies its tag and bits.
if (host.inlineFieldGet(a)) return host.emitReadReceiver(a);
// A scalar keyed record read writes its tag and bits.
if (a.kind === "recordKeyGet" && scalarRecordKeyGet(host, a))
return { name: emitStackRecordKeyGet(host, a), type: a.type };
}
if (inputs[index]) return host.emitReadReceiver(a);
}
@@ -257,6 +257,16 @@ function emitDiscarded(host: LlvmEmitterContext, e: IrExpr): void {
const start = host.emitExpr(e.args[0]!);
const count = e.args[1] ? host.emitExpr(e.args[1]).name : F64_INF;
const values = items.map((item) => host.emitExpr(item));
// Before the values move into the buffer: a throw releases them with the
// frame.
host.emitPublishedArrayGuard(
r.name,
e.receiver.type,
10,
start.name,
count,
f64Lit(values.length),
);
if (acc === "ref") values.forEach((value) => host.toHeap(value));
let buffer = "null";
if (values.length > 0) {
@@ -978,7 +988,25 @@ export function emitArrIntrinsic(
case "fill":
case "fillUndefined": {
const args = e.args.map((arg) => host.emitExpr(arg));
host.emitPublishedArrayGuard(r.name, e.receiver.type, 7);
// fill's range is its last two arguments; copyWithin's target,
// start and end are its three.
if (e.method === "copyWithin")
host.emitPublishedArrayGuard(
r.name,
e.receiver.type,
9,
args[0]!.name,
args[1]!.name,
args[2]!.name,
);
else
host.emitPublishedArrayGuard(
r.name,
e.receiver.type,
8,
args[args.length - 2]!.name,
args[args.length - 1]!.name,
);
if (host.checkedFills.has(e)) {
// A new int32 array with checked stores, filled with a value that
// may not be an int32: widen it first then.
@@ -1042,7 +1070,7 @@ export function emitArrIntrinsic(
// removes to the end (+Infinity, the slice convention).
const start = host.emitExpr(e.args[0]!);
const cnt = e.args[1] ? host.emitExpr(e.args[1]).name : F64_INF;
host.emitPublishedArrayGuard(r.name, e.receiver.type, 7);
host.emitPublishedArrayGuard(r.name, e.receiver.type, 10, start.name, cnt, f64Lit(0));
host.declare(`declare ptr @scr_arr_splice(ptr, double, double)`);
const t = B.tmp();
B.line(`${t} = call ptr @scr_arr_splice(ptr ${r.name}, double ${start.name}, double ${cnt})`);
@@ -1052,7 +1080,12 @@ export function emitArrIntrinsic(
const start = host.emitExpr(e.args[0]!);
const count = host.emitExpr(e.args[1]!);
const items = emitBorrowedInput(host, e.args[2]!);
host.emitPublishedArrayGuard(r.name, e.receiver.type, 7);
if (host.publishedTypes?.containerGuarded(e.receiver.type) === true) {
host.declare(`declare double @scr_arr_len(ptr)`);
const n = B.tmp();
B.line(`${n} = call double @scr_arr_len(ptr ${items.name})`);
host.emitPublishedArrayGuard(r.name, e.receiver.type, 10, start.name, count.name, n);
}
host.declare(`declare ptr @scr_arr_splice_insert(ptr, double, double, ptr)`);
const t = B.tmp();
B.line(
@@ -1076,9 +1076,10 @@ export function emitContainerExpr(
const idx = host.emitExpr(e.index);
if (e.arr.type.kind !== "array")
throw new InternalCompilerError("llvm emitter bug: arrayState on non-array");
const proven = host.provenDenseIndex(e.arr, e.index);
const t = host.checkedInt32Array(e.arr)
? host.emitByElementKind(arr.name, "double", (kind) =>
emitDenseArrayState(host, arr.name, idx, e.index, "state", slotWidth(kind)),
emitDenseArrayState(host, arr.name, idx, e.index, "state", slotWidth(kind), proven),
)!
: emitDenseArrayState(
host,
@@ -1087,6 +1088,7 @@ export function emitContainerExpr(
e.index,
"state",
host.int32Array(e.arr) ? slotWidth("i32") : denseSlotWidth(host, e.arr.type.elem),
proven,
);
return { name: t, type: e.type };
}
@@ -7,6 +7,7 @@ import type { LlvmEmitterContext, LlValue } from "./expr-context.js";
import { f64Lit, llvmCommentText } from "./common.js";
import { borrowableInputs, emitBorrowedInputs, emitBorrowedInput } from "./borrowed-inputs.js";
import { regexLiteralTemplate } from "./expr-primitives.js";
import { isScalarUnionDef, loadInlineUnion, presentBits } from "./inline-unions.js";
import { preservesRegexInputs } from "./reference-effects.js";
/** Widen one tagged read. The source is borrowed unless the overflow map
@@ -365,6 +366,114 @@ export function emitRecordKeyGet(
return host.own({ name: t, type: e.type });
}
/** A keyed read whose result is a scalar union (`r[k]` on a
* `Record<string, number>`, at `number | undefined`) and whose every hit
* is a number or boolean arm of it, or an inline field of the same union.
* Projections (`??`, `!`, tag tests) read it as a private stack box, so
* the read allocates nothing. */
export function scalarRecordKeyGet(host: LlvmEmitterContext, e: IrExpr): boolean {
if (e.kind !== "recordKeyGet" || e.missNaN === true || e.type.kind !== "union") return false;
const resultType = e.type;
const def = host.unionsById.get(resultType.unionId);
if (!isScalarUnionDef(def) || undefinedArmTag(resultType, host.unionsById) < 0) return false;
const isArm = (t: IrType): boolean =>
(t.kind === "f64" || t.kind === "bool") && def!.arms.some((a) => typeEquals(a, t));
const shape = host.recordShape(e.shapeId);
if (e.overflowOnly !== true) {
for (const f of shape.fields) {
const inline = host.recordFieldInline(e.shapeId, f.name);
if (!isArm(f.type) && !(inline && typeEquals(f.type, resultType))) return false;
}
}
return shape.indexValue === undefined || isArm(shape.indexValue);
}
/** A scalarRecordKeyGet read as a private stack box: the tag and bits of
* the declared field or overflow entry the key names, or the undefined arm
* for a miss. The box must never reach an RC entry point. */
export function emitStackRecordKeyGet(
host: LlvmEmitterContext,
e: IrExpr & { kind: "recordKeyGet" },
): string {
const B = host.B;
if (e.type.kind !== "union")
throw new InternalCompilerError("llvm emitter bug: stack keyed read of a non-union");
const resultType = e.type;
const def = host.unionsById.get(resultType.unionId)!;
const armTag = (t: IrType): number => def.arms.findIndex((a) => typeEquals(a, t));
const obj = host.emitExpr(e.obj);
const key = host.emitExpr(e.key);
const box = B.slot();
B.entryAllocas.push(`${box} = alloca %ScrUnion`);
const tagPtr = B.tmp(),
bitsPtr = B.tmp();
B.line(`store ${host.sizeType} -1, ptr ${box}`);
B.line(`${tagPtr} = getelementptr inbounds %ScrUnion, ptr ${box}, i64 0, i32 1`);
B.line(`${bitsPtr} = getelementptr inbounds %ScrUnion, ptr ${box}, i64 0, i32 5`);
const join = B.newLabel("rks.j");
const storeScalar = (t: IrType, value: string): void => {
const bits = B.tmp();
if (t.kind === "f64") B.line(`${bits} = bitcast double ${value} to i64`);
else B.line(`${bits} = zext i1 ${value} to i64`);
B.line(`store i32 ${armTag(t)}, ptr ${tagPtr}`);
B.line(`store i64 ${bits}, ptr ${bitsPtr}`);
};
const shape = host.recordShape(e.shapeId);
for (const f of e.overflowOnly === true ? [] : shape.fields) {
const hit = B.tmp();
B.line(
`${hit} = call zeroext i1 ${host.literalEquals(f.name)}(ptr ${key.name}) ; ${llvmCommentText(f.name)}`,
);
const lh = B.newLabel("rks.h");
const ln = B.newLabel("rks.n");
B.condBr(hit, lh, ln);
B.startBlock(lh);
const { ptr, type, inline } = host.recordFieldPtr(obj.name, e.shapeId, f.name);
if (inline) {
const { tag, bits } = loadInlineUnion(B, ptr);
const present = presentBits(B, tag, bits, undefinedArmTag(type, host.unionsById));
B.line(`store i32 ${tag}, ptr ${tagPtr}`);
B.line(`store i64 ${present}, ptr ${bitsPtr}`);
} else {
storeScalar(type, host.loadRecordField(ptr, type, false));
}
B.br(join);
B.startBlock(ln);
}
const iv = shape.indexValue;
if (iv) {
const ovf = host.recordOvfPtr(obj.name, e.shapeId);
const outTy = iv.kind === "f64" ? "double" : "i8";
const outSlot = B.slot();
B.entryAllocas.push(`${outSlot} = alloca ${outTy}`);
B.line(`store ${outTy} ${iv.kind === "f64" ? f64Lit(0) : "0"}, ptr ${outSlot}`);
const getter = `scr_map_get_str_${iv.kind === "f64" ? "f64" : "bool"}`;
host.declare(`declare zeroext i1 @${getter}(ptr, ptr, ptr)`);
const found = B.tmp();
B.line(`${found} = call zeroext i1 @${getter}(ptr ${ovf}, ptr ${key.name}, ptr ${outSlot})`);
const lh = B.newLabel("rks.h");
const ln = B.newLabel("rks.n");
B.condBr(found, lh, ln);
B.startBlock(lh);
const raw = B.tmp();
B.line(`${raw} = load ${outTy}, ptr ${outSlot}`);
let value = raw;
if (iv.kind === "bool") {
value = B.tmp();
B.line(`${value} = trunc i8 ${raw} to i1`);
}
storeScalar(iv, value);
B.br(join);
B.startBlock(ln);
}
// A miss is the undefined arm.
B.line(`store i32 ${undefinedArmTag(resultType, host.unionsById)}, ptr ${tagPtr}`);
B.line(`store i64 0, ptr ${bitsPtr}`);
B.br(join);
B.startBlock(join);
return box;
}
export function keyedRecordReadInto(
host: LlvmEmitterContext,
slot: string,
@@ -25,6 +25,9 @@ import { rawBytes } from "./lib-abi.js";
import { emitInlineAtomicsOp } from "./expr-bytes.js";
import { regexLiteralCaptures } from "./regex-matchers.js";
/** ScrDyn.v.typed_ref.type_key_len: the low 27 bits of its bit-field word. */
const TYPE_KEY_LEN_MASK = (1 << 27) - 1;
export function emitAssertInspectLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue {
const B = host.B;
if (e.fn === "assert.shapeStr" || e.fn === "assert.shapeRe") {
@@ -281,6 +284,38 @@ export function emitGenericLibCall(host: LlvmEmitterContext, e: LibCallExpr): Ll
B.line(`store i1 true, ptr ${slot}`);
B.condBr(same, done, compare);
B.startBlock(compare);
// A capsule with another key pointer usually belongs to another
// class: reject it on its key length and last eight bytes before
// the out-of-line name comparison, so a class table's chain of
// probes costs a few instructions per class it skips.
B.line(`store i1 false, ptr ${slot}`);
const bytes = Buffer.from(literal, "utf8");
const lenSlot = B.tmp(),
lenWord = B.tmp(),
len = B.tmp(),
lenEq = B.tmp();
B.line(
`${lenSlot} = getelementptr inbounds i8, ptr ${value.name}, i64 ${host.abiOffset(88, 52)} ; ->v.typed_ref.type_key_len`,
);
B.line(`${lenWord} = load i32, ptr ${lenSlot}`);
B.line(`${len} = and i32 ${lenWord}, ${TYPE_KEY_LEN_MASK}`);
B.line(`${lenEq} = icmp eq i32 ${len}, ${bytes.length}`);
const full = B.newLabel("brand.full");
if (bytes.length >= 8) {
const tail = B.newLabel("brand.tail");
B.condBr(lenEq, tail, done);
B.startBlock(tail);
const tailPtr = B.tmp(),
word = B.tmp(),
wordEq = B.tmp();
B.line(`${tailPtr} = getelementptr inbounds i8, ptr ${keyPtr}, i64 ${bytes.length - 8}`);
B.line(`${word} = load i64, ptr ${tailPtr}, align 1`);
B.line(`${wordEq} = icmp eq i64 ${word}, ${bytes.readBigInt64LE(bytes.length - 8)}`);
B.condBr(wordEq, full, done);
} else {
B.condBr(lenEq, full, done);
}
B.startBlock(full);
}
}
if (e.fn === "dyn.typedRefIs") {
@@ -805,6 +840,19 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue {
return { name: t, type: e.type };
}
if (e.fn === "bytes.absent") return host.own({ name: "null", type: e.type });
if (e.fn === "record.isFrozen") {
// The frozen bit of the record's rc word (scr_rc.h SCR_RC_FROZEN) is
// its second-highest bit; an immortal word sets the highest one too.
const value = host.emitReadReceiver(e.args[0]!);
const S = host.sizeType;
const rc = host.B.tmp(),
top = host.B.tmp(),
t = host.B.tmp();
host.B.line(`${rc} = load ${S}, ptr ${value.name}`);
host.B.line(`${top} = lshr ${S} ${rc}, ${S === "i64" ? 62 : 30}`);
host.B.line(`${t} = icmp eq ${S} ${top}, 1`);
return { name: t, type: e.type };
}
const prefix = e.fn.slice(0, e.fn.indexOf(".")) as LibCallPrefix;
// events.once builds typed listener adapters beside the emitter calls.
if (e.fn === "events.once" || e.fn === "events.onceSignal")
@@ -913,6 +961,7 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue {
case "blob":
case "wasm":
case "array":
case "collection":
case "record":
case "disposable":
case "punycode":
@@ -958,6 +958,7 @@ export const LIB_FN_SYMS: Record<string, string> = {
"fetch.streamIs": "scr_fetch_stream_is",
"zlib.streamIs": "scr_zlib_stream_is",
"fetch.streamNew": "scr_fetch_stream_new",
"fetch.transformNew": "scr_fetch_transform_new",
"fetch.streamFrom": "scr_fetch_stream_from",
"fetch.readerRead": "scr_fetch_reader_read",
"json.parse": "scr_json_parse",
@@ -1021,6 +1022,12 @@ export const LIB_FN_SYMS: Record<string, string> = {
"class.descriptorHasValue": "scr_class_descriptor_has_value",
"class.throwReadOnly": "scr_class_throw_read_only",
"class.throwNotExtensible": "scr_class_throw_not_extensible",
"record.freeze": "scr_record_freeze",
"record.throwFrozen": "scr_record_throw_frozen",
"array.freeze": "scr_arr_freeze",
"array.isFrozen": "scr_arr_is_frozen",
"collection.preventExtensions": "scr_map_prevent_extensions",
"collection.isExtensible": "scr_map_is_extensible",
"dyn.nativeSetNew": "scr_dyn_native_set_new",
"dyn.nativeSetIs": "scr_dyn_native_set_is",
"dyn.nativeMapIs": "scr_dyn_native_map_is",
@@ -1693,6 +1700,7 @@ export const USES_TIMERS_LIB_FNS = new Set<string>([
"fetch.abortAny",
"fetch.streamNew",
"fetch.streamFrom",
"fetch.transformNew",
"readable.new",
// fs.ReadStream / fs.WriteStream open, read and write on the worker pool.
"fs.createReadStream",
@@ -797,7 +797,33 @@ export function emitLocalArrayRead(
);
return null;
});
else if (inline)
else if (
inline &&
loopIndex !== null &&
read.element.kind === "f64" &&
host.provenDenseIndex(read.array, read.index)
) {
// In a dense array loop's fast copy, the entry test proved the counter
// below the array's hole-free prefix: the slot holds a present value,
// and the read has no missing arm.
const dataPtr = B.tmp(),
data = B.tmp(),
valuePtr = B.tmp(),
value = B.tmp(),
width = int32 ? "i32" : "i64";
B.line(
`${dataPtr} = getelementptr inbounds %ScrArr, ptr ${array.name}, i32 0, i32 ${SCR_ARR_FIELD.data}`,
);
host.markMemoryPointer(dataPtr, "array:header");
B.line(`${data} = load ptr, ptr ${dataPtr}${host.fieldAliasAttachment(dataPtr)}`);
B.line(
`${valuePtr} = getelementptr inbounds ${width}, ptr ${data}, ${host.sizeType} ${loopIndex}`,
);
host.markMemoryPointer(valuePtr, "array:elements");
B.line(`${value} = load ${width}, ptr ${valuePtr}${host.fieldAliasAttachment(valuePtr)}`);
storeValue(value);
B.br(join);
} else if (inline)
emitDenseReferenceArrayRead(
host,
array,
@@ -18,6 +18,7 @@ import { canStackUnion, emitStackUnion } from "./stack-unions.js";
import { emitDowncastGuard, emitUnionNarrowGuard } from "./narrow-guards.js";
import { emitScalarOptional, isScalarOptionalSource } from "./local-union-storage.js";
import { borrowableInputs } from "./borrowed-inputs.js";
import { emitStackRecordKeyGet, scalarRecordKeyGet } from "./expr-records.js";
import { type WalkBorrowHost, analyzeWalks } from "./walk-borrows.js";
/** Read a receiver whose value is consumed immediately by a field or tag
@@ -95,6 +96,8 @@ export function emitReadReceiver(emitter: LlEmitter, e: IrExpr): LlValue {
const narrowed = narrow ? emitCheckedNarrowReceiver(emitter, narrow) : null;
if (narrowed) return narrowed;
if (canStackUnion(e, emitter.boxedUnionsById)) return emitStackUnion(emitter, e).value;
if (e.kind === "recordKeyGet" && scalarRecordKeyGet(emitter, e))
return { name: emitStackRecordKeyGet(emitter, e), type: e.type };
if (e.kind === "unionNarrow") {
const union = emitter.emitUnionProjection(e.value);
emitUnionNarrowGuard(emitter, union.name, e.unionId, e.tag);
@@ -410,6 +413,7 @@ export function isStackUnionSource(emitter: LlEmitter, e: IrExpr): boolean {
emitter.optionalArrayReads.get(e) !== null ||
isScalarOptionalSource(e, emitter.unionSources) ||
emitter.scalarResultCall(e) ||
emitter.inlineFieldGet(e)
emitter.inlineFieldGet(e) ||
scalarRecordKeyGet(emitter, e)
);
}
@@ -14,6 +14,7 @@ import {
type ShapeHost,
vAdapters,
} from "./shapes.js";
import { mangleRecordRelease } from "../mangle.js";
import { NullableUnions } from "./nullable-unions.js";
import { InlineUnionFields } from "./inline-unions.js";
@@ -315,6 +316,42 @@ describe("emitted object releases", () => {
root.indexOf("call void @scr_cyc_on_release(ptr %o)"),
);
});
test("freezable records count their references without the frozen bit", () => {
const mod: IrModule = {
irVersion: 17,
sourceFile: "frozen.ts",
entry: "main",
functions: [],
records: [
{ id: "plain", fields: [{ name: "n", type: F64 }] },
{ id: "frozen", fields: [{ name: "n", type: F64 }], freezable: true },
],
};
const body = (defs: string, name: string): string =>
new RegExp(`^define internal void @${name}\\([^]*?^}`, "m").exec(defs)?.[0] ?? "";
for (const [pointerBits, notFrozen, sharedBelow] of [
[64, "-4611686018427387905", "-4611686018427387907"],
[32, "-1073741825", "-1073741827"],
] as const) {
const { host } = declarationHost(pointerBits);
const defs = emitRecordShapes({ ...host, objectAudit: false }, mod).defs.join("\n");
const S = pointerBits === 64 ? "i64" : "i32";
const plain = body(defs, mangleRecordRelease("plain"));
expect(plain).toContain(`%shared = icmp ult ${S} %rcm2, -3`);
expect(plain).not.toContain("%refs");
// The immortal word's count is ~FROZEN, which the compare sends to the
// immortal test, like counts 0 and 1.
const fast = body(defs, mangleRecordRelease("frozen"));
expect(fast).toContain(`%refs = and ${S} %rc, ${notFrozen}`);
expect(fast).toContain(`%rcm2 = sub ${S} %refs, 2`);
expect(fast).toContain(`%shared = icmp ult ${S} %rcm2, ${sharedBelow}`);
expect(fast).toContain(`%imm = icmp eq ${S} %rc, -1`);
const slow = body(defs, `${mangleRecordRelease("frozen")}_slow`);
expect(slow).toContain(`%refs = and ${S} %n, ${notFrozen}`);
expect(slow).toContain(`%dead = icmp eq ${S} %refs, 0`);
}
});
});
describe("live-object audit notes", () => {
+249 -28
View File
@@ -34,6 +34,7 @@ import {
cycTypeDescriptor,
emitObjectAlloc,
emitObjectFree,
emitSegmentFree,
CYC_APPEND,
cycleAppendDecls,
cycleAppendHelper,
@@ -1405,17 +1406,23 @@ export interface DirectDestroy {
/** The release of child `value` (a loaded field, possibly NULL) inside a
* teardown whose chain length is `%d`: the child's own release for a shared
* or immortal reference, its teardown for the last one. Labels use `tag`;
* the lines end in the open block `${tag}.next`. */
* the lines end in the open block `${tag}.next`.
*
* With `segment` (the child has the teardown's own type, in a segmented
* teardown; segmentPrologueLines), the child's teardown appends its blocks
* to the teardown's segment. */
export function directChildReleaseLines(
host: ShapeHost,
tag: string,
value: string,
target: DirectDestroy,
comment: string,
segment = false,
): string[] {
const S = host.sizeType;
const C = target.count ?? S;
host.declare(`declare void @scr_rc_destroy(ptr, ptr)`);
if (segment) host.declare(`declare ptr @scr_rc_destroy_seg(ptr, ptr, ptr, ptr)`);
else host.declare(`declare void @scr_rc_destroy(ptr, ptr)`);
const lines = [
` %${tag}.nn = icmp ne ptr ${value}, null`,
` br i1 %${tag}.nn, label %${tag}.chk, label %${tag}.next`,
@@ -1447,16 +1454,139 @@ export function directChildReleaseLines(
` br i1 %${tag}.short, label %${tag}.direct, label %${tag}.bounded`,
`${tag}.direct:`,
` %${tag}.d = add i32 %d, 1`,
` call void @${target.destroy}_d(ptr ${value}, i32 %${tag}.d)`,
);
if (!segment) {
lines.push(
` call void @${target.destroy}_d(ptr ${value}, i32 %${tag}.d)`,
` br label %${tag}.next`,
`${tag}.bounded:`,
` call void @scr_rc_destroy(ptr ${value}, ptr @${target.destroy})`,
` br label %${tag}.next`,
`${tag}.next:`,
);
return lines;
}
lines.push(
` %${tag}.t = load ptr, ptr %sg.t`,
` %${tag}.dt = call ptr @${target.destroy}_s(ptr ${value}, i32 %${tag}.d, ptr %${tag}.t)`,
` store ptr %${tag}.dt, ptr %sg.t`,
` br label %${tag}.next`,
`${tag}.bounded:`,
` call void @scr_rc_destroy(ptr ${value}, ptr @${target.destroy})`,
` %${tag}.bt = load ptr, ptr %sg.t`,
` %${tag}.rt = call ptr @scr_rc_destroy_seg(ptr ${value}, ptr @${target.destroy}_sc, ptr @${target.destroy}, ptr %${tag}.bt)`,
` store ptr %${tag}.rt, ptr %sg.t`,
` br label %${tag}.next`,
`${tag}.next:`,
);
return lines;
}
/* Teardown segments. A typed teardown that frees each block onto its free
* list as it goes frees a tree in walk order, and the free lists, which are
* LIFO, hand the blocks to the next allocations in the reverse order: a tree
* built again in the same order as the last (each generation of a structure
* a loop rebuilds) is laid out mirrored, and every walk over it, the next
* teardown included, runs through memory backwards, which defeats the
* hardware prefetchers. The runtime frees large dead sets of the cycle
* collector in address order for the same reason (scr_mem_order_for_free).
*
* So the teardown of a type with children of its own type (the `_s` form,
* segmentedTeardown) keeps the blocks it frees in a segment, linked through
* their first words, which it builds in order through a tail pointer: the
* `_s` form takes the segment's last link (the first word of its last block,
* or the head slot of an empty segment) and returns the new one. A node
* first releases its own block (emitObjectFree, keeping it), then appends it
* ahead of its children's blocks when its address is the lower, behind them
* otherwise (segmentTeardownLines), and the children append theirs in field
* order, so a structure
* built parent first (object literals allocate before their fields) or
* children first (constructor arguments) comes back in ascending order. The
* chain's first teardown (`_d`) frees the whole segment onto the class's
* free list in one step (emitSegmentFree), and the walks over the next
* generation run forward. A teardown past DIRECT_DESTROY_CHAIN appends
* through scr_rc_destroy_seg, under the same depth budget as
* scr_rc_destroy. Blocks outside the calling thread's arena are freed as
* before; one type's blocks are all of one class. */
/** Whether the teardown of a type whose reference fields have `fieldTypes`
* (with direct targets `targets`) and whose own teardown is `destroy` keeps
* a segment: a child field has its own type, and the inline allocation paths
* are in use. */
export function segmentedTeardown(
host: ShapeHost,
destroy: string,
fieldTypes: readonly IrType[],
targets: ReadonlyMap<string, DirectDestroy>,
): boolean {
if (host.inlineAlloc !== true) return false;
return fieldTypes.some((t) => directDestroyTarget(host, t, targets)?.destroy === destroy);
}
/** The first lines of a segmented teardown, in its entry block: the
* segment's last link (`%sg.t`, from the `%tail` argument) and the kept own
* block (`%sg.own`, for emitObjectFree). */
export function segmentPrologueLines(): string[] {
return [
` %sg.t = alloca ptr`,
` %sg.own = alloca ptr`,
` store ptr %tail, ptr %sg.t`,
` store ptr null, ptr %sg.own`,
];
}
/** The rest of a segmented teardown after emitObjectFree kept (or freed)
* its own block: the block goes ahead of its children's when its address is
* below the first non-NULL one of `children` (loaded fields of the
* teardown's own type, which stand for their blocks) or there is none, and
* behind them otherwise; then the children's releases (`releases`, given a
* label prefix), and the segment's last link is returned. Each order has its
* own copy of the releases, so neither keeps the decision, or the block once
* it is appended, across the children's teardowns. */
export function segmentTeardownLines(
children: readonly string[],
releases: (prefix: string) => string[],
): string[] {
const lines = [` %sg.o = load ptr, ptr %sg.own`, ` %sg.kept = icmp ne ptr %sg.o, null`];
let ref = "null";
[...children].reverse().forEach((child, i) => {
lines.push(
` %sg.nn${i} = icmp ne ptr ${child}, null`,
` %sg.r${i} = select i1 %sg.nn${i}, ptr ${child}, ptr ${ref}`,
);
ref = `%sg.r${i}`;
});
lines.push(
` %sg.leaf = icmp eq ptr ${ref}, null`,
` %sg.lower = icmp ult ptr %sg.o, ${ref}`,
` %sg.first = or i1 %sg.leaf, %sg.lower`,
` %sg.ahead = and i1 %sg.kept, %sg.first`,
` br i1 %sg.ahead, label %sg.front, label %sg.back`,
`sg.front:`,
...segmentAppendLines("a"),
...releases("ta"),
` br label %sg.ret`,
`sg.back:`,
...releases("tb"),
` %sg.keep = icmp ne ptr %sg.o, null`,
` br i1 %sg.keep, label %sg.last, label %sg.ret`,
`sg.last:`,
...segmentAppendLines("l"),
` br label %sg.ret`,
`sg.ret:`,
` %sg.tr = load ptr, ptr %sg.t`,
` ret ptr %sg.tr`,
);
return lines;
}
function segmentAppendLines(tag: string): string[] {
return [
` %sg.t${tag} = load ptr, ptr %sg.t`,
` store ptr %sg.o, ptr %sg.t${tag}`,
` store ptr %sg.o, ptr %sg.t`,
];
}
/** The direct-teardown target of a reference of `type`, if `targets` (keyed
* `record:<id>` / `object:<class>`) has one. Nullable unions hold the arm's
* pointer, NULL or an immortal sentinel, which the child release handles. */
@@ -1499,12 +1629,42 @@ export function usedDirectTargets(
/** The teardown definitions for `destroy`: with `chained`, the body takes
* the chain length `%d` (directChildReleaseLines) and the runtime's
* one-argument entry starts a chain. */
* one-argument entry starts a chain. With `segment` (the host, for a
* segmented teardown), the body is a segment teardown's
* (segmentPrologueLines to segmentTeardownLines), defined as `_s`, with
* `_sc` for scr_rc_destroy_seg, and `_d` starts a segment and frees it. */
export function destroyDefinitions(
destroy: string,
body: readonly string[],
chained: boolean,
segment: ShapeHost | null = null,
): string[] {
if (segment !== null)
return [
`define internal ptr @${destroy}_s(ptr %o, i32 %d, ptr %tail) ${FN_ATTRS} {`,
`entry:`,
...body,
`}`,
`define internal ptr @${destroy}_sc(ptr %o, ptr %tail) ${FN_ATTRS} {`,
`entry:`,
` %t = call ptr @${destroy}_s(ptr %o, i32 0, ptr %tail)`,
` ret ptr %t`,
`}`,
`define internal void @${destroy}_d(ptr %o, i32 %d) ${FN_ATTRS} {`,
`entry:`,
` %head = alloca ptr`,
` store ptr null, ptr %head`,
` %last = call ptr @${destroy}_s(ptr %o, i32 %d, ptr %head)`,
` %first = load ptr, ptr %head`,
...emitSegmentFree(segment, "sgf", "%first", "%last"),
` ret void`,
`}`,
`define internal void @${destroy}(ptr %o) ${FN_ATTRS} {`,
`entry:`,
` call void @${destroy}_d(ptr %o, i32 0)`,
` ret void`,
`}`,
];
if (!chained)
return [
`define internal void @${destroy}(ptr %o) ${FN_ATTRS} {`,
@@ -1527,6 +1687,17 @@ export function destroyDefinitions(
];
}
/** The frozen bit of a freezable record's rc word (scr_rc.h SCR_RC_FROZEN):
* bit 62 on 64-bit targets, bit 30 on 32-bit ones. */
function recordFrozenBit(sizeType: RcCount): bigint {
return 1n << (sizeType === "i64" ? 62n : 30n);
}
/** `value` as a signed LLVM integer literal of `sizeType`. */
function sizeLiteral(sizeType: RcCount, value: bigint): string {
return BigInt.asIntN(sizeType === "i64" ? 64 : 32, value).toString();
}
/** Common NULL/immortal/decrement skeleton for ordinary object releases.
* `freeBody` owns the zero-ref teardown and must leave the current block at
* its end; traced objects also get the possible-cycle-root branch.
@@ -1543,13 +1714,17 @@ export function releaseBody(
bounded = true,
immortal = false,
chained = false,
segmented = false,
frozen = false,
): string[] {
const S = host.sizeType;
if (bounded) host.declare(`declare void @scr_rc_destroy(ptr, ptr)`);
const destroy = `${fnName}_destroy`;
// A freezable record's count is the word without its frozen bit.
const notFrozen = sizeLiteral(S, ~recordFrozenBit(S));
const lines = [
...(bounded ? destroyDefinitions(destroy, freeBody, chained) : []),
...releaseFastPath(host, fnName, `${fnName}_slow`, traced, comment, immortal),
...(bounded ? destroyDefinitions(destroy, freeBody, chained, segmented ? host : null) : []),
...releaseFastPath(host, fnName, `${fnName}_slow`, traced, comment, immortal, S, frozen),
`define internal void @${fnName}_slow(ptr %o) noinline ${FN_ATTRS} {${comment ? ` ; ${comment}` : ""}`,
`entry:`,
` %isnull = icmp eq ptr %o, null`,
@@ -1561,7 +1736,9 @@ export function releaseBody(
`dec:`,
` %n = sub ${S} %rc, 1`,
` store ${S} %n, ptr %o`,
` %dead = icmp eq ${S} %n, 0`,
...(frozen
? [` %refs = and ${S} %n, ${notFrozen}`, ` %dead = icmp eq ${S} %refs, 0`]
: [` %dead = icmp eq ${S} %n, 0`]),
` br i1 %dead, label %free, label %${traced ? "root" : "done"}`,
`free:`,
...(traced ? cycleDeadLines(host, "teardown") : []),
@@ -1591,9 +1768,14 @@ export function releaseFastPath(
comment = "",
immortal = false,
count: RcCount = host.sizeType,
frozen = false,
): string[] {
const S = count;
if (immortal) return immortalRcBody(host, fnName, "void", comment);
// A freezable record compares its count without the frozen bit: an
// immortal word's count is then ~FROZEN, which the same compare sends to
// `slow`.
const notFrozen = ~recordFrozenBit(S);
return [
`define internal void @${fnName}(ptr %o) ${FN_ATTRS} {${comment ? ` ; ${comment} (fast path)` : ""}`,
`entry:`,
@@ -1601,8 +1783,13 @@ export function releaseFastPath(
` br i1 %isnull, label %done, label %check`,
`check:`,
` %rc = load ${S}, ptr %o`,
` %rcm2 = sub ${S} %rc, 2`,
` %shared = icmp ult ${S} %rcm2, -3`,
...(frozen
? [
` %refs = and ${S} %rc, ${sizeLiteral(S, notFrozen)}`,
` %rcm2 = sub ${S} %refs, 2`,
` %shared = icmp ult ${S} %rcm2, ${sizeLiteral(S, notFrozen - 2n)}`,
]
: [` %rcm2 = sub ${S} %rc, 2`, ` %shared = icmp ult ${S} %rcm2, -3`]),
` br i1 %shared, label %dec, label %slow`,
`dec:`,
` %n = sub ${S} %rc, 1`,
@@ -1780,26 +1967,58 @@ export function emitRecordShapes(
const chained =
directTargets.has(`record:${shape.id}`) ||
refMembers.some((m) => directDestroyTarget(host, m.type, directTargets) !== null);
for (const m of refMembers) {
freeBody.push(
` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${m.index}`,
` %v${t} = load ptr, ptr %f${t}`,
const destroy = `${mangleRecordRelease(shape.id)}_destroy`;
const bounded = refMembers.some((member) => needsBoundedRelease(member.type));
const segmented =
bounded &&
segmentedTeardown(
host,
destroy,
refMembers.map((m) => m.type),
directTargets,
);
const direct = directDestroyTarget(host, m.type, directTargets);
if (direct) {
freeBody.push(
...directChildReleaseLines(host, `td${t}`, `%v${t}`, direct, llvmCommentText(m.name)),
);
} else {
freeBody.push(
` call void ${releaseSym(host, m.type)}(ptr %v${t}) ; ${llvmCommentText(m.name)}`,
);
}
t++;
}
// Closed layouts can carry a kept record source (record-sources.ts).
const observers = recordMayKeepSource(shape) ? "weak-or-source" : "weak";
freeBody.push(...emitObjectFree(host, traced, observers));
// A segmented teardown loads its fields, releases its own block, and
// then releases the fields (segmentTeardownLines).
const loads: string[] = [];
const ownType: string[] = [];
const release = (prefix: string, m: (typeof refMembers)[number], i: number): string[] => {
const direct = directDestroyTarget(host, m.type, directTargets);
if (direct === null)
return [` call void ${releaseSym(host, m.type)}(ptr %v${i}) ; ${llvmCommentText(m.name)}`];
const own = segmented && direct.destroy === destroy;
return directChildReleaseLines(
host,
`${prefix}${i}`,
`%v${i}`,
direct,
llvmCommentText(m.name),
own,
);
};
for (const m of refMembers) {
const fieldLines = [
` %f${t} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${m.index}`,
` %v${t} = load ptr, ptr %f${t}`,
];
if (segmented) {
loads.push(...fieldLines);
if (directDestroyTarget(host, m.type, directTargets)?.destroy === destroy)
ownType.push(`%v${t}`);
} else freeBody.push(...fieldLines, ...release("td", m, t));
t++;
}
if (segmented)
freeBody.push(
...segmentPrologueLines(),
...loads,
...emitObjectFree(host, traced, observers, "%sg.own"),
...segmentTeardownLines(ownType, (prefix) =>
refMembers.flatMap((m, i) => release(prefix, m, i)),
),
);
else freeBody.push(...emitObjectFree(host, traced, observers));
defs.push(
...releaseBody(
host,
@@ -1807,9 +2026,11 @@ export function emitRecordShapes(
traced,
freeBody,
"",
refMembers.some((member) => needsBoundedRelease(member.type)),
bounded,
false,
chained,
segmented,
shape.freezable === true,
),
``,
);
@@ -44,8 +44,9 @@ export type StackObjectFieldOwner = (fields: readonly StackObjectField[]) => voi
* standalone scalar class whose constructor never lets `this` escape
* (record-results.ts). The caller proves that every use of the value only
* projects it. Record fields may own references; the frame releases them
* (stackObjectFields). Published shapes keep heap storage: their field
* guards treat the immortal header as a frozen object. */
* (stackObjectFields). Published and freezable shapes keep heap storage:
* their field guards treat the immortal header as a frozen object, and a
* frozen record keeps its frozen bit in a counted header. */
export function canStackObject(host: LlvmEmitterContext, e: IrExpr): e is StackObjectSource {
// A ThreadSlots key keeps its slot index for its lifetime and has its
// values released when it is freed: it lives on the heap.
@@ -53,6 +54,8 @@ export function canStackObject(host: LlvmEmitterContext, e: IrExpr): e is StackO
return false;
if (e.type.kind === "record" && host.publishedTypes?.recordGuarded(e.type.shapeId) === true)
return false;
if (e.type.kind === "record" && host.recordsById.get(e.type.shapeId)?.freezable === true)
return false;
// A producer with a result-slot entry builds its fresh result in the
// caller's storage.
if (e.kind === "call") return host.fnByName.has(resultEntryName(e.callee));
@@ -85,3 +85,34 @@ test("published types close over fields, containers and whole hierarchies", () =
computePublishedTypes({ ...withStores, functions: withStores.functions.slice(1) }),
).toBeNull();
});
test("freezable array types carry the array write guard without a publish", () => {
const numbers = arrayOf(F64);
const push: IrStmt = {
kind: "exprStmt",
expr: {
kind: "arrIntrinsic",
method: "push",
receiver: ref("a", numbers),
args: [{ kind: "numLit", value: 1, type: F64, loc }],
type: F64,
loc,
},
loc,
};
const mod: IrModule = {
irVersion: 17,
sourceFile: loc.file,
entry: "main",
functions: [fn("main", []), fn("pushNumber", [push])],
freezableArrays: ["array<f64>"],
};
const guarded = computePublishedTypes(mod)!;
expect(guarded.containerGuarded(numbers)).toBe(true);
expect(guarded.arrayGuards(numbers)).toEqual({ published: false, frozen: true });
expect(guarded.containerGuarded(arrayOf(STRING))).toBe(false);
// A frozen array's push throws, so the function may throw.
expect(computeMayThrow(mod).fns.has("pushNumber")).toBe(true);
const { freezableArrays: _frozen, ...plain } = mod;
expect(computePublishedTypes(plain)).toBeNull();
});
+22 -5
View File
@@ -46,6 +46,10 @@ export class PublishedTypes {
readonly records = new Set<string>();
/** typeKeys of array, map and set types. */
readonly containers = new Set<string>();
/** typeKeys of the array types the program may freeze
* (IrModule.freezableArrays): a frozen array refuses writes like a
* published one, so their stores carry the same guard. */
readonly frozenArrays = new Set<string>();
/** Constructor stores into a `this` nothing else can reach yet
* (immutability.ts): never a published target, so never guarded. */
privateStores: ReadonlySet<IrStmt> = new Set();
@@ -64,10 +68,16 @@ export class PublishedTypes {
}
containerGuarded(t: IrType): boolean {
return (
(t.kind === "array" || t.kind === "map" || t.kind === "set") &&
this.containers.has(typeKey(t))
);
if (t.kind !== "array" && t.kind !== "map" && t.kind !== "set") return false;
const key = typeKey(t);
return this.containers.has(key) || this.frozenArrays.has(key);
}
/** Whether an array of type `t` may be published, and whether it may be
* frozen: the two states its write guard tests. */
arrayGuards(t: IrType): { published: boolean; frozen: boolean } {
const key = typeKey(t);
return { published: this.containers.has(key), frozen: this.frozenArrays.has(key) };
}
}
@@ -87,7 +97,13 @@ export function computePublishedTypes(mod: IrModule, nodes?: IrNodeIndex): Publi
)
roots.push(node.type);
});
if (roots.length === 0) return null;
const frozenArrays = mod.freezableArrays ?? [];
if (roots.length === 0 && frozenArrays.length === 0) return null;
if (roots.length === 0) {
const frozen = new PublishedTypes();
for (const key of frozenArrays) frozen.frozenArrays.add(key);
return frozen;
}
const classes = new Map((mod.classes ?? []).map((c) => [c.name, c]));
const children = new Map<string, string[]>();
@@ -100,6 +116,7 @@ export function computePublishedTypes(mod: IrModule, nodes?: IrNodeIndex): Publi
const records = new Map((mod.records ?? []).map((r) => [r.id, r]));
const unions = new Map((mod.unions ?? []).map((u) => [u.id, u]));
const out = new PublishedTypes();
for (const key of frozenArrays) out.frozenArrays.add(key);
out.privateStores = computeConstructionFacts(mod, nodes).privateStores;
const seen = new Set<string>();
const stack = [...roots];
@@ -900,7 +900,7 @@ export const NODE24_FETCH_COMPAT_PROFILE = {
{
name: "WritableStream/TransformStream",
reason:
"native support currently covers readable fetch bodies; general Web Streams graphs are unsupported",
"native support covers readable fetch bodies and the identity `new TransformStream()`; the WritableStream constructor, transformers, and general Web Streams graphs are unsupported",
},
{
name: "Readable byte/BYOB streams",
@@ -15,6 +15,7 @@ import {
typeKey,
unionContainerArmsOk,
} from "../../ir/ir.js";
import { everyExpr } from "../../ir/traverse.js";
import type { Lowerer } from "./lowerer.js";
import type * as ts from "../ts7/adapter.js";
@@ -193,6 +194,81 @@ export function unionArrayValueRead(
return result;
}
/** Expression kinds that evaluate without running other code or writing
* a binding. */
const PURE_KINDS = new Set<IrExpr["kind"]>([
"varRef",
"numLit",
"strLit",
"boolLit",
"unitLit",
"bin",
"unary",
"logical",
"ternary",
"toBool",
"fieldGet",
"recordGet",
"arrayGet",
"arrayState",
"arrayHas",
"unionIsTag",
"unionNarrow",
"unionWrap",
]);
/** The locals evaluating `value` may assign, and whether it may run other
* code (anything but the pure kinds and the read-only element helpers),
* which can assign locals and fields through closures. */
function valueEffects(lowerer: Lowerer, value: IrExpr): { writes: Set<string>; opaque: boolean } {
const readers = new Set<string>();
for (const [key, name] of lowerer.arrHofHelpers) if (key.startsWith("idxOr:")) readers.add(name);
const writes = new Set<string>();
let opaque = false;
everyExpr(value, {
expr: (e) => {
if (e.kind === "assignExpr" || e.kind === "incDec") writes.add(e.localId);
else if (e.kind === "call" ? !readers.has(e.callee) : !PURE_KINDS.has(e.kind)) opaque = true;
return true;
},
stmt: (s) => {
if (s.kind === "varDecl" || s.kind === "assign") writes.add(s.localId);
else opaque = true;
return true;
},
});
return { writes, opaque };
}
/** Whether evaluating `e` after `value` gives what it gives before it: a
* literal, or a local and field read `value` cannot change. */
function unaffectedBy(e: IrExpr, effects: { writes: Set<string>; opaque: boolean }): boolean {
switch (e.kind) {
case "numLit":
case "strLit":
return true;
case "varRef":
return !effects.writes.has(e.localId);
case "bin":
return unaffectedBy(e.left, effects) && unaffectedBy(e.right, effects);
case "fieldGet":
case "recordGet":
return !effects.opaque && unaffectedBy(e.obj, effects);
default:
return false;
}
}
/** Whether `e` reads a `let` or `var` binding, which code the value runs
* can assign through a closure. */
function readsMutableLocal(lowerer: Lowerer, e: IrExpr): boolean {
return !everyExpr(e, {
expr: (x) =>
x.kind !== "varRef" || lowerer.ctx.locals.find((l) => l.id === x.localId)?.mutable === false,
stmt: () => true,
});
}
/** Store a stabilized value without changing the array's element ABI. */
export function arrayValueStore(
lowerer: Lowerer,
@@ -210,10 +286,31 @@ export function arrayValueStore(
// owned union temporary to release on every path.
const unionId = value.type.unionId;
let stable = value;
let prefix: IrStmt[] = [];
const prefix: IrStmt[] = [];
// The stores below evaluate the receiver and the index after the value.
// JavaScript evaluates them first (`out[n++] = src[n]` reads src[1]), so
// either may need its value held before the value runs.
if (value.kind !== "varRef") {
const valueFx = valueEffects(lowerer, value);
const hold = (e: IrExpr, name: string, effects: typeof valueFx): IrExpr => {
if (unaffectedBy(e, effects) && !(effects.opaque && readsMutableLocal(lowerer, e)))
return e;
const temp = lowerer.declareHiddenLocal(name, e.type);
prefix.push({ kind: "varDecl", localId: temp.id, init: e, loc });
return varRef(temp.id, temp.type, loc);
};
// The receiver runs before the index and the value, so the index's
// effects can change it too (`a[(a = b, 0)] = c[0]!`).
const indexFx = valueEffects(lowerer, index);
arr = hold(arr, "%arrayTarget", {
writes: new Set([...valueFx.writes, ...indexFx.writes]),
opaque: valueFx.opaque || indexFx.opaque,
});
index = hold(index, "%arrayIndex", valueFx);
}
if (value.kind !== "varRef") {
const temp = lowerer.declareHiddenLocal("%arrayValue", value.type);
prefix = [{ kind: "varDecl", localId: temp.id, init: value, loc }];
prefix.push({ kind: "varDecl", localId: temp.id, init: value, loc });
stable = varRef(temp.id, temp.type, loc);
}
const write: IrStmt = {
@@ -282,6 +282,11 @@ interface PropertyDispatch {
fn: IrFunction;
classes: Set<string>;
reflect?: boolean;
/** The helper's fallthrough statements as created, and the same access
* applied to the receiver itself: what the fallthrough reduces to for a
* value that is neither a class capsule nor a function, whose bag is the
* value. */
plain?: { tail: readonly IrStmt[]; access: IrStmt[] };
}
export function classInstanceOf(
@@ -2724,7 +2729,28 @@ export class ClassDynamicDispatch {
loc,
});
}
dispatch = { name, write: !!write, fn: helper, classes: new Set() };
const receiverAccess: IrStmt[] = read
? [{ kind: "return", value: { ...read, value: varRef("p.0", DYN, loc) }, loc }]
: [
{
kind: "exprStmt",
expr: {
...write!,
args: [varRef("p.0", DYN, loc), write!.args[1]!, varRef("p.1", DYN, loc)],
},
loc,
},
{ kind: "return", value: null, loc },
];
dispatch = {
name,
write: !!write,
fn: helper,
classes: new Set(),
...(name !== "constructor" && name !== "toString"
? { plain: { tail: [...helper.body], access: receiverAccess } }
: {}),
};
this.properties.set(key, dispatch);
this.generated.add(helper);
lowerer.liftedFns.push(helper);
@@ -4896,6 +4922,7 @@ export class ClassDynamicDispatch {
);
}
if (lowerer) this.guardClassTables(lowerer);
if (lowerer) this.guardPropertyHelpers(lowerer);
}
/** Each tracked class's bag initializer over a typed receiver (`this`),
@@ -4985,6 +5012,87 @@ export class ClassDynamicDispatch {
}
}
/** Fast exits for the generated property helpers: when every class
* branch tests a capsule key or a class key and the fallthrough is still
* the bag access the helper was created with, a value of another kind
* (a plain object, array or primitive) normalizes to itself, has itself
* as its bag, and so gets the plain access directly, without the
* normalization, bag and static-bag calls. */
private guardPropertyHelpers(lowerer: Lowerer): void {
// Every class table the fallthrough or normalization calls must answer
// a plain value with the value itself: its kind exit is in place.
const exits = (name: string): boolean => {
const helper = lowerer.liftedFnNamed(name);
const first = helper?.body[0];
const param = helper?.params[0]?.localId;
return (
param !== undefined &&
first?.kind === "if" &&
first.cond.kind === "dynTest" &&
first.cond.negated === true &&
first.then.length === 1 &&
first.then[0]!.kind === "return" &&
first.then[0]!.value !== null &&
returnsParam(first.then[0]!.value, param)
);
};
const tables = new Set([
"%dyn.class.properties",
READ_PROPERTIES,
READ_PROPERTIES_OR_EMPTY,
"%dyn.class.normalize",
"%dyn.class.staticProperties",
]);
const callsOnlyGuardedTables = (statements: IrStmt[]): boolean =>
everyStmtList(statements, {
stmt: () => true,
expr: (expression) =>
expression.kind !== "call" || (tables.has(expression.callee) && exits(expression.callee)),
});
for (const dispatch of this.properties.values()) {
const plain = dispatch.plain;
if (!plain) continue;
const fn = dispatch.fn;
const param = fn.params[0]?.localId;
if (param === undefined) continue;
const head = fn.body.length - plain.tail.length;
if (head < 0 || plain.tail.some((stmt, i) => fn.body[head + i] !== stmt)) continue;
let ok = true;
for (const stmt of fn.body.slice(0, head)) {
if (
stmt.kind === "assign" &&
stmt.localId === param &&
stmt.value.kind === "call" &&
stmt.value.callee === "%dyn.class.normalize"
)
continue;
if (stmt.kind === "if" && stmt.else_ === null && branchKind(stmt.cond, param) !== null)
continue;
ok = false;
break;
}
// Class branches never run for a plain value; the rest always does.
if (!ok || !callsOnlyGuardedTables(fn.body.filter((stmt) => stmt.kind !== "if"))) continue;
const loc = fn.loc;
const not = (test: "classCapsule" | "funcKind"): IrExpr => ({
kind: "dynTest",
test,
negated: true,
value: varRef(param, DYN, loc),
type: BOOL,
loc,
});
fn.body.unshift({
kind: "if",
cond: not("classCapsule"),
then: [{ kind: "if", cond: not("funcKind"), then: plain.access, else_: null, loc }],
else_: null,
loc,
});
delete dispatch.plain;
}
}
private normalizedDispatches(): IrFunction[] {
return [
...(this.propertyBag ? [this.propertyBag] : []),
@@ -1028,7 +1028,7 @@ function lowerGuardedRecordWrite(
const value = stabilized(lowerer, lowerer.lowerExpr(expr.right), "%recordValue", receiver.stmts);
const write = lowerer.withExpressionOverride(target.expression, receiver.present, () =>
lowerer.withExpressionOverride(target.argumentExpression, key, () =>
lowerer.withExpressionOverride(expr.right, value, () => lowerElementWrite(lowerer, expr)),
lowerer.withExpressionOverride(expr.right, value, () => lowerer.lowerElementWrite(expr)),
),
);
return {
@@ -1499,6 +1499,25 @@ export function lowerPropertyAccessExpr(
) {
return afterReceiver(recvLowered, dynUndefinedExpr(loc), loc);
}
// A typed array under a checker `any` (a JS binding its own initializer's
// inference refers to: hono's etag digest `nextChunk`) reads through its
// box, as the method-call twin dispatches. The box shares the native
// storage, so it needs no fresh-result restriction.
if (
recvLowered.type.kind === "bytes" &&
(lowerer.typeOf(expr.expression).flags & ts.TypeFlags.Any) !== 0
) {
return lowerer.maybeNarrow(
{
kind: "dynKeyGet",
value: lowerer.coerceToExpected(recvLowered, DYN),
key: { kind: "strLit", value: expr.name.text, type: STRING, loc },
type: DYN,
loc,
},
expr,
);
}
lowerer.unsupported(
"SC1090",
expr,
@@ -2,6 +2,7 @@ import { nodeThrowExpr, varRef } from "../../../ir/build.js";
import * as ts from "../../ts7/adapter.js";
import type { Lowerer } from "../lowerer.js";
import { narrowStoredClassValue } from "../class-unions.js";
import { markSloppyRecordWrites } from "../frozen-records.js";
import {
BOOL,
type IrExpr,
@@ -246,6 +247,8 @@ export function lowerUnionFieldWrite(
value: slotValue,
loc,
};
// A sloppy-mode write to a frozen record leaves it unchanged.
if (write.kind === "recordSet") markSloppyRecordWrites(lowerer, write, access);
branch = { kind: "seqExpr", stmts: [write], result, type: result.type, loc };
} else {
branch = nodeThrowExpr(
@@ -0,0 +1,354 @@
import * as ts from "../ts7/adapter.js";
import {
BOOL,
VOID,
shapeHasAccessorSlots,
typeKey,
type IrExpr,
type IrFunction,
type IrStmt,
type IrType,
type SrcLoc,
} from "../../ir/ir.js";
import { numLit, strLit, varRef } from "../../ir/build.js";
import { everyStmtList, transformStmtList } from "../../ir/traverse.js";
import type { Lowerer } from "./lowerer.js";
import { recordHasOwnHelper } from "./lower-calls.js";
import { isStrictCode } from "./strict-mode.js";
/* Object.freeze of records and arrays that other references can still
* write. Only the types a program may freeze pay for it. A record keeps its
* frozen bit in the rc word (scr_rc.h SCR_RC_FROZEN), which its release
* ignores, and every write to it checks the bit first
* (guardFrozenWrites). An array keeps it in its storage flags
* (scr_array.h SCR_ARR_FROZEN), which the backend's array write guards test
* for the array types the module lists (IrModule.freezableArrays). A type is
* freezable when a static Object.freeze names it, or, in a program that
* freezes untyped values, when a value of it or of a type reaching it can be
* held as a live untyped reference. */
type RecordType = Extract<IrType, { kind: "record" }>;
type RecordWrite = Extract<IrStmt, { kind: "recordSet" | "recordKeySet" | "recordKeyDelete" }>;
type FreezeFn = "record.freeze" | "array.freeze" | "collection.preventExtensions";
/** Object.freeze of a record or an array, or Object.freeze, seal or
* preventExtensions of a Map or Set: `fn` sets the value's bit, and the
* call answers the value. */
export function lowerNativeFreeze(
lowerer: Lowerer,
fn: FreezeFn,
target: IrExpr,
loc: SrcLoc,
): IrExpr {
const type = target.type;
const slot = lowerer.declareHiddenLocal("%freezeTarget", type);
const receiver = (): IrExpr => varRef(slot.id, type, loc);
return {
kind: "seqExpr",
stmts: [
{ kind: "varDecl", localId: slot.id, init: target, loc },
{
kind: "exprStmt",
expr: { kind: "libCall", fn, args: [receiver()], type: VOID, loc },
loc,
},
],
result: receiver(),
type,
loc,
};
}
function isRecordWrite(s: IrStmt): s is RecordWrite {
return s.kind === "recordSet" || s.kind === "recordKeySet" || s.kind === "recordKeyDelete";
}
/** The key of a record write's source range: strict mode is lexical, so
* every write lowered from the same range shares its mode. */
function writeKey(loc: SrcLoc): string {
return `${loc.file}\0${loc.start}\0${loc.end}`;
}
/** Records the record writes of `stmt`, lowered from `node`, as sloppy-mode
* writes when `node` is sloppy-mode code: a write to a frozen record then
* leaves it unchanged instead of throwing. `deletes` says `stmt` is the
* absent-state write of `delete r.f`, which Node refuses only while the
* field is present. */
export function markSloppyRecordWrites(
lowerer: Lowerer,
stmt: IrStmt,
node: ts.Node,
deletes = false,
): IrStmt {
if (deletes) lowerer.recordFieldDeletes.add(writeKey(stmt.loc));
if (isStrictCode(node)) return stmt;
everyStmtList([stmt], {
stmt: (s) => {
if (s.kind === "recordSet" || s.kind === "recordKeySet" || s.kind === "recordKeyDelete")
lowerer.sloppyRecordWrites.add(writeKey(s.loc));
return true;
},
expr: () => true,
});
return stmt;
}
/** Whether a field write's value can be an optional field's absent state,
* which only a field write itself can hold. */
function holdsAbsentState(e: IrExpr): boolean {
if (e.kind === "fieldAbsent") return true;
return e.kind === "ternary" && (holdsAbsentState(e.then) || holdsAbsentState(e.else_));
}
/** The record shapes and array types (typeKeys) whose values the program
* may freeze. */
function freezableTypes(
lowerer: Lowerer,
functions: IrFunction[],
): { shapes: Set<string>; arrays: Set<string> } {
const shapes = new Set<string>();
const arrays = new Set<string>();
const live: IrType[] = [];
let dynFreeze = false;
for (const fn of functions)
everyStmtList(fn.body, {
expr: (e) => {
if (e.kind === "libCall" && e.fn === "record.freeze") {
const target = e.args[0];
if (target && target.type.kind === "record") shapes.add(target.type.shapeId);
} else if (e.kind === "libCall" && e.fn === "array.freeze") {
const target = e.args[0];
if (target && target.type.kind === "array") arrays.add(typeKey(target.type));
} else if (e.kind === "libCall" && e.fn === "dyn.freeze") dynFreeze = true;
else if (e.kind === "dynFrom") live.push(e.value.type);
return true;
},
stmt: () => true,
});
if (!dynFreeze) return { shapes, arrays };
// An untyped reference to a class instance, or to a record or array
// boxed live, reaches the values its fields and elements hold as further
// live references. Class instances also cross into untyped code through
// compiled callbacks, so every class is a root.
const stack = [...live];
for (const info of lowerer.classes.values())
for (const field of info.def.fields) stack.push(field.type);
const seen = new Set<string>();
while (stack.length > 0) {
const type = stack.pop()!;
switch (type.kind) {
case "record": {
if (seen.has(`record:${type.shapeId}`)) break;
seen.add(`record:${type.shapeId}`);
shapes.add(type.shapeId);
const shape = lowerer.shapes.get(type.shapeId);
for (const field of shape?.fields ?? []) stack.push(field.type);
if (shape?.indexValue) stack.push(shape.indexValue);
break;
}
case "union":
if (seen.has(`union:${type.unionId}`)) break;
seen.add(`union:${type.unionId}`);
for (const arm of lowerer.unions.get(type.unionId)?.arms ?? []) stack.push(arm);
break;
case "array": {
const key = typeKey(type);
if (seen.has(key)) break;
seen.add(key);
arrays.add(key);
stack.push(type.elem);
break;
}
case "set":
stack.push(type.elem);
break;
case "map":
stack.push(type.key, type.value);
break;
case "promise":
stack.push(type.inner);
break;
case "generator":
stack.push(type.yieldT, type.retT, type.nextT);
break;
case "func":
for (const param of type.params) stack.push(param);
stack.push(type.ret);
break;
default:
break;
}
}
return { shapes, arrays };
}
/** The guarded form of a write to a freezable record: the receiver and the
* operands are evaluated first, in source order, then the frozen bit decides
* between the write and `reject`. */
function guardedWrite(
receiver: IrExpr,
decls: IrStmt[],
reject: IrStmt[],
store: IrStmt,
loc: SrcLoc,
): IrStmt {
return {
kind: "block",
body: [
...decls,
{
kind: "if",
cond: { kind: "libCall", fn: "record.isFrozen", args: [receiver], type: BOOL, loc },
then: reject,
else_: [store],
loc,
},
],
loc,
};
}
/** Checks the frozen bit before every write to a record of a freezable
* shape, and lists the freezable array types, whose writes the backend
* guards (IrModule.freezableArrays). A write to a frozen record throws
* Node's TypeError in strict mode code; in sloppy-mode code it leaves the
* record unchanged. Deleting a key the record does not have succeeds either
* way. */
export function guardFrozenWrites(lowerer: Lowerer, functions: IrFunction[]): void {
const { shapes: freezable, arrays } = freezableTypes(lowerer, functions);
for (const id of freezable) {
const shape = lowerer.shapes.get(id);
if (shape) shape.freezable = true;
}
for (const key of arrays) lowerer.freezableArrays.add(key);
if (freezable.size === 0) return;
const lifted = lowerer.liftedFns.length;
for (const fn of functions) {
const local = (type: IrType, loc: SrcLoc): Extract<IrExpr, { kind: "varRef" }> => {
const id = `%frozenWrite.${fn.locals.length}`;
fn.locals.push({ id, name: id, type, mutable: false });
return { kind: "varRef", localId: id, type, loc };
};
const guarded = new Set<IrStmt>();
fn.body = transformStmtList(fn.body, {
stmt: (statement) => {
if (!isRecordWrite(statement) || guarded.has(statement)) return statement;
if (!freezable.has(statement.shapeId)) return statement;
const sloppy = lowerer.sloppyRecordWrites.has(writeKey(statement.loc));
const deletes =
statement.kind === "recordSet" && lowerer.recordFieldDeletes.has(writeKey(statement.loc));
const loc = statement.loc;
const shapeId = statement.shapeId;
const receiver = local(statement.obj.type, loc);
const decls: IrStmt[] = [
{ kind: "varDecl", localId: receiver.localId, init: statement.obj, loc },
];
const owner = strLit(
lowerer.shapes.get(shapeId)?.tuple ? "[object Array]" : "#<Object>",
loc,
);
const thrown = (key: IrExpr, operation: number): IrStmt => ({
kind: "exprStmt",
expr: {
kind: "libCall",
fn: "record.throwFrozen",
args: [key, owner, numLit(operation, loc)],
type: VOID,
loc,
},
loc,
});
if (statement.kind === "recordSet") {
let value = statement.value;
if (!holdsAbsentState(value)) {
const slot = local(value.type, loc);
decls.push({ kind: "varDecl", localId: slot.localId, init: value, loc });
value = slot;
}
const store: IrStmt = {
kind: "recordSet",
obj: receiver,
shapeId,
field: statement.field,
value,
loc,
};
guarded.add(store);
const field = strLit(statement.field, loc);
// `delete r.f` of an optional field writes its absent state: Node
// only refuses it when the field is present.
const reject: IrStmt[] = sloppy
? []
: deletes
? [
{
kind: "if",
cond: lowerer.recordFieldPresent(receiver, shapeId, statement.field, loc),
then: [thrown(field, 2)],
else_: null,
loc,
},
]
: [thrown(field, 0)];
return guardedWrite(receiver, decls, reject, store, loc);
}
const key = local(statement.key.type, loc);
decls.push({ kind: "varDecl", localId: key.localId, init: statement.key, loc });
const named = key.type.kind === "string";
const present: IrExpr = named
? {
kind: "call",
callee: recordHasOwnHelper(lowerer, shapeId, loc),
args: [receiver, key],
type: BOOL,
loc,
}
: { kind: "boolLit", value: true, type: BOOL, loc };
const text: IrExpr = named ? key : strLit("[key]", loc);
if (statement.kind === "recordKeySet") {
const value = local(statement.value.type, loc);
decls.push({ kind: "varDecl", localId: value.localId, init: statement.value, loc });
const store: IrStmt = {
kind: "recordKeySet",
obj: receiver,
shapeId,
key,
value,
...(statement.overflowOnly ? { overflowOnly: true as const } : {}),
loc,
};
guarded.add(store);
const reject: IrStmt[] = sloppy
? []
: [
{
kind: "if",
cond: present,
then: [thrown(text, 0)],
else_: [thrown(text, 1)],
loc,
},
];
return guardedWrite(receiver, decls, reject, store, loc);
}
const store: IrStmt = { kind: "recordKeyDelete", obj: receiver, shapeId, key, loc };
guarded.add(store);
const reject: IrStmt[] = sloppy
? []
: [{ kind: "if", cond: present, then: [thrown(text, 2)], else_: null, loc }];
return guardedWrite(receiver, decls, reject, store, loc);
},
expr: (expression) => expression,
});
}
// The key-presence helpers the guards call.
for (const helper of lowerer.liftedFns.slice(lifted)) functions.push(helper);
}
/** Whether a statically typed Object.freeze can freeze a value of `type`:
* a record whose shape has no accessor properties. */
export function freezableRecordType(lowerer: Lowerer, type: IrType): type is RecordType {
if (type.kind !== "record") return false;
const shape = lowerer.shapes.get(type.shapeId);
return shape !== undefined && !shapeHasAccessorSlots(shape);
}
@@ -2,11 +2,19 @@ import * as ts from "../ts7/adapter.js";
import { isClassOwnEnumerableFieldName } from "../../ir/ir.js";
import type { ClassInfo } from "./lower-classes.js";
/** Reflection that can remove a property or change its attributes. */
const PROPERTY_REFLECTION = new Set(["deleteProperty", "freeze", "seal", "preventExtensions"]);
/* Object.freeze, Object.seal and Object.preventExtensions do not appear
* here: they never create or remove a property, and a class instance keeps
* its integrity state outside the field layout. A statically typed call
* sets the instance's CLASS_ATTRIBUTES word (class-attributes.ts), which
* TypeScript classes use with the same fixed layouts; through an untyped
* value the runtime refuses the operation on any class instance
* (scr_dyn_class_integrity_fence). Lifting that fence must revisit this. */
/** `Object.freeze({ ... })` and the like apply to a new literal, which is
* not a class instance. */
/** Reflection that can remove any property. */
const PROPERTY_REFLECTION = new Set(["deleteProperty"]);
/** `Reflect.deleteProperty({ ... }, key)` applies to a new literal, which
* is not a class instance. */
function freshLiteralArgument(callee: ts.PropertyAccessExpression): boolean {
const call = callee.parent;
if (!ts.isCallExpression(call) || call.expression !== callee) return false;
@@ -141,12 +149,35 @@ export function constructionPrefix(
return { created, sealed };
}
/** Whether a class declares a getter or setter, which can intercept a
* base constructor's field assignment. */
function declaresAccessor(info: ClassInfo): boolean {
return [...info.methods.keys()].some(
(name) => name.startsWith("get:") || name.startsWith("set:"),
);
/** The names of the getters and setters among a class's own methods. */
export function accessorNames(methods: Iterable<string>): Set<string> {
const names = new Set<string>();
for (const name of methods)
if (name.startsWith("get:") || name.startsWith("set:")) names.add(name.slice(4));
return names;
}
/** Whether one of `accessors` can intercept an assignment the base
* constructions make: a base constructor's `this.name = value` runs a
* subclass accessor of that name instead of creating the field. Only field
* names can reach the created prefix; any other assignment already ends
* it. */
export function accessorInterceptsBase(accessors: ReadonlySet<string>, base: ClassInfo): boolean {
for (const name of accessors) if (base.fields.has(name)) return true;
return false;
}
/** Whether an accessor that instances of `info` inherit or declare shares
* a name with one of their fields. Such a field and accessor disagree on
* what the instance's property is; any other accessor lives on the
* prototype beside the fields and never touches the layout. */
function accessorShadowsField(info: ClassInfo): boolean {
for (let current: ClassInfo | null = info; current; current = current.base) {
if (current.def.runtime) break;
for (const name of accessorNames(current.methods.keys()))
if (info.fields.has(name)) return true;
}
return false;
}
/** `constructionPrefix` for a collected class and its bases. */
@@ -156,7 +187,8 @@ export function construction(info: ClassInfo): Construction | null {
if (info.base) {
base = construction(info.base);
if (!base) return null;
if (declaresAccessor(info)) base = { created: base.created, sealed: false };
if (accessorInterceptsBase(accessorNames(info.methods.keys()), info.base))
base = { created: base.created, sealed: false };
}
return constructionPrefix(info.decl, base, (name) => info.fields.has(name));
}
@@ -169,7 +201,7 @@ export function construction(info: ClassInfo): Construction | null {
export function fixedFieldFamily(root: ClassInfo, deleted: Set<string> | null): boolean {
if (deleted === null) return false;
const fixed = (info: ClassInfo): boolean => {
if (declaresAccessor(info)) return false;
if (accessorShadowsField(info)) return false;
const built = construction(info);
if (!built) return false;
for (const field of info.fieldOrder) {
@@ -247,6 +247,7 @@ import {
import { fenceSymbolFieldCopy } from "./symbol-fields.js";
import { lowerClassDataDescriptor, lowerClassDescriptorRead } from "./class-descriptors.js";
import { lowerClassFieldDefine, lowerClassIntegrity } from "./class-attributes.js";
import { freezableRecordType, lowerNativeFreeze } from "./frozen-records.js";
import { classStaticDataFor } from "./class-static-data.js";
import { lowerUrlFactory } from "./lower-url.js";
import { nullishValueUnitOf } from "./binding-analysis.js";
@@ -3763,6 +3764,14 @@ function lowerDynReceiverMethodCall(
// 'unknown' slot does.
if (checkerUntyped && DYN_HANDLE_KINDS.has(recv.type.kind))
recv = lowerer.coerceToExpected(recv, DYN);
// A typed array whose checker type collapsed to `any` (a JS binding
// referenced in its own initializer's inference — hono's etag digest
// `const nextChunk = new Uint8Array(new ArrayBuffer(f(chunk)))` beside
// `chunk = nextChunk`) still lowered to its native bytes. The typed
// lowerings key on the checker type and decline it, so it boxes by
// reference (the box shares the native storage, so writes through the
// call stay visible) and the call dispatches on the runtime kind.
if (checkerUntyped && recv.type.kind === "bytes") recv = lowerer.coerceToExpected(recv, DYN);
// The last-resort retry (boxTypedTemporary — every other lowering
// already declined): a checker-`any` receiver whose value is a FRESH
// typed result of an earlier step in the chain (`out.split("\n")`
@@ -7279,7 +7288,7 @@ export function recordKeysArrayCall(
* (a key is own exactly when Object.keys would list it — the two share
* the test), everything else true. Unmatched keys probe the overflow
* map when present, so explicit undefined values remain own keys. */
function recordHasOwnHelper(lowerer: Lowerer, shapeId: string, loc: SrcLoc): string {
export function recordHasOwnHelper(lowerer: Lowerer, shapeId: string, loc: SrcLoc): string {
const key = `obj.hasOwn:${shapeId}`;
const existing = lowerer.arrHofHelpers.get(key);
if (existing) return existing;
@@ -8494,6 +8503,22 @@ function lowerObjectStaticCall(
locOf(call),
);
if (fnIntegrity) return fnIntegrity;
// A Map or Set has no own properties: freeze, seal and preventExtensions
// all only make it not extensible.
if (value.type.kind === "map" || value.type.kind === "set") {
if (member === "preventExtensions" || member === "seal")
return lowerNativeFreeze(lowerer, "collection.preventExtensions", value, locOf(call));
const extensible: IrExpr = {
kind: "libCall",
fn: "collection.isExtensible",
args: [value],
type: BOOL,
loc: locOf(call),
};
return member === "isExtensible"
? extensible
: { kind: "unary", op: "!", operand: extensible, type: BOOL, loc: locOf(call) };
}
if (member === "preventExtensions" || member === "seal") {
const integrity = lowerClassIntegrity(lowerer, call, member, value);
if (integrity) return integrity;
@@ -8528,10 +8553,44 @@ function lowerObjectStaticCall(
// the argument — no alias exists, so the frozen bit is unobservable
// (writes through the Readonly<T> result are compile errors, and no
// other reference can write). Primitives pass through per ES2015.
// Aliased objects keep a fence: a later write through the original
// reference would need the runtime frozen bit (strict mode throws).
// Aliased records get the frozen bit of their rc word (frozen-records.ts);
// other aliased objects keep a fence: a later write through the original
// reference would need a runtime frozen bit (strict mode throws).
if (member === "isFrozen" && call.arguments.nodes.length === 1) {
const value = lowerer.lowerExprExpecting(call.arguments.nodes[0]!, DYN);
const argNode = call.arguments.nodes[0]!;
const probedKind = tryLowerExpression(lowerer, argNode)?.type.kind;
if (
probedKind === "record" ||
probedKind === "array" ||
probedKind === "map" ||
probedKind === "set"
) {
const native = lowerer.lowerExpr(argNode);
const loc = locOf(call);
if (native.type.kind === "record" || native.type.kind === "array")
return {
kind: "libCall",
fn: native.type.kind === "record" ? "record.isFrozen" : "array.isFrozen",
args: [native],
type: BOOL,
loc,
};
if (native.type.kind === "map" || native.type.kind === "set")
return {
kind: "unary",
op: "!",
operand: {
kind: "libCall",
fn: "collection.isExtensible",
args: [native],
type: BOOL,
loc,
},
type: BOOL,
loc,
};
}
const value = lowerer.lowerExprExpecting(argNode, DYN);
return { kind: "libCall", fn: "dyn.isFrozen", args: [value], type: BOOL, loc: locOf(call) };
}
if (member === "freeze") {
@@ -8581,6 +8640,12 @@ function lowerObjectStaticCall(
if (fnIntegrity) return fnIntegrity;
const integrity = lowerClassIntegrity(lowerer, call, "freeze", value);
if (integrity) return integrity;
if (freezableRecordType(lowerer, value.type))
return lowerNativeFreeze(lowerer, "record.freeze", value, locOf(call));
if (value.type.kind === "array")
return lowerNativeFreeze(lowerer, "array.freeze", value, locOf(call));
if (value.type.kind === "map" || value.type.kind === "set")
return lowerNativeFreeze(lowerer, "collection.preventExtensions", value, locOf(call));
lowerer.noLowering(
"Object.freeze of a possibly-aliased value",
call,
@@ -177,7 +177,12 @@ import { hasLexicalPrivateReference, hasStaticThis, rejectStaticThis } from "./s
import { lowerUrlNew } from "./lower-url.js";
import { isNativeProxyInitializer, lowerNativeProxy } from "./expressions/native-proxy.js";
import { classStaticDataFor } from "./class-static-data.js";
import { construction, constructionPrefix } from "./js-class-fields.js";
import {
accessorInterceptsBase,
accessorNames,
construction,
constructionPrefix,
} from "./js-class-fields.js";
import { lowerObjectFactoryNew } from "./object-factory-new.js";
import { tryLowerExpression } from "./expressions/try-lower-expression.js";
import { lowerInstanceConstructorNew } from "./class-instance-constructor.js";
@@ -4114,7 +4119,7 @@ export function collectClassShapeInner(
// A subclass field created before anything can observe the instance
// keeps its type: super() cannot run code that reads it earlier.
const baseConstruction =
base && ![...methods.keys()].some((m) => m.startsWith("get:") || m.startsWith("set:"))
base && !accessorInterceptsBase(accessorNames(methods.keys()), base)
? construction(base)
: null;
const sealedPrefix = baseConstruction
@@ -1945,6 +1945,19 @@ export function lowerStaticReadableStreamNew(
};
}
/** `new TransformStream()`: the identity transform, a native handle whose
* readable side is a native ReadableStream and whose writable side is the
* runtime's WritableStream. A transformer or queuing strategies have no
* native lowering yet and keep the SC2020 refusal. */
export function lowerStaticTransformStreamNew(
lowerer: Lowerer,
expr: ts.NewExpression,
): IrExpr | null {
if (stdlibGlobalNameOf(lowerer, expr.expression) !== "TransformStream") return null;
if ((expr.arguments?.nodes.length ?? 0) > 0) return null;
return { kind: "libCall", fn: "fetch.transformNew", args: [], type: DYN, loc: locOf(expr) };
}
/** Native Request and Headers values share the fetch runtime's handles. */
export function lowerFetchWebNew(lowerer: Lowerer, expr: ts.NewExpression): IrExpr | null {
const name = stdlibGlobalNameOf(lowerer, expr.expression);
@@ -105,6 +105,7 @@ import {
requireSpecOf,
} from "../program.js";
import { isStrictCode } from "./strict-mode.js";
import { markSloppyRecordWrites } from "./frozen-records.js";
import {
COMPOUND_ASSIGN_OPS,
type CompoundOp,
@@ -7160,7 +7161,11 @@ function lowerDeleteStatement(lowerer: Lowerer, expr: ts.DeleteExpression): IrSt
if (obj.type.kind === "record") {
const shape = lowerer.shapes.get(obj.type.shapeId);
if (shape?.indexValue && shape.fields.length === 0 && !shape.tuple) {
return { kind: "recordKeyDelete", obj, shapeId: obj.type.shapeId, key: lowerKey(), loc };
return markSloppyRecordWrites(
lowerer,
{ kind: "recordKeyDelete", obj, shapeId: obj.type.shapeId, key: lowerKey(), loc },
expr,
);
}
// `delete r.f` of a declared OPTIONAL field (undefined-armed slot)
// writes the field's ABSENT state: a monomorphic shape cannot remove
@@ -7181,14 +7186,19 @@ function lowerDeleteStatement(lowerer: Lowerer, expr: ts.DeleteExpression): IrSt
if (field) {
const absent = lowerer.absentFieldValue(field.type, loc);
if (absent) {
return {
kind: "recordSet",
obj,
shapeId: obj.type.shapeId,
field: field.name,
value: absent,
loc,
};
return markSloppyRecordWrites(
lowerer,
{
kind: "recordSet",
obj,
shapeId: obj.type.shapeId,
field: field.name,
value: absent,
loc,
},
expr,
true,
);
}
}
if (shape?.indexValue) {
@@ -150,6 +150,7 @@ import { defaultAfterUndefined, lowerStaticallyUndefinedArgument } from "./optio
import { type ProtocolDispatch } from "./class-protocol-dispatch.js";
import { ClassDynamicDispatch, trackClassFieldCreation } from "./class-dynamic-dispatch.js";
import { guardClassAttributeWrites } from "./class-attributes.js";
import { guardFrozenWrites, markSloppyRecordWrites } from "./frozen-records.js";
import { applyHandleProperties } from "./handle-properties.js";
import { ClassProtocols } from "./class-protocols.js";
import { earlyReadVarSymbols } from "./var-hoisting.js";
@@ -558,6 +559,7 @@ import {
lowerStaticReadableStreamCancelCall,
lowerStaticReadableStreamControllerCall,
lowerStaticReadableStreamNew,
lowerStaticTransformStreamNew,
lowerStaticReadableStreamReaderCall,
lowerStaticResponseCall,
lowerStdlibMethodCall,
@@ -3255,6 +3257,12 @@ export class Lowerer {
// Stack of function contexts (bottom = the function being declared at
// top level, top = the innermost nested function currently lowering).
fnStack: FnCtx[] = [];
/** Source ranges of record writes in sloppy-mode code, and of the
* absent-state writes of `delete r.f` (markSloppyRecordWrites). */
readonly sloppyRecordWrites = new Set<string>();
readonly recordFieldDeletes = new Set<string>();
/** typeKeys of the array types the program may freeze (IrModule.freezableArrays). */
readonly freezableArrays = new Set<string>();
readonly liftedFns: IrFunction[] = [];
/** The first lifted function of each name (liftedFnNamed), for the
* liftedFns indexed so far. liftedFns only grows. */
@@ -7712,6 +7720,7 @@ export class Lowerer {
finalizePrivateBrands(this, functions);
trackClassFieldCreation(this, functions);
guardClassAttributeWrites(this, functions);
guardFrozenWrites(this, functions);
applyHandleProperties(this, functions);
// Globals typed by a class that never REGISTERED (a JS class whose
@@ -7821,6 +7830,9 @@ export class Lowerer {
...(this.threadSlotKeys.size > 0
? { threadSlotKeys: [...this.threadSlotKeys].sort() }
: {}),
...(this.freezableArrays.size > 0
? { freezableArrays: [...this.freezableArrays].sort() }
: {}),
...(isNodeEsmFile(this.entry, this.program) ? {} : { commonJsEntry: true }),
...(this.ffiImports.length > 0 ? { ffiImports: [...this.ffiImports] } : {}),
};
@@ -13014,7 +13026,7 @@ export class Lowerer {
}
lowerElementWrite(expr: ts.BinaryExpression): IrStmt {
return lowerElementWrite(this, expr);
return markSloppyRecordWrites(this, lowerElementWrite(this, expr), expr);
}
lowerLiveWebValue(node: ts.Expression): IrExpr {
@@ -13411,7 +13423,7 @@ export class Lowerer {
}
fieldSetStmt(target: FieldTarget, value: IrExpr, loc: SrcLoc, blame: ts.Node): IrStmt {
return fieldSetStmt(this, target, value, loc, blame);
return markSloppyRecordWrites(this, fieldSetStmt(this, target, value, loc, blame), blame);
}
lowerFieldCompound(
@@ -13451,6 +13463,7 @@ export class Lowerer {
lowerBlobNew(this, expr) ??
lowerNetAddressNew(this, expr) ??
lowerStaticReadableStreamNew(this, expr) ??
lowerStaticTransformStreamNew(this, expr) ??
lowerConsoleNew(this, expr);
return (
builtin ?? this.withRuntimeOptionalClassValue(expr.expression, () => lowerNew(this, expr))
+48 -3
View File
@@ -371,15 +371,60 @@ test("repeated cycle edges reuse a file's binding index", () => {
test.for([
[
// A shared instance built by a user constructor: the reads of it in
// the partner's function can run early; the class cannot.
// A shared instance built by a user constructor that calls the
// partner's function: its read of the instance can run early.
{
"main.ts": 'import { shared } from "./a.ts"; console.log(shared.n);',
"a.ts":
'import { f } from "./b.ts"; export class C { n = 1; constructor() { console.log(f()); } } export const shared = new C();',
"b.ts": 'import { shared } from "./a.ts"; export function f(): number { return shared.n; }',
},
["shared"],
],
[
// User code runs while the instance is built, but nothing it reaches
// calls the partner's function before the declaration completed.
{
"main.ts": 'import { shared } from "./a.ts"; console.log(shared.n);',
"a.ts":
'import { f } from "./b.ts"; export class C { n = 1; } export const shared = new C(); console.log(f());',
"b.ts": 'import { shared } from "./a.ts"; export function f(): number { return shared.n; }',
},
["shared"],
[],
],
[
// The partner's top level runs first and calls a hoisted function of
// the importer that reads a const declared later; a second function
// reads another const but nothing calls it early.
{
"main.ts": 'import { late } from "./a.ts"; console.log(late());',
"a.ts":
'import { probe } from "./b.ts"; export function read(): string { return name; } export function late(): string { return other + probe; } export const name = "n"; export const other = "o";',
"b.ts":
'import { read } from "./a.ts"; function attempt(): string { try { return read(); } catch { return "tdz"; } } export const probe = attempt();',
},
["name"],
],
[
// The partner's class calls a method through `this` in a static
// initializer; the method reads a const of the importer declared later.
{
"main.ts": 'import { name } from "./a.ts"; console.log(name);',
"a.ts": 'import { Probe } from "./b.ts"; export const name = "n"; console.log(Probe.value);',
"b.ts":
'import { name } from "./a.ts"; export class Probe { static read(): string { return name; } static value = this.read(); }',
},
["name"],
],
[
// The same through a static block of a class stored by a const.
{
"main.ts": 'import { name } from "./a.ts"; console.log(name);',
"a.ts": 'import { Box } from "./b.ts"; export const name = "n"; console.log(Box.value);',
"b.ts":
'import { name } from "./a.ts"; export const Box = class { static value = ""; static read(): string { return name; } static { this.value = this.read(); } };',
},
["name"],
],
[
// The partner's top level reads the importer's const before it ran.
+345 -63
View File
@@ -3903,22 +3903,7 @@ function computeCycleEarlyBindings7(
const evaluation = known
? comp.slice().sort((a, b) => position.get(a)! - position.get(b)!)
: comp.slice();
interface Candidate {
symbol: ts.Symbol;
name: ts.Identifier;
kind: CycleEarlyBinding["kind"];
module: ts.SourceFile;
/** User code can run between the cycle's start and the end of the
* declaration (always true when the order is unknown). */
exposed: boolean;
rank: number;
start: number;
end: number;
reference: ts.Identifier | null;
folded: ts.Identifier | null;
localNames: Map<string, string>;
}
const candidates = new Map<ts.Symbol, Candidate>();
const candidates = new Map<ts.Symbol, CycleCandidate7>();
const names = new Set<string>();
let dirty = !known;
evaluation.forEach((m, rank) => {
@@ -3955,6 +3940,7 @@ function computeCycleEarlyBindings7(
start: ts.tokenStartOfNode(stmt),
end: stmt.end,
reference: null,
deferredReference: null,
folded: null,
localNames: new Map(),
});
@@ -4023,15 +4009,27 @@ function computeCycleEarlyBindings7(
ref.end <= c.end
)
continue;
if (inDeferredPosition7(ref)) {
if (c.exposed && c.deferredReference === null) c.deferredReference = ref;
continue;
}
let early: boolean;
if (inDeferredPosition7(ref)) early = c.exposed;
else if (sf === c.module) early = ts.tokenStartOfNode(ref) < c.start;
if (sf === c.module) early = ts.tokenStartOfNode(ref) < c.start;
else if (!memberSet.has(sf) || !rankOf.has(sf)) early = false;
else early = !known || rankOf.get(sf)! < c.rank;
if (early) c.reference = ref;
}
}
// With a static evaluation order, a deferred reference only runs early
// when the code that runs before the declaration completes can reach
// the function holding it.
const reached =
known && [...candidates.values()].some((c) => c.reference === null && c.deferredReference)
? cycleReachedDeferredReferences7(program, evaluation, candidates)
: null;
for (const c of candidates.values()) {
if (c.reference === null && c.deferredReference !== null)
c.reference = reached === null ? c.deferredReference : (reached.get(c.symbol) ?? null);
if (c.reference === null) continue;
const settled = new Set<ts.SourceFile>(order.filter((sf) => !rankOf.has(sf)));
if (known) for (const m of evaluation.slice(c.rank + 1)) settled.add(m);
@@ -4050,6 +4048,299 @@ function computeCycleEarlyBindings7(
return out;
}
/** Collects into `units` the callables an initializer expression stores
* (see moduleEarlyBindingsOf7), recording in `immediate` the parts of a
* stored node that run where it is evaluated. */
function storedCallables7(
checker: ts.TypeChecker,
e: ts.Expression,
units: ts.Node[],
immediate: Map<ts.Node, ts.Node[]>,
): void {
while (
ts.isParenthesizedExpression(e) ||
ts.isAsExpression(e) ||
ts.isSatisfiesExpression(e) ||
ts.isNonNullExpression(e)
)
e = e.expression;
if (ts.isArrowFunction(e) || ts.isFunctionExpression(e)) {
units.push(e);
} else if (ts.isClassExpression(e)) {
units.push(e);
immediate.set(e, classSyncParts7(e));
} else if (ts.isObjectLiteralExpression(e)) {
for (const p of e.properties.nodes) {
if (
ts.isMethodDeclaration(p) ||
ts.isGetAccessorDeclaration(p) ||
ts.isSetAccessorDeclaration(p)
) {
// A computed key evaluates where the literal does; a plain
// member-chain key (`[Equal.symbol]`) runs no user code.
if (!ts.isComputedPropertyName(p.name)) units.push(p);
else if (memberChainKey7(p.name.expression)) {
units.push(p);
immediate.set(p, [p.name]);
}
} else if (ts.isPropertyAssignment(p))
storedCallables7(checker, p.initializer, units, immediate);
}
} else if (ts.isArrayLiteralExpression(e)) {
for (const el of e.elements.nodes)
if (!ts.isSpreadElement(el)) storedCallables7(checker, el, units, immediate);
} else if (ts.isCallExpression(e)) {
// `const f = wrap(callback)` where wrap only returns closures over
// the callback: it runs when the stored result is called.
e.arguments.nodes.forEach((arg, index) => {
let fn: ts.Expression = arg;
while (ts.isParenthesizedExpression(fn)) fn = fn.expression;
if (
(ts.isArrowFunction(fn) || ts.isFunctionExpression(fn)) &&
returnsOnlyClosuresOver7(checker, e.expression, index)
)
units.push(fn);
});
}
}
/** A binding of an import cycle and what the reference scan found. */
interface CycleCandidate7 {
symbol: ts.Symbol;
name: ts.Identifier;
kind: CycleEarlyBinding["kind"];
module: ts.SourceFile;
/** User code can run between the cycle's start and the end of the
* declaration (always true when the order is unknown). */
exposed: boolean;
rank: number;
start: number;
end: number;
/** A reference read at top level before the declaration ran. */
reference: ts.Identifier | null;
/** A deferred reference (function body, parameter default, instance
* field) when user code may run before the declaration completes. */
deferredReference: ts.Identifier | null;
folded: ts.Identifier | null;
localNames: Map<string, string>;
}
/** For one import cycle with a static evaluation order: the first deferred
* reference to each candidate that code running before the candidate's
* declaration completes can reach, or null when the walk cannot be precise
* (top-level await, a default-exported function or class declaration).
*
* The cycle's top-level code runs in evaluation order. A stored callable
* (a function or class declaration, or a function, class or object-literal
* method stored by a variable initializer) becomes reachable once
* reachable code names its binding, through an import alias or a
* namespace member too; every other function-like in reachable code counts
* as reachable, since it may be handed to a callee. A namespace import of
* a cycle member used other than for a member read makes every stored
* callable reachable. moduleEarlyBindingsOf7 is the single-module form. */
function cycleReachedDeferredReferences7(
program: ts.Program,
evaluation: readonly ts.SourceFile[],
candidates: ReadonlyMap<ts.Symbol, CycleCandidate7>,
): Map<ts.Symbol, ts.Identifier> | null {
const checker = program.getTypeChecker();
const members = new Set(evaluation);
for (const sf of evaluation) {
if (hasTopLevelAwait7(sf)) return null;
for (const stmt of sf.statements.nodes) {
if (
(ts.isFunctionDeclaration(stmt) || ts.isClassDeclaration(stmt)) &&
(ts.getCombinedModifierFlags(stmt as unknown as ts.Declaration) &
ts.ModifierFlags.Default) !==
0
)
return null;
}
}
const stored = new Set<ts.Node>();
const immediate = new Map<ts.Node, ts.Node[]>();
const storedNames: ts.Identifier[] = [];
const unitsOfName = new Map<ts.Identifier, ts.Node[]>();
const store = (name: ts.Identifier, units: ts.Node[]): void => {
if (units.length === 0) return;
for (const u of units) stored.add(u);
storedNames.push(name);
unitsOfName.set(name, units);
};
for (const sf of evaluation) {
for (const stmt of sf.statements.nodes) {
if (
(ts.getCombinedModifierFlags(stmt as unknown as ts.Declaration) &
ts.ModifierFlags.Ambient) !==
0
)
continue;
if (ts.isFunctionDeclaration(stmt) && stmt.name !== undefined && stmt.body !== undefined) {
store(stmt.name, [stmt]);
} else if (ts.isClassDeclaration(stmt) && stmt.name !== undefined) {
store(stmt.name, [stmt]);
} else if (ts.isVariableStatement(stmt)) {
for (const d of stmt.declarationList.declarations.nodes) {
if (ts.isIdentifier(d.name) && d.initializer !== undefined) {
const units: ts.Node[] = [];
storedCallables7(checker, d.initializer, units, immediate);
store(d.name, units);
}
}
}
}
}
const unitsBySymbol = new Map<ts.Symbol, ts.Node[]>();
const allUnits: ts.Node[] = [];
for (const name of storedNames) {
const symbol = checker.getSymbolAtLocation(name);
if (symbol === undefined) continue;
const units = unitsOfName.get(name)!;
unitsBySymbol.set(symbol, [...(unitsBySymbol.get(symbol) ?? []), ...units]);
allUnits.push(...units);
}
// A named class expression's inner name (`class Impl { static x =
// Impl.m(); }`) names the class inside its own body, as the binding it
// is stored in names it outside.
const innerNames: ts.Identifier[] = [];
for (const unit of allUnits) {
if (!ts.isClassExpression(unit) || unit.name === undefined) continue;
const symbol = checker.getSymbolAtLocation(unit.name);
if (symbol === undefined) continue;
innerNames.push(unit.name);
unitsBySymbol.set(symbol, [...(unitsBySymbol.get(symbol) ?? []), unit]);
}
// The identifier texts worth resolving: declared names, and the local
// names of imports (named, default and namespace) in cycle members.
const names = new Set<string>();
for (const c of candidates.values()) names.add(c.name.text);
for (const name of storedNames) names.add(name.text);
for (const name of innerNames) names.add(name.text);
for (const sf of evaluation) {
for (const stmt of sf.statements.nodes) {
if (!ts.isImportDeclaration(stmt) || stmt.importClause === undefined) continue;
const clause = stmt.importClause;
if (clause.name !== undefined) names.add(clause.name.text);
const bindings = clause.namedBindings;
if (bindings === undefined) continue;
if (ts.isNamedImports(bindings))
for (const element of bindings.elements.nodes) names.add(element.name.text);
else names.add(bindings.name.text);
}
}
const idents: ts.Identifier[] = [];
for (const sf of evaluation) {
ts.walkPreorder(sf, (node) => {
if (ts.isImportDeclaration(node)) return "skip";
if (ts.isIdentifier(node) && names.has(node.text)) idents.push(node);
return undefined;
});
}
checker.prefetchSymbolNodesExact(idents);
const symbolOf = (id: ts.Identifier): ts.Symbol | undefined => {
const parent = id.parent;
let symbol = checker.getSymbolAtLocation(id);
if (parent !== undefined && ts.isShorthandPropertyAssignment(parent) && parent.name === id) {
symbol = checker.getShorthandAssignmentValueSymbol(parent) ?? symbol;
}
if (symbol !== undefined && symbol.flags & ts.SymbolFlags.Alias) {
symbol = checker.getAliasedSymbol(symbol);
}
return symbol;
};
const isMemberNamespace = (symbol: ts.Symbol): boolean =>
(symbol.flags & ts.SymbolFlags.ValueModule) !== 0 &&
symbol.declarations.some((d) => ts.isSourceFile(d) && members.has(d));
const declared = new Set<ts.Symbol>();
const reachedUnits = new Set<ts.Node>();
const out = new Map<ts.Symbol, ts.Identifier>();
const work: ts.Node[] = [];
const reach = (unit: ts.Node): void => {
if (reachedUnits.has(unit)) return;
reachedUnits.add(unit);
work.push(unit);
};
const visit = (root: ts.Node): void => {
ts.walkPreorder(root, (node) => {
if (node !== root && stored.has(node)) {
for (const part of immediate.get(node) ?? []) visit(part);
return "skip";
}
if (ts.isImportDeclaration(node)) return "skip";
if (node.kind === ts.SyntaxKind.ThisKeyword) {
// `this` in a static initializer or block names the class, so it
// reaches the class's methods as naming the class does.
const owner = staticThisClass7(node);
if (owner !== undefined && stored.has(owner)) reach(owner);
return undefined;
}
if (!ts.isIdentifier(node) || !names.has(node.text)) return undefined;
if (inTypePosition7(node)) return undefined;
const parent = node.parent;
if (parent !== undefined && ts.isExportSpecifier(parent)) return undefined;
if (parent !== undefined && declaredName7(parent) === node) return undefined;
const symbol = symbolOf(node);
if (symbol === undefined) return undefined;
if (isMemberNamespace(symbol)) {
// `ns.member` resolves through the member's own identifier; any
// other use can reach every export.
if (
!(
parent !== undefined &&
ts.isPropertyAccessExpression(parent) &&
parent.expression === node
)
)
for (const unit of allUnits) reach(unit);
return undefined;
}
const c = candidates.get(symbol);
if (
c !== undefined &&
node !== c.name &&
!declared.has(symbol) &&
!out.has(symbol) &&
inDeferredPosition7(node) &&
!(
c.kind === "class" &&
ts.getSourceFileOfNode(node) === c.module &&
ts.tokenStartOfNode(node) >= c.start &&
node.end <= c.end
)
)
out.set(symbol, node);
for (const unit of unitsBySymbol.get(symbol) ?? []) reach(unit);
return undefined;
});
};
const drain = (): void => {
while (work.length > 0) visit(work.pop()!);
};
const declare = (name: ts.BindingName): void => {
for (const id of boundNames7(name)) {
const symbol = checker.getSymbolAtLocation(id);
if (symbol !== undefined) declared.add(symbol);
}
};
for (const sf of evaluation) {
for (const stmt of sf.statements.nodes) {
if (ts.isClassDeclaration(stmt)) {
for (const part of classSyncParts7(stmt)) visit(part);
drain();
if (stmt.name !== undefined) declare(stmt.name);
} else if (ts.isVariableStatement(stmt)) {
for (const d of stmt.declarationList.declarations.nodes) {
visit(d);
drain();
declare(d.name);
}
} else if (!ts.isFunctionDeclaration(stmt)) visit(stmt);
drain();
}
}
return out;
}
/** Top-level let/const and class bindings, in modules outside the
* exactly-initialized cycles, that code running before their declaration
* completes can read.
@@ -4114,51 +4405,8 @@ function moduleEarlyBindingsOf7(
storedBy.set(name.text, list);
};
const checker = program.getTypeChecker();
const storedIn = (e: ts.Expression, units: ts.Node[]): void => {
while (
ts.isParenthesizedExpression(e) ||
ts.isAsExpression(e) ||
ts.isSatisfiesExpression(e) ||
ts.isNonNullExpression(e)
)
e = e.expression;
if (ts.isArrowFunction(e) || ts.isFunctionExpression(e)) {
units.push(e);
} else if (ts.isClassExpression(e)) {
units.push(e);
immediate.set(e, classSyncParts7(e));
} else if (ts.isObjectLiteralExpression(e)) {
for (const p of e.properties.nodes) {
if (
ts.isMethodDeclaration(p) ||
ts.isGetAccessorDeclaration(p) ||
ts.isSetAccessorDeclaration(p)
) {
// A computed key evaluates where the literal does; a plain
// member-chain key (`[Equal.symbol]`) runs no user code.
if (!ts.isComputedPropertyName(p.name)) units.push(p);
else if (memberChainKey7(p.name.expression)) {
units.push(p);
immediate.set(p, [p.name]);
}
} else if (ts.isPropertyAssignment(p)) storedIn(p.initializer, units);
}
} else if (ts.isArrayLiteralExpression(e)) {
for (const el of e.elements.nodes) if (!ts.isSpreadElement(el)) storedIn(el, units);
} else if (ts.isCallExpression(e)) {
// `const f = wrap(callback)` where wrap only returns closures over
// the callback: it runs when the stored result is called.
e.arguments.nodes.forEach((arg, index) => {
let fn: ts.Expression = arg;
while (ts.isParenthesizedExpression(fn)) fn = fn.expression;
if (
(ts.isArrowFunction(fn) || ts.isFunctionExpression(fn)) &&
returnsOnlyClosuresOver7(checker, e.expression, index)
)
units.push(fn);
});
}
};
const storedIn = (e: ts.Expression, units: ts.Node[]): void =>
storedCallables7(checker, e, units, immediate);
sf.statements.nodes.forEach((stmt, index) => {
if (ambient(stmt)) return;
if (ts.isFunctionDeclaration(stmt) && stmt.name !== undefined && stmt.body !== undefined) {
@@ -4411,6 +4659,40 @@ function returnsOnlyClosuresOver7(
* expressions, decorators, computed member keys, static field
* initializers and static blocks. Methods and instance fields wait for a
* call or a construction. */
/** The class whose static field initializer or static block evaluates `this`
* (through arrow functions, which keep the enclosing `this`), or undefined
* when another function or a class's instance side binds it. */
function staticThisClass7(node: ts.Node): ts.ClassLikeDeclaration | undefined {
let child = node;
for (let n = node.parent; n !== undefined; child = n, n = n.parent) {
if (ts.isArrowFunction(n)) continue;
if (ts.isClassStaticBlockDeclaration(n)) return n.parent as ts.ClassLikeDeclaration;
if (ts.isPropertyDeclaration(n)) {
if (n.initializer !== child) return undefined;
return (ts.getCombinedModifierFlags(n) & ts.ModifierFlags.Static) !== 0
? (n.parent as ts.ClassLikeDeclaration)
: undefined;
}
if (ts.isComputedPropertyName(n)) {
// A computed key evaluates in the scope around the class.
const member = n.parent;
const cls = member?.parent;
if (cls === undefined) return undefined;
child = cls;
n = cls;
continue;
}
if (
ts.isFunctionLike(n) ||
ts.isClassDeclaration(n) ||
ts.isClassExpression(n) ||
ts.isSourceFile(n)
)
return undefined;
}
return undefined;
}
function classSyncParts7(c: ts.ClassLikeDeclaration): ts.Node[] {
const decorators = (n: ts.Node): ts.Node[] =>
((n as { modifiers?: ts.NodeArray<ts.Node> }).modifiers?.nodes ?? []).filter(
@@ -0,0 +1,221 @@
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterAll, afterEach, expect, test } from "vitest";
import type { SourceFile } from "./ast-types.js";
import {
reusedLibDiagnostics,
setLibDiagnosticsStore,
type LibDiagnosticsStore,
} from "./lib-diagnostics.js";
import type { Ts7DiagnosticData } from "./session-schema.generated.js";
import type { Ts7CheckedFileDiagnosticData, Ts7SessionProgram } from "./session.js";
import { closeSourceParser, parseSourceFile } from "./source-parser-node.js";
const dir = mkdtempSync(join(tmpdir(), "scriptc-lib-diagnostics-"));
const lib = join(dir, "lib.es5.d.ts");
const fallback = join(dir, "scriptc-node-fallback.d.ts");
const libText = "declare var NaN: number;\ninterface Array<T> { length: number }\n";
const fallbackText = "export {};\ndeclare global {\n var process: { argv: string[] };\n}\n";
writeFileSync(lib, libText);
writeFileSync(fallback, fallbackText);
const main = join(dir, "main.ts");
const util = join(dir, "\u{1F600}.ts");
const other = join(dir, "�.ts");
afterEach(() => setLibDiagnosticsStore(null));
afterAll(() => {
closeSourceParser();
rmSync(dir, { recursive: true, force: true });
});
function memoryStore(): LibDiagnosticsStore & { entries: Map<string, string> } {
const entries = new Map<string, string>();
return {
entries,
read: (key) => entries.get(key) ?? null,
write: (key, text) => void entries.set(key, text),
};
}
function diagnostic(fileName: string | undefined, pos: number, code = 2322): Ts7DiagnosticData {
return { fileName, pos, end: pos + 1, code, category: 1, text: `at ${pos}` };
}
/** A session program whose full-check and checked-file answers are given; it
* records which requests it was asked. */
function sessionProgram(answers: {
semantic: Map<string, Ts7DiagnosticData[]>;
global?: Ts7DiagnosticData[];
checked?: boolean;
}): Ts7SessionProgram & { asked: string[] } {
const asked: string[] = [];
const all = [...answers.semantic.values()].flat();
return {
asked,
getCompilerOptions: () => ({ strict: true }),
diagnosticData: (method: string, file?: string) => {
asked.push(file === undefined ? method : `${method} ${file}`);
if (method === "getGlobalDiagnostics") return answers.global ?? [];
return file === undefined ? all : (answers.semantic.get(file) ?? []);
},
checkedFileDiagnosticData: (files: readonly string[]): Ts7CheckedFileDiagnosticData | null => {
asked.push(`checked ${files.join(",")}`);
if (answers.checked === false) return null;
return { files: files.map((file) => answers.semantic.get(file) ?? []), global: [] };
},
} as unknown as Ts7SessionProgram & { asked: string[] };
}
/** The program's files, parsed: the supplied two and the given others. */
function files(texts: Record<string, string>): (name: string) => SourceFile | undefined {
const parsed = new Map<string, SourceFile>([
[lib, parseSourceFile(lib, libText, "ts")],
[fallback, parseSourceFile(fallback, fallbackText, "ts")],
]);
for (const [name, text] of Object.entries(texts))
parsed.set(name, parseSourceFile(name, text, "ts"));
return (name) => parsed.get(name);
}
const names = [lib, fallback, main];
const supplied = (name: string): boolean => name === lib || name === fallback;
const moduleFile = files({ [main]: "export const n: number = 'x';\n" });
test("without a store the full check answers", () => {
const program = sessionProgram({ semantic: new Map() });
expect(reusedLibDiagnostics(program, names, supplied, moduleFile)).toBeNull();
expect(program.asked).toEqual([]);
});
test("a stored answer stands in for checking the supplied files, merged by file name", () => {
const store = memoryStore();
setLibDiagnosticsStore(store, "checker");
const semantic = new Map([
[lib, [diagnostic(lib, 1)]],
[fallback, []],
[main, [diagnostic(main, 5), diagnostic(main, 9)]],
]);
const first = sessionProgram({ semantic });
const full = reusedLibDiagnostics(first, names, supplied, moduleFile);
// The first build runs the full check and stores the supplied files' answers.
expect(first.asked).toEqual([
"getGlobalDiagnostics",
"getSemanticDiagnostics",
"checked ",
`getSemanticDiagnostics ${lib}`,
`getSemanticDiagnostics ${fallback}`,
]);
expect(store.entries.size).toBe(1);
const second = sessionProgram({ semantic });
const reused = reusedLibDiagnostics(second, names, supplied, moduleFile);
expect(second.asked).toEqual([`checked ${main}`]);
// The files' diagnostics in the order of their names (lib.es5.d.ts, main.ts), as the full answer.
expect(reused).toEqual({
semantic: [diagnostic(lib, 1), diagnostic(main, 5), diagnostic(main, 9)],
global: [],
});
expect(full?.semantic).toEqual(reused?.semantic);
});
test("files merge in code point order, which differs from UTF-16 order", () => {
const store = memoryStore();
setLibDiagnosticsStore(store, "checker");
const all = [lib, fallback, util, other];
const sources = files({ [util]: "export {};\n", [other]: "export {};\n" });
const semantic = new Map([
[util, [diagnostic(util, 1)]],
[other, [diagnostic(other, 2)]],
]);
reusedLibDiagnostics(sessionProgram({ semantic }), all, supplied, sources);
const reused = reusedLibDiagnostics(sessionProgram({ semantic }), all, supplied, sources);
// U+FFFD sorts before U+1F600 by code point (its surrogates sort after it by UTF-16 unit).
expect(reused?.semantic).toEqual([diagnostic(other, 2), diagnostic(util, 1)]);
});
test("a full check with global diagnostics is not stored", () => {
const store = memoryStore();
setLibDiagnosticsStore(store, "checker");
const program = sessionProgram({
semantic: new Map([[main, [diagnostic(main, 1)]]]),
global: [diagnostic(undefined, 0, 2318)],
});
expect(reusedLibDiagnostics(program, names, supplied, moduleFile)?.global).toEqual([
diagnostic(undefined, 0, 2318),
]);
expect(store.entries.size).toBe(0);
});
test("scripts reuse the answer unless they declare a global the supplied files declare", () => {
const store = memoryStore();
setLibDiagnosticsStore(store, "checker");
const semantic = new Map([[main, []]]);
// A script declaring only new names: it stores the answer, and a later one reuses it.
const fresh = files({
[main]: "const total = 1;\nfunction run() {}\nif (total) { var later = 2; }\n",
});
reusedLibDiagnostics(sessionProgram({ semantic }), names, supplied, fresh);
expect(store.entries.size).toBe(1);
const again = sessionProgram({ semantic });
expect(reusedLibDiagnostics(again, names, supplied, fresh)).not.toBeNull();
expect(again.asked).toEqual([`checked ${main}`]);
// Scripts declaring a supplied global, at the top level, in a block, in a
// `declare global` of a supplied module, or an ambient module: the full check.
for (const text of [
"interface Array<T> { extra: T }\n",
"if (true) { var NaN = 1; }\n",
"declare var process: unknown;\n",
'declare module "fs" {}\n',
]) {
const program = sessionProgram({ semantic });
const result = reusedLibDiagnostics(program, names, supplied, files({ [main]: text }));
expect(program.asked[0], text).not.toBe(`checked ${main}`);
if (result !== null) expect(program.asked[0]).toBe("getGlobalDiagnostics");
}
expect(store.entries.size).toBe(1);
});
test("a program whose script declares a supplied global does not store its answer", () => {
const store = memoryStore();
setLibDiagnosticsStore(store, "checker");
const program = sessionProgram({ semantic: new Map([[main, []]]) });
reusedLibDiagnostics(program, names, supplied, files({ [main]: "declare var NaN: number;\n" }));
expect(store.entries.size).toBe(0);
});
test("modules that can declare something global, and servers without the method, ask as before", () => {
const store = memoryStore();
setLibDiagnosticsStore(store, "checker");
const semantic = new Map([[main, []]]);
reusedLibDiagnostics(sessionProgram({ semantic }), names, supplied, moduleFile);
expect(store.entries.size).toBe(1);
for (const text of [
"export {};\ndeclare global { var extra: number }\n",
'export {};\ndeclare module "./other" { const extra: number }\n',
"export as namespace Lib;\nexport const n = 1;\n",
]) {
const program = sessionProgram({ semantic });
expect(reusedLibDiagnostics(program, names, supplied, files({ [main]: text }))).toBeNull();
expect(program.asked).toEqual([]);
}
const old = sessionProgram({ semantic, checked: false });
expect(reusedLibDiagnostics(old, names, supplied, moduleFile)).toBeNull();
// Nor does a server without the method store an answer it could not use.
const fresh = memoryStore();
setLibDiagnosticsStore(fresh, "checker");
reusedLibDiagnostics(sessionProgram({ semantic, checked: false }), names, supplied, moduleFile);
expect(fresh.entries.size).toBe(0);
});
test("a supplied file that changes is a different entry", () => {
const store = memoryStore();
setLibDiagnosticsStore(store, "checker");
const semantic = new Map([[main, []]]);
reusedLibDiagnostics(sessionProgram({ semantic }), names, supplied, moduleFile);
writeFileSync(fallback, fallbackText + "// edited\n");
const program = sessionProgram({ semantic });
reusedLibDiagnostics(program, names, supplied, moduleFile);
expect(program.asked[0]).toBe("getGlobalDiagnostics");
expect(store.entries.size).toBe(2);
writeFileSync(fallback, fallbackText);
});
@@ -0,0 +1,324 @@
/* The semantic diagnostics of the declaration files scriptc supplies (the
* TypeScript libs and scriptc's own declaration files), reused across
* builds. Checking them is most of the full check of a small program. When
* no other file of a program can change what checking those files reports
* (every one is a module without global or module augmentations, or a
* script declaring no global name the supplied files declare), the answer
* depends only on the checker, the compiler options and the files
* themselves, so a stored answer for the same three stands in for checking
* them again: the server checks only the program's other files
* (getDiagnosticsOfCheckedFiles), and the semantic diagnostics merge in the
* order the full check sorts them, by file name. An answer is stored only
* when the full check reports no global diagnostics: the global diagnostics
* of checking the other files are then the full check's. */
import { createHash } from "node:crypto";
import { statSync } from "node:fs";
import type { Node, SourceFile } from "./ast-types.js";
import type { Ts7DiagnosticData } from "./session-schema.generated.js";
import type { Ts7SessionProgram } from "./session.js";
import { NodeFlags, SyntaxKind } from "./enums.js";
import { walkPreorder } from "./ast.js";
/** Key-value storage for reused diagnostics (best effort: failures miss). */
export interface LibDiagnosticsStore {
read(key: string): string | null;
write(key: string, text: string): void;
}
const SCHEMA = "scriptc.lib-diagnostics.v1";
let configured: { store: LibDiagnosticsStore; checker: string } | null = null;
/** Has later builds of this process reuse the diagnostics of the supplied
* declaration files through `store`, for the checker identified by
* `checker` (its executable's identity), or stop reusing them (null). */
export function setLibDiagnosticsStore(store: LibDiagnosticsStore | null, checker = ""): void {
configured = store === null ? null : { store, checker };
}
/** A checker executable's identity for the stored diagnostics' key: its
* path and file identity, or the path alone when it cannot be read. */
export function checkerExecutableIdentity(path: string): string {
try {
const info = statSync(path);
return JSON.stringify([path, info.size, info.mtimeMs, info.ino]);
} catch {
return JSON.stringify([path]);
}
}
/** The program's semantic and global diagnostics as the full check reports
* them, using stored diagnostics for the supplied declaration files; null
* when they cannot be reused for this program (then the caller asks for
* them as before). `supplied` tells whether a program file is one of the
* supplied declaration files. */
export function reusedLibDiagnostics(
program: Ts7SessionProgram,
fileNames: readonly string[],
supplied: (fileName: string) => boolean,
sourceFile: (fileName: string) => SourceFile | undefined,
): { semantic: Ts7DiagnosticData[]; global: Ts7DiagnosticData[] } | null {
if (configured === null || process.env["SCRIPTC_LIB_DIAGNOSTICS"] === "0") return null;
const { store, checker } = configured;
const libs: string[] = [];
const own: string[] = [];
for (const name of fileNames) (supplied(name) ? libs : own).push(name);
if (libs.length === 0) return null;
// The global names each of the program's scripts declares; a module that
// declares nothing global contributes none.
const scripts: Set<string>[] = [];
for (const name of own) {
const file = sourceFile(name);
if (file === undefined) return null;
if (file.externalModuleIndicator !== undefined) {
if (!declaresNothingGlobal(file)) return null;
continue;
}
const names = isJsFileName(name) ? null : scriptGlobalNames(file);
if (names === null) return null;
scripts.push(names);
}
// A script's globals cannot change what the supplied files report unless
// one of them is also a global the supplied files declare.
const collides = (globals: ReadonlySet<string>): boolean =>
scripts.some((names) => [...names].some((name) => globals.has(name)));
// The options without the configuration file's path: it names where the
// project is, which checking the supplied files does not read (the files a
// project's location brings in are part of the key).
const options = { ...program.getCompilerOptions(), configFilePath: undefined };
const key = entryKey(checker, options, libs);
if (key === null) return null;
const stored = readEntry(store, key, libs);
if (stored !== null && !collides(stored.globals)) {
const checked = program.checkedFileDiagnosticData(own);
if (checked === null) return null;
const byFile = new Map<string, Ts7DiagnosticData[]>(stored.files);
own.forEach((name, index) => byFile.set(name, checked.files[index] ?? []));
return { semantic: inFileOrder(byFile), global: checked.global };
}
// The full check, as before. Its answers for the supplied files are stored
// when it reports no global diagnostics and the program's scripts declare
// none of the supplied files' globals (so nothing of the program reached
// what those files report).
const global = program.diagnosticData("getGlobalDiagnostics");
const semantic = program.diagnosticData("getSemanticDiagnostics");
// A server that cannot check only some files could never use the answer.
if (global.length === 0 && stored === null && program.checkedFileDiagnosticData([]) !== null) {
const globals = suppliedGlobalNames(libs, sourceFile);
if (globals !== null && !collides(globals)) {
const files: Record<string, Ts7DiagnosticData[]> = {};
for (const name of libs) files[name] = program.diagnosticData("getSemanticDiagnostics", name);
try {
store.write(key, JSON.stringify({ schema: SCHEMA, globals: [...globals], files }));
} catch {
// Best effort: a failed write is a later miss.
}
}
}
return { semantic, global };
}
/** Whether a file cannot add to or change the global declarations the
* supplied files declare: a module with no `declare global`, no module
* augmentation and no UMD `export as namespace` declaration. */
function declaresNothingGlobal(file: SourceFile): boolean {
if (file.externalModuleIndicator === undefined) return false;
if (file.moduleAugmentations.length > 0) return false;
for (const statement of file.statements.nodes) {
if (statement.kind === SyntaxKind.NamespaceExportDeclaration) return false;
// `declare global` (a module declaration named by the identifier `global`).
if (
statement.kind === SyntaxKind.ModuleDeclaration &&
statement.name?.kind === SyntaxKind.Identifier &&
statement.name.text === "global"
)
return false;
}
return true;
}
function isJsFileName(name: string): boolean {
return /\.[cm]?jsx?$/.test(name);
}
/** The names a script file declares in the global scope: its top-level
* declarations, and the `var` and function declarations of its top-level
* blocks; null when it declares an ambient module (`declare module "x"`),
* which can merge with one the supplied files declare. Over-approximates:
* a name more only makes the stored answer apply less often. */
function scriptGlobalNames(file: SourceFile): Set<string> | null {
const names = new Set<string>();
let ambientModule = false;
walkPreorder(file, (node) => {
if (node === file) return undefined;
if (node.kind === SyntaxKind.ModuleDeclaration && node.name?.kind === SyntaxKind.StringLiteral)
ambientModule = true;
const topLevel = node.parent === file;
if (topLevel || node.kind === SyntaxKind.FunctionDeclaration) addDeclaredNames(node, names);
else if (
node.kind === SyntaxKind.VariableDeclarationList &&
(node.flags & NodeFlags.BlockScoped) === 0
)
for (const declaration of node.declarations?.nodes ?? [])
addBindingNames(declaration.name, names);
// Functions, classes and namespaces have scopes of their own.
return GLOBAL_SCOPE_BOUNDARIES.has(node.kind) ? "skip" : undefined;
});
return ambientModule ? null : names;
}
/** The global names the supplied files declare: the declarations of their
* scripts (as scriptGlobalNames finds them, with ambient modules as their
* quoted names) and those of the `declare global` blocks of their modules;
* null when one of them is not available. */
function suppliedGlobalNames(
libs: readonly string[],
sourceFile: (fileName: string) => SourceFile | undefined,
): Set<string> | null {
const names = new Set<string>();
for (const name of libs) {
const file = sourceFile(name);
if (file === undefined) return null;
if (file.externalModuleIndicator === undefined) {
walkPreorder(file, (node) => {
if (node === file) return undefined;
if (
node.kind === SyntaxKind.ModuleDeclaration &&
node.name?.kind === SyntaxKind.StringLiteral
)
names.add(JSON.stringify(node.name.text ?? ""));
if (node.parent === file || node.kind === SyntaxKind.FunctionDeclaration)
addDeclaredNames(node, names);
else if (
node.kind === SyntaxKind.VariableDeclarationList &&
(node.flags & NodeFlags.BlockScoped) === 0
)
for (const declaration of node.declarations?.nodes ?? [])
addBindingNames(declaration.name, names);
return GLOBAL_SCOPE_BOUNDARIES.has(node.kind) ? "skip" : undefined;
});
continue;
}
for (const statement of file.statements.nodes) {
if (
statement.kind !== SyntaxKind.ModuleDeclaration ||
statement.name?.kind !== SyntaxKind.Identifier ||
statement.name.text !== "global"
)
continue;
for (const inner of statement.body?.statements?.nodes ?? []) addDeclaredNames(inner, names);
}
}
return names;
}
const GLOBAL_SCOPE_BOUNDARIES = new Set<SyntaxKind>([
SyntaxKind.FunctionDeclaration,
SyntaxKind.FunctionExpression,
SyntaxKind.ArrowFunction,
SyntaxKind.MethodDeclaration,
SyntaxKind.Constructor,
SyntaxKind.GetAccessor,
SyntaxKind.SetAccessor,
SyntaxKind.ClassDeclaration,
SyntaxKind.ClassExpression,
SyntaxKind.ModuleDeclaration,
]);
/** The names a statement declares in the scope it is in. */
function addDeclaredNames(node: Node, names: Set<string>): void {
switch (node.kind) {
case SyntaxKind.VariableStatement:
for (const declaration of node.declarationList?.declarations?.nodes ?? [])
addBindingNames(declaration.name, names);
return;
case SyntaxKind.FunctionDeclaration:
case SyntaxKind.ClassDeclaration:
case SyntaxKind.InterfaceDeclaration:
case SyntaxKind.TypeAliasDeclaration:
case SyntaxKind.EnumDeclaration:
case SyntaxKind.ModuleDeclaration:
case SyntaxKind.ImportEqualsDeclaration:
if (node.name?.kind === SyntaxKind.Identifier) names.add(node.name.text ?? "");
return;
}
}
function addBindingNames(name: Node | undefined, names: Set<string>): void {
if (name === undefined) return;
if (name.kind === SyntaxKind.Identifier) {
names.add(name.text ?? "");
return;
}
// Object and array binding patterns.
for (const element of name.elements?.nodes ?? []) addBindingNames(element.name, names);
}
/** The stored diagnostics' key: the checker, the compiler options and the
* supplied files in program order with each one's file identity; null when
* one of them cannot be read. */
function entryKey(checker: string, options: unknown, libs: readonly string[]): string | null {
const files: unknown[] = [];
for (const name of libs) {
try {
const info = statSync(name);
files.push([name, info.size, info.mtimeMs, info.ino]);
} catch {
return null;
}
}
return createHash("sha256")
.update(JSON.stringify({ schema: SCHEMA, checker, options, files }))
.digest("hex");
}
function readEntry(
store: LibDiagnosticsStore,
key: string,
libs: readonly string[],
): { files: Map<string, Ts7DiagnosticData[]>; globals: Set<string> } | null {
let text: string | null;
try {
text = store.read(key);
} catch {
return null;
}
if (text === null) return null;
let entry: { schema?: unknown; globals?: unknown; files?: Record<string, Ts7DiagnosticData[]> };
try {
entry = JSON.parse(text) as typeof entry;
} catch {
return null;
}
if (entry.schema !== SCHEMA || entry.files === undefined || !Array.isArray(entry.globals))
return null;
const files = new Map<string, Ts7DiagnosticData[]>();
for (const name of libs) {
const list = entry.files[name];
if (!Array.isArray(list)) return null;
files.set(name, list);
}
return { files, globals: new Set(entry.globals.filter((name) => typeof name === "string")) };
}
/** Each file's diagnostics (already in the checker's order) in the order the
* checker sorts files: by file name, compared by code point. */
function inFileOrder(byFile: ReadonlyMap<string, Ts7DiagnosticData[]>): Ts7DiagnosticData[] {
const names = [...byFile.keys()].sort(compareCodePoints);
const out: Ts7DiagnosticData[] = [];
for (const name of names) for (const d of byFile.get(name)!) out.push(d);
return out;
}
function compareCodePoints(a: string, b: string): number {
let i = 0;
let j = 0;
while (i < a.length && j < b.length) {
const x = a.codePointAt(i)!;
const y = b.codePointAt(j)!;
if (x !== y) return x < y ? -1 : 1;
i += x > 0xffff ? 2 : 1;
j += y > 0xffff ? 2 : 1;
}
return i < a.length ? 1 : j < b.length ? -1 : 0;
}
@@ -46,6 +46,10 @@ import {
trackedRealpath,
} from "../input-tracker.js";
import { packageShimDeclarationText } from "../package-shims.js";
import { ambientDtsPath, fallbackDtsPath, overridesDtsPath } from "../dts-paths.js";
import { reusedLibDiagnostics } from "./lib-diagnostics.js";
import { ts7Diagnostic } from "./diagnostic.js";
import type { Ts7DiagnosticData } from "./session-schema.generated.js";
/** The compiler options our createProgram accepts: TS7's CompilerOptions
* shape (numeric enums for target/module/moduleResolution — the enums module
@@ -393,6 +397,22 @@ export class Ts7Program {
return this.project.program.getSourceFileMetadata(resolve(fileName));
}
/** The semantic and global diagnostics with the stored diagnostics of the
* declaration files scriptc supplies (lib-diagnostics.ts), or null. */
reusedLibDiagnostics(): { semantic: Ts7DiagnosticData[]; global: Ts7DiagnosticData[] } | null {
this.snapshot.ensureActive();
const program = this.project.program;
const declarations = new Set([ambientDtsPath(), overridesDtsPath(), fallbackDtsPath()]);
return reusedLibDiagnostics(
program,
this.getSourceFileNames(),
(name) =>
declarations.has(tsgoPath(name)) ||
program.getSourceFileMetadataByPath(name)?.isDefaultLibrary === true,
(name) => this.getSourceFile(name),
);
}
isSourceFileDefaultLibrary(file: SourceFile): boolean {
return this.project.program.isSourceFileDefaultLibrary(file);
}
@@ -423,9 +443,20 @@ export class Ts7Program {
* package dropped the aggregate). Order mirrors 5.9.3: config, options/
* program, global, syntactic, semantic. */
export function getPreEmitDiagnostics(program: Ts7Program): readonly Diagnostic[] {
const config = program.getConfigFileParsingDiagnostics();
const options = program.getProgramDiagnostics();
const reused = program.reusedLibDiagnostics();
if (reused !== null)
return [
...config,
...options,
...reused.global.map(ts7Diagnostic),
...program.getSyntacticDiagnostics(),
...reused.semantic.map(ts7Diagnostic),
];
return [
...program.getConfigFileParsingDiagnostics(),
...program.getProgramDiagnostics(),
...config,
...options,
...program.getGlobalDiagnostics(),
...program.getSyntacticDiagnostics(),
...program.getSemanticDiagnostics(),
@@ -55,6 +55,13 @@ interface ProgramQuery {
snapshot?: number;
project?: string;
file?: SemanticDocument;
files?: string[];
}
/** The answer of getDiagnosticsOfCheckedFiles, as the server sends it. */
export interface Ts7CheckedFileDiagnosticData {
files: Ts7DiagnosticData[][];
global: Ts7DiagnosticData[];
}
/** Platform-independent session over an already connected transport. The
@@ -344,6 +351,29 @@ export class Ts7SessionProgram {
result.push(ts7Diagnostic(data));
return result;
}
/** The diagnostics of `method` for `file`, or for the whole program, as
* the server sends them. */
diagnosticData(method: string, file?: SemanticDocument): Ts7DiagnosticData[] {
return this.session.request<Ts7DiagnosticData[] | null>(method, this.query(file)) ?? [];
}
/** The semantic diagnostics of each of `files` and the global diagnostics,
* after the server checks only those files (getDiagnosticsOfCheckedFiles),
* or null when the server does not answer that method. */
checkedFileDiagnosticData(files: readonly string[]): Ts7CheckedFileDiagnosticData | null {
try {
return this.session.request<Ts7CheckedFileDiagnosticData>("getDiagnosticsOfCheckedFiles", {
...this.query(),
files: [...files],
});
} catch (error) {
if (
error instanceof Error &&
error.message.includes('unknown API method "getDiagnosticsOfCheckedFiles"')
)
return null;
throw error;
}
}
getSyntacticDiagnostics(file?: SemanticDocument): Ts7Diagnostic[] {
return this.diagnostics("getSyntacticDiagnostics", file);
}
@@ -213,6 +213,9 @@ export const FETCH_AMBIENT_TYPES = [
"ReadableStreamReadResult",
"ReadableStreamReadValueResult",
"ReadableStreamReadDoneResult",
"TransformStream",
"WritableStream",
"WritableStreamDefaultWriter",
"Blob",
"File",
"BlobPropertyBag",
@@ -3109,7 +3112,7 @@ function mapTypeInner(type: ts.Type, ctx: TypeMapperCtx): IrType | null {
psym &&
(FETCH_AMBIENT_TYPES as readonly string[]).includes(ts.symbolName(psym)) &&
!(
ts.symbolName(psym) === "ReadableStream" &&
(ts.symbolName(psym) === "ReadableStream" || ts.symbolName(psym) === "WritableStream") &&
psym.declarations.some(
(d) => ts.isInterfaceDeclaration(d) && isDeclaredInAmbientNamespace(d, "NodeJS"),
)
+31
View File
@@ -9,6 +9,7 @@ import {
} from "./library/prepare.js";
import { analyzeWithFrontend } from "./frontend/analysis.js";
import { checkerCacheIdentity } from "./frontend/ts7/checker-backend.js";
import { setLibDiagnosticsStore } from "./frontend/ts7/lib-diagnostics.js";
import { analyzeWithVerdictCache, fileVerdictStore } from "./coverage/verdict-cache.js";
import { coverageCompilerIdentity } from "./coverage/verdict-startup.js";
import { commentOnlyReplay } from "./coverage/verdict-semantic.js";
@@ -454,6 +455,7 @@ export async function compile(
);
return withCompileWarnings(result, warnings);
} finally {
setLibDiagnosticsStore(null);
frontendProcess?.release();
}
}
@@ -1206,10 +1208,36 @@ async function prepareExecutableInputInChild(
* `requestPath` as `compileTracked` would in-process, and writes what it prepared, without the LLVM
* text, to `resultPath`.
*/
/** Has the build reuse the diagnostics of the declaration files scriptc
* supplies (lib-diagnostics.ts) through the build cache at `root`, when the
* build uses one (executables; source outputs create no cache) and the
* checker is tsc-ts. */
function configureLibDiagnostics(root: string | null): void {
let checker: string | undefined;
try {
checker = root === null ? undefined : checkerCacheIdentity();
} catch {
// A missing checker fails where the build starts it.
checker = undefined;
}
setLibDiagnosticsStore(
root === null || checker === undefined
? null
: fileVerdictStore(join(root, "lib-diagnostics-v1")),
checker ?? "",
);
}
export async function runFrontendProcess(requestPath: string, resultPath: string): Promise<void> {
const request = JSON.parse(await readFile(requestPath, "utf8")) as FrontendProcessRequest;
clearCompileSessionCaches();
setProvenanceSources(request.provenance);
// The compile that started this process prepared the cache root.
configureLibDiagnostics(
(request.opts.outputKind ?? "exe") === "exe" && provenanceSources() === null
? await prepareBuildCacheRoot(buildCacheRoot())
: null,
);
const frontendInputs = new FrontendInputTracker();
const warnings: CompileWarningSink = { warnings: [], sourceTexts: null };
let result: FrontendProcessResult = { kind: "failed" };
@@ -1257,6 +1285,8 @@ export async function runFrontendProcess(requestPath: string, resultPath: string
}
} catch {
result = { kind: "failed" };
} finally {
setLibDiagnosticsStore(null);
}
await writeFile(resultPath, JSON.stringify(result), { mode: 0o600 });
}
@@ -1494,6 +1524,7 @@ async function compileTracked(
outputKind === "exe" && provenanceSources() === null
? await prepareBuildCacheRoot(buildCacheRoot())
: null;
configureLibDiagnostics(cacheRoot);
let earlyCacheOptions: EarlyExecutableCacheOptions | null = null;
if (outputKind === "exe") {
const helperObjectRoute =
@@ -193,3 +193,34 @@ test("number and boolean writes to class fields run no user code", () => {
// A reference stored into a field releases the old one.
expect(preserves([fieldSet(arr("b"))], "i", new Set(), scope)).toBe(false);
});
test("optional element reads, their assertion errors and present-undefined stores at an in-range index keep the length", () => {
const optional = (receiver: string, index: IrExpr): IrExpr => ({
kind: "call",
callee: "%arr.idxOr.f64",
args: [arr(receiver), index],
type: F64,
loc,
});
const undefinedAt = (receiver: string, index: IrExpr): IrStmt => ({
kind: "arraySetUndefined",
arr: arr(receiver),
index,
loc,
});
const thrown: IrExpr = { kind: "libCall", fn: "error.nodeThrow", args: [num(1)], type: F64, loc };
expect(preserves([stmt(optional("a", ref("i"))), stmt(thrown)])).toBe(true);
expect(preserves([undefinedAt("a", ref("i"))])).toBe(true);
// Past the counter, a present undefined extends the array.
expect(preserves([undefinedAt("a", plus(ref("i"), num(1)))])).toBe(false);
// The high counter of a two-counter loop is in range as well.
const paired = (body: IrStmt[]): boolean =>
preservesArrayLength(
[cond, ...body],
{ receiver: "a", inRange: "i", inRangeHigh: "j" },
locals,
new Set(),
);
expect(paired([undefinedAt("a", ref("j")), set("a", ref("j"), num(0))])).toBe(true);
expect(preserves([set("a", ref("j"), num(0))])).toBe(false);
});
@@ -162,6 +162,10 @@ export interface ArrayLengthLoop {
* long as the length itself has not changed). An element write at this
* index replaces an existing element or sets a non-index property. */
inRange: string | null;
/** A second such binding: the high counter of a two-counter loop
* (`for (let i = lo, j = hi; i < j; i++, j--)`), whose values stay
* between the low counter's start and its own. */
inRangeHigh?: string | null;
}
/** Whether running `nodes` (a loop's condition, body and update) can never
@@ -191,7 +195,10 @@ export function preservesArrayLength(
// escaping.
const receiver = (e: IrExpr): boolean =>
(e.kind === "varRef" && e.localId === loop.receiver) || expr(e);
const written = (id: string): boolean => id === loop.receiver || id === loop.inRange;
const inRange = (id: string): boolean =>
(loop.inRange !== null && id === loop.inRange) ||
(loop.inRangeHigh != null && id === loop.inRangeHigh);
const written = (id: string): boolean => id === loop.receiver || inRange(id);
const expr = (e: IrExpr): boolean => {
switch (e.kind) {
case "varRef":
@@ -224,12 +231,21 @@ export function preservesArrayLength(
case "call":
// A checked union extraction (`x!`, coercions/unions.ts): the
// asserted arm's payload or a TypeError, and no user code.
return e.callee.startsWith("%union.narrow.") && e.args.every(expr);
if (e.callee.startsWith("%union.narrow.")) return e.args.every(expr);
// An element read as `T | undefined` (int32-arrays.ts isElementReader).
return (
e.callee.startsWith("%arr.idxOr.") &&
e.args.length === 2 &&
receiver(e.args[0]!) &&
expr(e.args[1]!)
);
case "mapIntrinsic":
return READING_MAP_METHODS.has(e.method) && expr(e.receiver) && e.args.every(expr);
case "setIntrinsic":
return READING_SET_METHODS.has(e.method) && expr(e.receiver) && e.args.every(expr);
case "libCall":
// The TypeError of an assertion past a union arm leaves the loop.
if (e.fn === "error.nodeThrow") return e.args.every(expr);
return (
isBorrowSafeMath(e.fn) &&
e.args.every((arg) => arg.type.kind === "f64") &&
@@ -254,14 +270,18 @@ export function preservesArrayLength(
break;
case "arraySet":
return (
(separate(s.arr) ||
(s.index.kind === "varRef" &&
loop.inRange !== null &&
s.index.localId === loop.inRange)) &&
(separate(s.arr) || (s.index.kind === "varRef" && inRange(s.index.localId))) &&
receiver(s.arr) &&
expr(s.index) &&
expr(s.value)
);
case "arraySetUndefined":
// A present undefined replacing an element below the length.
return (
(separate(s.arr) || (s.index.kind === "varRef" && inRange(s.index.localId))) &&
receiver(s.arr) &&
expr(s.index)
);
case "return":
return s.value === null || expr(s.value);
case "fieldSet":
@@ -696,6 +696,7 @@ export const MAY_THROW_LIB_FNS: ReadonlySet<IrLibFn> = new Set([
"class.defineField",
"class.throwReadOnly",
"class.throwNotExtensible",
"record.throwFrozen",
"dyn.toString",
"dyn.valueOf",
"string.fromCodePoint",
@@ -163,6 +163,7 @@ export const LIB_FN_SIGS = defineLibFnSignatures({
"fetch.abortNow": { argTypes: [DYN], result: DYN },
"fetch.abortAny": { argTypes: [DYN], result: DYN },
"fetch.streamNew": { argTypes: [DYN], result: DYN },
"fetch.transformNew": { argTypes: [], result: DYN },
// Program-dependent iterable: typed arrays/bytes/string stay intact so
// the native stream can pull lazily; checked-dynamic values are the
// fallback. The libCall validator in validate.ts checks the closed set.
@@ -281,6 +282,28 @@ export const LIB_FN_SIGS = defineLibFnSignatures({
/** Throws Node's TypeError for a class field initializer on a
* non-extensible instance: the field name. */
"class.throwNotExtensible": { argTypes: [STRING], result: VOID },
/** Object.freeze of a record whose shape is freezable (frozen-records.ts):
* sets the frozen bit in its rc word (scr_record_freeze). The receiver is
* any record type. */
"record.freeze": { argTypes: [null], result: VOID },
/** Whether a record is frozen: its rc word's frozen bit, set by
* record.freeze and dyn.freeze of a capsule. Emitted inline. The receiver
* is any record type. */
"record.isFrozen": { argTypes: [null], result: BOOL },
/** Throws Node's TypeError for a write to a frozen record: the key, the
* object's text in the message ("#<Object>") and the operation (0 assign,
* 1 add, 2 delete). */
"record.throwFrozen": { argTypes: [STRING, STRING, F64], result: VOID },
/** Object.freeze of an array (scr_arr_freeze): sets its frozen bit, which
* the write guards of IrModule.freezableArrays test, and isFrozen reads.
* The receiver is any array type. */
"array.freeze": { argTypes: [null], result: VOID },
"array.isFrozen": { argTypes: [null], result: BOOL },
/** Object.freeze, seal or preventExtensions of a Map or Set
* (scr_map_prevent_extensions), which only makes it not extensible: its
* entries stay writable. The receiver is any map or set type. */
"collection.preventExtensions": { argTypes: [null], result: VOID },
"collection.isExtensible": { argTypes: [null], result: BOOL },
"dyn.nativeSetNew": { argTypes: [DYN], result: DYN },
"dyn.nativeSetIs": { argTypes: [DYN], result: BOOL },
"dyn.nativeMapIs": { argTypes: [DYN], result: BOOL },
@@ -219,6 +219,7 @@ const MODULE_FIELDS: Record<StringFields<Omit<IrModule, "functions">>, Role> = {
"wasmUntranslated[].sha256": "other",
"wasmUntranslated[].reason": "other",
"threadSlotKeys[]": "other",
"freezableArrays[]": "other",
"nodeIsland.libnode": "other",
"nodeIsland.nodeVersion": "other",
"nodeIsland.modules.root": "other",
+391 -52
View File
@@ -1,3 +1,4 @@
import { exprFunctionReference, moduleFunctionReferences } from "./function-references.js";
import { F64, type IrExpr, type IrFunction, type IrModule, type IrStmt } from "./ir.js";
type SwitchCase = Extract<IrStmt, { kind: "switch" }>["cases"][number];
@@ -6,22 +7,29 @@ import {
everyExprChild,
everyStmtChild,
everyStmtList,
mapExprChildren,
mapStmtChildren,
transformExpr,
transformStmt,
transformStmtList,
type IrTransform,
} from "./traverse.js";
/** Local number arrays the backend may store as int32 slots.
/** Number arrays the backend may store as int32 slots.
*
* A candidate is a local `number[]` of a synchronous function that no
* other binding can hold and only compiled element operations reach:
* every write of the local stores a fresh array (an array literal, or
* `new Array(n).fill(v)`), and every read of it is the receiver of an
* element read, write, state test, `length`, `push` or `pop` (a discarded
* `pop` included), or the array of an element-reader helper call. The
* array never reaches a runtime function that reads its slots, a library
* call, a closure, a return or untyped code.
* A candidate is a `number[]` binding that only compiled element
* operations reach: a local of a synchronous function, a module global, or
* a parameter. Every write of a local or a global stores a fresh array (an
* array literal, or `new Array(n).fill(v)`); a parameter is never written.
* Every read of a candidate is the receiver of an element read, write,
* state test, `length`, `push` or `pop` (a discarded `pop` included), the
* array of an element-reader helper call, or the argument of a direct call
* whose parameter at that position is a candidate itself. A parameter is a
* candidate only when its function is reached by direct calls alone and
* every call passes a candidate there, so every binding that can hold one
* of these arrays is a candidate: the array never reaches a runtime
* function that reads its slots, a library call, a closure, a return or
* untyped code.
*
* Candidates are only proposed here. The int32 slot analysis
* (int32-slots.ts) keeps the ones whose every stored element is a proven
@@ -33,6 +41,18 @@ import {
* calls are inlined first (rewriteCandidates), so no shared helper ever
* receives an int32 array. */
/** The candidate key of the binding `localId` as `fnName` reads it:
* `function\0local` for a local or parameter, `\0global` for a module
* global (`%g.` ids), which every function reads alike. */
export function int32ArrayKey(fnName: string, localId: string): string {
return localId.startsWith("%g.") ? `\0${localId}` : `${fnName}\0${localId}`;
}
/** The int32 element slot (int32-slots.ts) of a candidate key. */
export function int32ArraySlot(key: string): string {
return key.startsWith("\0") ? `g:${key.slice(1)}` : `l:${key}`;
}
/** Array methods a candidate may be the receiver of: the runtime paths
* behind them read and write int32 slots through the double-valued slot
* accessors. */
@@ -202,10 +222,70 @@ function writesLocal(e: IrExpr, id: string): boolean {
return !expr(e);
}
/** The candidate locals of `fn` (see the module comment). */
export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet<string>): Set<string> {
/** Expression kinds an index can be built from without running any code. */
const PURE_INDEX_KINDS: ReadonlySet<string> = new Set([
"varRef",
"numLit",
"boolLit",
"bin",
"unary",
"ternary",
"logical",
"arrayGet",
"arrayState",
]);
/** Whether evaluating an index runs no code (no call, accessor or
* conversion): plain reads, literals and their operators only. */
function pureIndex(e: IrExpr): boolean {
return PURE_INDEX_KINDS.has(e.kind) && everyExprChild(e, pureIndex, () => false);
}
/** The parameters and globals a module's functions share as candidates,
* still assumed to qualify. */
export interface SharedCandidates {
/** Candidate parameters, as `function\0local` keys. */
params: ReadonlySet<string>;
/** Candidate globals, as `%g.` ids. */
globals: ReadonlySet<string>;
/** Whether the parameter at `index` of `callee` is a candidate. */
paramAt: (callee: string, index: number) => boolean;
}
const NO_SHARED: SharedCandidates = {
params: new Set(),
globals: new Set(),
paramAt: () => false,
};
/** The candidates of `fn` (see the module comment): its own locals, its
* parameters in `shared.params`, and the globals of `shared.globals` that
* it reads only in candidate positions. `written` receives the globals it
* assigns fresh arrays. */
export function int32ArrayCandidates(
fn: IrFunction,
readers: ReadonlySet<string>,
shared: SharedCandidates = NO_SHARED,
written: Set<string> = new Set(),
): Set<string> {
const result = new Set<string>();
if (fn.async || fn.generator) return result;
if (fn.async || fn.generator) {
// Globals read here leave the candidates.
for (const id of shared.globals) result.add(id);
everyStmtList(fn.body, {
expr: (e) => {
if ((e.kind === "varRef" || e.kind === "assignExpr") && result.has(e.localId))
result.delete(e.localId);
return true;
},
stmt: (s) => {
if ((s.kind === "varDecl" || s.kind === "assign") && result.has(s.localId))
result.delete(s.localId);
return true;
},
});
return result;
}
const captured = new Set(
[...(fn.captures ?? []), ...(fn.classCaptures ?? [])].map((c) => c.localId),
);
@@ -217,11 +297,12 @@ export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet<string
!local.boxed &&
!local.tdz &&
!captured.has(local.id) &&
!params.has(local.id)
(!params.has(local.id) || shared.params.has(`${fn.name}\0${local.id}`))
)
result.add(local.id);
for (const id of shared.globals) result.add(id);
if (result.size === 0) return result;
const written = new Set<string>();
const allocated = new Set<string>();
// A receiver position: the candidate is read there without escaping.
const receiver = (e: IrExpr): void => {
if (e.kind !== "varRef") expr(e);
@@ -255,13 +336,28 @@ export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet<string
return true;
}
case "call": {
if (!readers.has(e.callee) || e.args.length !== 2) break;
const [array, index] = e.args;
// Inlining evaluates the index before the array read.
if (array!.kind === "varRef" && writesLocal(index!, array!.localId))
result.delete(array!.localId);
receiver(array!);
expr(index!);
if (readers.has(e.callee) && e.args.length === 2) {
const [array, index] = e.args;
// Inlining evaluates the index before the array read: an index
// that rebinds the array keeps it out, and so does one that can run
// code when the array is a module global (a call can reassign it).
if (
array!.kind === "varRef" &&
(writesLocal(index!, array!.localId) ||
(array!.localId.startsWith("%g.") && !pureIndex(index!)))
)
result.delete(array!.localId);
receiver(array!);
expr(index!);
return true;
}
// An argument at a candidate parameter position passes the array
// to another candidate binding (moduleInt32ArrayCandidates checks
// every such call).
e.args.forEach((arg, index) => {
if (shared.paramAt(e.callee, index)) receiver(arg);
else expr(arg);
});
return true;
}
}
@@ -272,7 +368,8 @@ export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet<string
case "varDecl":
case "assign": {
const value = s.kind === "varDecl" ? s.init : s.value;
if (value && int32ArrayAllocation(value)) written.add(s.localId);
if (params.has(s.localId)) result.delete(s.localId);
else if (value && int32ArrayAllocation(value)) allocated.add(s.localId);
else if (value) result.delete(s.localId);
if (value) expr(value);
return true;
@@ -297,10 +394,228 @@ export function int32ArrayCandidates(fn: IrFunction, readers: ReadonlySet<string
return everyStmtChild(s, expr, stmt);
};
fn.body.every(stmt);
for (const id of result) if (!written.has(id)) result.delete(id);
for (const id of result) {
if (shared.globals.has(id)) {
if (allocated.has(id)) written.add(id);
} else if (!params.has(id) && !allocated.has(id)) result.delete(id);
}
return result;
}
/** The members of `ids` (none when absent) that are in `live`. */
function liveIn(ids: ReadonlySet<string> | undefined, live: ReadonlySet<string>): Set<string> {
const result = new Set<string>();
if (ids) for (const id of ids) if (live.has(id)) result.add(id);
return result;
}
/** The candidates of every function of `mod` that has any, only those
* whose keys are in `keep` when given. Parameters and globals are shared:
* starting from every one that may qualify, each pass drops the ones a
* function reads outside candidate positions, the parameters some call
* passes anything but a candidate, and the globals never assigned a fresh
* array, until a pass drops nothing. Functions in `redirected` keep their
* parameters: some of their calls reach another function. */
export function moduleInt32ArrayCandidates(
mod: IrModule,
readers: ReadonlySet<string>,
keep: ReadonlySet<string> | null = null,
redirected: ReadonlySet<string> = new Set(),
): Map<string, Set<string>> {
const kept = (key: string): boolean => keep === null || keep.has(key);
const arrayParams = new Map<string, number[]>();
for (const fn of mod.functions) {
// Class methods, constructors and runtime helpers (`%` names) keep
// their parameters.
if (fn.name.startsWith("%") || fn.async || fn.generator || redirected.has(fn.name)) continue;
const indexes: number[] = [];
fn.params.forEach((p, i) => {
if (p.type.kind === "array" && p.type.elem.kind === "f64") indexes.push(i);
});
if (indexes.length > 0) arrayParams.set(fn.name, indexes);
}
let globals = new Set<string>();
for (const g of mod.globals ?? [])
if (
g.type.kind === "array" &&
g.type.elem.kind === "f64" &&
g.initFlag === undefined &&
kept(`\0${g.id}`)
)
globals.add(g.id);
const result = new Map<string, Set<string>>();
if (arrayParams.size === 0 && globals.size === 0) {
for (const fn of mod.functions) {
const candidates = int32ArrayCandidates(fn, readers);
for (const id of candidates) if (!kept(`${fn.name}\0${id}`)) candidates.delete(id);
if (candidates.size > 0) result.set(fn.name, candidates);
}
return result;
}
// One walk: the functions reached other than by direct calls, the direct
// calls of functions with number[] parameters, the globals each function
// reads or writes, and whether a module namespace object exists (its
// member reads can reach any global by name).
const reached = new Set(moduleFunctionReferences(mod));
const calls: { caller: string; call: IrExpr & { kind: "call" } }[] = [];
const globalsIn = new Map<string, Set<string>>();
let namespaces = false;
for (const fn of mod.functions) {
const used = new Set<string>();
const note = (id: string): void => {
if (globals.has(id)) used.add(id);
};
everyStmtList(fn.body, {
expr: (e) => {
const name = exprFunctionReference(e);
if (name !== undefined) reached.add(name);
if (e.kind === "moduleNsRef") namespaces = true;
if (e.kind === "call" && arrayParams.has(e.callee))
calls.push({ caller: fn.name, call: e });
if (e.kind === "varRef" || e.kind === "assignExpr") note(e.localId);
return true;
},
stmt: (s) => {
if (s.kind === "varDecl" || s.kind === "assign") note(s.localId);
return true;
},
});
if (used.size > 0) globalsIn.set(fn.name, used);
}
if (namespaces) {
globals = new Set();
globalsIn.clear();
}
const paramIds = new Map<string, string[]>();
let params = new Set<string>();
for (const fn of mod.functions) {
const indexes = arrayParams.get(fn.name);
if (!indexes || reached.has(fn.name)) continue;
paramIds.set(
fn.name,
fn.params.map((p) => p.localId),
);
for (const i of indexes) {
const key = `${fn.name}\0${fn.params[i]!.localId}`;
if (kept(key)) params.add(key);
}
}
for (;;) {
const current = params;
const written = new Set<string>();
const liveGlobals = new Set(globals);
result.clear();
for (const fn of mod.functions) {
const shared: SharedCandidates = {
params: current,
globals: liveIn(globalsIn.get(fn.name), globals),
paramAt: (callee, index) => {
const id = paramIds.get(callee)?.[index];
return id !== undefined && current.has(`${callee}\0${id}`);
},
};
const candidates = int32ArrayCandidates(fn, readers, shared, written);
for (const id of shared.globals) if (!candidates.has(id)) liveGlobals.delete(id);
for (const id of candidates)
if (!shared.globals.has(id) && !kept(`${fn.name}\0${id}`)) candidates.delete(id);
if (candidates.size > 0) result.set(fn.name, candidates);
}
for (const id of liveGlobals) if (!written.has(id)) liveGlobals.delete(id);
const live = new Set(
[...current].filter((key) => {
const cut = key.indexOf("\0");
return result.get(key.slice(0, cut))?.has(key.slice(cut + 1)) === true;
}),
);
// A parameter needs a candidate at that position of every call; a call
// with fewer arguments passes undefined.
for (const { caller, call } of calls) {
const ids = paramIds.get(call.callee);
if (!ids) continue;
ids.forEach((id, i) => {
const key = `${call.callee}\0${id}`;
if (!live.has(key)) return;
const arg = call.args[i];
const ok =
arg?.kind === "varRef" &&
(arg.localId.startsWith("%g.")
? liveGlobals.has(arg.localId)
: result.get(caller)?.has(arg.localId) === true);
if (!ok) live.delete(key);
});
}
if (live.size === current.size && liveGlobals.size === globals.size) break;
params = live;
globals = liveGlobals;
}
return result;
}
/** The candidates one array can reach together: a direct call's argument
* and the parameter it binds hold the same array, so their element facts
* stand or fall together. `group` maps each of `keys` to its group's keys
* (itself alone for most), and `globalReaders` each candidate global's
* key to the functions that read or write it. */
export function int32ArrayGroups(
mod: IrModule,
keys: ReadonlySet<string>,
): { group: Map<string, string[]>; globalReaders: Map<string, Set<string>> } {
const parent = new Map<string, string>();
const root = (key: string): string => {
let r = key;
while (parent.has(r)) r = parent.get(r)!;
if (r !== key) parent.set(key, r);
return r;
};
const globalReaders = new Map<string, Set<string>>();
const params = new Map(mod.functions.map((fn) => [fn.name, fn.params]));
const paramKeys = new Set(
mod.functions.flatMap((fn) => fn.params.map((p) => `${fn.name}\0${p.localId}`)),
);
if ([...keys].some((key) => key.startsWith("\0") || paramKeys.has(key))) {
for (const fn of mod.functions) {
const touch = (id: string): void => {
const key = int32ArrayKey(fn.name, id);
if (!key.startsWith("\0") || !keys.has(key)) return;
const set = globalReaders.get(key) ?? new Set<string>();
set.add(fn.name);
globalReaders.set(key, set);
};
everyStmtList(fn.body, {
expr: (e) => {
if (e.kind === "varRef" || e.kind === "assignExpr") touch(e.localId);
if (e.kind === "call")
e.args.forEach((arg, i) => {
const param = params.get(e.callee)?.[i];
if (arg.kind !== "varRef" || !param) return;
const a = int32ArrayKey(fn.name, arg.localId),
p = `${e.callee}\0${param.localId}`;
if (!keys.has(a) || !keys.has(p)) return;
const ra = root(a),
rp = root(p);
if (ra !== rp) parent.set(ra, rp);
});
return true;
},
stmt: (s) => {
if (s.kind === "varDecl" || s.kind === "assign") touch(s.localId);
return true;
},
});
}
}
const members = new Map<string, string[]>();
for (const key of keys) {
const r = root(key);
const list = members.get(r);
if (list) list.push(key);
else members.set(r, [key]);
}
const group = new Map<string, string[]>();
for (const key of keys) group.set(key, members.get(root(key))!);
return { group, globalReaders };
}
/** The element values stored into each candidate: element writes, pushed
* values, literal elements and fill values. */
export function int32ArrayStores(
@@ -341,16 +656,34 @@ export function int32ArrayStores(
return stores;
}
const REPEATED_INDEX_OPS = new Set(["+", "-", "*", "&", "|", "^", "<<", ">>", ">>>"]);
/** An index the inlined read may evaluate at each of its uses: a plain read,
* a literal, or integer arithmetic over them. Repeating it keeps the forms
* the backend derives from its operands at every use, such as the exact i64
* of an int32 product, where a bound local holds only the double. */
function repeatableIndex(e: IrExpr): boolean {
if (e.kind === "varRef" || e.kind === "numLit") return true;
if (e.kind === "unary") return e.op === "-" && repeatableIndex(e.operand);
return (
e.kind === "bin" &&
REPEATED_INDEX_OPS.has(e.op) &&
repeatableIndex(e.left) &&
repeatableIndex(e.right)
);
}
/** Rewrite a function's candidates for the backend. A discarded pop uses
* the candidate in place of its hidden local. An element-reader helper call
* whose array is a candidate is inlined: the helper's returned ternary, with
* the candidate for its array and the index for its index, so the read means
* the same. An index that is not a plain read or literal is bound to a fresh
* local first: in its own declaration just before the statement when the
* call is the whole value of a declaration or assignment (the form lowering
* gives `a[i]!`, and the one the backend reads without a box), otherwise in a
* sequence around the ternary. Selection keeps out indexes that write the
* array, so evaluating the index before the array read is unobservable. */
* the same. An index that is not repeatable (repeatableIndex) is bound to a
* fresh local first: in its own declaration just before the statement when
* the call is the whole value of a declaration or assignment (the form
* lowering gives `a[i]!`, and the one the backend reads without a box),
* otherwise in a sequence around the ternary. Selection keeps out indexes
* that write the array, so evaluating the index before the array read is
* unobservable. */
function rewriteCandidates(
fn: IrFunction,
candidates: ReadonlySet<string>,
@@ -374,14 +707,13 @@ function rewriteCandidates(
candidates.has(e.args[0]!.localId)
? e
: null;
const simple = (e: IrExpr): boolean => e.kind === "varRef" || e.kind === "numLit";
// The statements of a list, with complex read indexes declared first.
const hoist = (list: IrStmt[]): IrStmt[] =>
list.flatMap((s): IrStmt[] => {
if (s.kind !== "varDecl" && s.kind !== "assign") return [s];
const value = s.kind === "varDecl" ? s.init : s.value;
const call = readerCall(value);
if (!call || simple(call.args[1]!)) return [s];
if (!call || repeatableIndex(call.args[1]!)) return [s];
const { decl, ref } = bind(call.args[1]!);
const read: IrExpr = { ...call, args: [call.args[0]!, ref] };
if (s.kind === "varDecl") return [decl, { ...s, init: read }];
@@ -444,21 +776,23 @@ function rewriteCandidates(
if (ret.kind !== "return" || !ret.value) return e;
let index = call.args[1]!;
const stmts: IrStmt[] = [];
if (!simple(index)) {
if (!repeatableIndex(index)) {
const { decl, ref } = bind(index);
stmts.push(decl);
index = ref;
}
const operand = index;
const body = transformExpr(reparseExpr(ret.value), {
expr: (x) => {
if (x.kind !== "varRef") return x;
if (x.localId === arrayParam!.localId) return reparseExpr(array);
if (x.localId === indexParam!.localId) return reparseExpr(operand);
return x;
},
stmt: (x) => x,
});
// The arguments' copies are not searched again: an index may read a
// local that has the id of one of the helper's parameters.
const substituteExpr = (x: IrExpr): IrExpr => {
if (x.kind !== "varRef") return mapExprChildren(x, substituteExpr, substituteStmt);
if (x.localId === arrayParam!.localId) return reparseExpr(array);
if (x.localId === indexParam!.localId) return reparseExpr(operand);
return x;
};
const substituteStmt = (x: IrStmt): IrStmt =>
mapStmtChildren(x, substituteExpr, substituteStmt);
const body = substituteExpr(reparseExpr(ret.value));
// The ternary is a fresh copy (reparseExpr): it takes the call's location.
const typed = body;
typed.loc = e.loc;
@@ -471,40 +805,45 @@ function rewriteCandidates(
return { ...fn, body, locals: locals.filter((local) => !popLocals.has(local.id)) };
}
/** Propose the module's int32 array candidates (`function\0local` keys in
/** Propose the module's int32 array candidates (int32ArrayKey keys in
* `int32ArrayCandidates`), only those in `keep` when given, and rewrite
* their functions. */
export function markInt32ArrayCandidates(
mod: IrModule,
keep: ReadonlySet<string> | null = null,
redirected: ReadonlySet<string> = new Set(),
): IrModule {
if (mod.lib !== undefined) return mod;
const readers = new Map<string, IrFunction>();
for (const fn of mod.functions) if (isElementReader(fn)) readers.set(fn.name, fn);
const names = new Set(readers.keys());
const keys: string[] = [];
const all = moduleInt32ArrayCandidates(mod, new Set(readers.keys()), keep, redirected);
if (all.size === 0) return mod;
const keys = new Set<string>();
const functions = mod.functions.map((fn) => {
const candidates = int32ArrayCandidates(fn, names);
for (const id of candidates)
if (keep === null || keep.has(`${fn.name}\0${id}`)) keys.push(`${fn.name}\0${id}`);
else candidates.delete(id);
return candidates.size === 0 ? fn : rewriteCandidates(fn, candidates, readers);
const candidates = all.get(fn.name);
if (!candidates) return fn;
for (const id of candidates) keys.add(int32ArrayKey(fn.name, id));
return rewriteCandidates(fn, candidates, readers);
});
return keys.length === 0 ? mod : { ...mod, functions, int32ArrayCandidates: keys };
return { ...mod, functions, int32ArrayCandidates: [...keys] };
}
/** The module the backend emits: rewritten for exactly the candidates the
* int32 slot analysis (`prove`, returning its int32Arrays) keeps, so a
* module without int32 arrays is `mod` itself. A dropped candidate's
* rewrite is undone by marking again without it, until every candidate
* marked is kept. */
* marked is kept. `redirected` names the functions some of whose calls the
* backend emits as calls of another function (a result-slot entry): their
* parameters keep doubles. */
export function selectInt32Arrays(
mod: IrModule,
prove: (marked: IrModule) => ReadonlySet<string>,
redirected: (mod: IrModule) => ReadonlySet<string> = () => new Set(),
): IrModule {
let keep: ReadonlySet<string> | null = null;
const others = redirected(mod);
for (let round = 0; round < 4; round++) {
const marked = markInt32ArrayCandidates(mod, keep);
const marked = markInt32ArrayCandidates(mod, keep, others);
if (marked === mod) return mod;
const kept = prove(marked);
const proposed = marked.int32ArrayCandidates ?? [];
+72 -30
View File
@@ -20,7 +20,13 @@ import {
type IrNodeKind,
} from "./node-index.js";
import { findInitializerBindings, withInitializerBindings } from "./initializer-bindings.js";
import { checkableInt32Allocations, int32ArrayStores } from "./int32-arrays.js";
import {
checkableInt32Allocations,
int32ArrayGroups,
int32ArrayKey,
int32ArraySlot,
int32ArrayStores,
} from "./int32-arrays.js";
import { everyExprChild, everyStmtChild, everyTypeChild } from "./traverse.js";
/** The storage slot an array-valued expression reads, as an
@@ -257,9 +263,10 @@ export class Int32Slots {
/** Whether the module has int32 array candidates (the narrowedReads
* range fact). */
narrowedReads = false;
/** Local number arrays (`function\0local`) stored as int32 slots: the
* module's int32 array candidates whose every stored element the
* fixpoint proved an int32. */
/** Number arrays stored as int32 slots (int32ArrayKey keys: locals and
* parameters as `function\0local`, globals as `\0global`): the module's
* int32 array candidates whose every stored element, through every
* binding that can hold the same array, the fixpoint proved an int32. */
readonly int32Arrays = new Set<string>();
/** Local number arrays that may hold int32 slots with checked stores: the
* candidates whose element stores the fixpoint proved exact integers but
@@ -847,17 +854,29 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}):
for (const key of contracts?.int32Locals ?? []) slots.addInt32Local(key);
for (const key of contracts?.int32Elements ?? []) slots.addInt32Elements(key);
// Int32 array candidates (ir/int32-arrays.ts): assumed to hold int32
// elements until a store the fixpoint cannot prove drops one.
// elements until a store the fixpoint cannot prove drops one, with every
// candidate the same array can reach (its group).
const arrayCandidates = new Map<string, Set<string>>();
slots.narrowedReads = (mod.int32ArrayCandidates?.length ?? 0) > 0;
for (const key of mod.int32ArrayCandidates ?? []) {
const globalArrays = new Set<string>();
const arrayKeys = new Set(mod.int32ArrayCandidates ?? []);
slots.narrowedReads = arrayKeys.size > 0;
for (const key of arrayKeys) {
const cut = key.indexOf("\0");
const fnName = key.slice(0, cut);
const set = arrayCandidates.get(fnName) ?? new Set<string>();
set.add(key.slice(cut + 1));
arrayCandidates.set(fnName, set);
slots.addInt32Elements(`l:${key}`);
if (cut === 0) globalArrays.add(key.slice(1));
else {
const fnName = key.slice(0, cut);
const set = arrayCandidates.get(fnName) ?? new Set<string>();
set.add(key.slice(cut + 1));
arrayCandidates.set(fnName, set);
}
slots.addInt32Elements(int32ArraySlot(key));
}
const arrayGroups = arrayKeys.size > 0 ? int32ArrayGroups(mod, arrayKeys) : null;
// Functions that read or write a candidate global.
const globalArrayFunctions = new Set<string>();
if (arrayGroups)
for (const names of arrayGroups.globalReaders.values())
for (const name of names) globalArrayFunctions.add(name);
/* ── candidate fields ── */
const children = new Map<string, string[]>();
@@ -1139,6 +1158,7 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}):
const candidateWrites = nodes?.scan(CANDIDATE_WRITE_KINDS);
const hasCandidateWrite = (fn: IrFunction): boolean => {
if (arrayCandidates.has(fn.name)) return true;
if (globalArrayFunctions.has(fn.name)) return true;
const box = (localId: string): boolean => slots.hasBox(slots.boxGroup(fn.name, localId));
const returns = slots.hasReturn(fn.name);
let found = false;
@@ -1209,25 +1229,49 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}):
// (checkableInt32Allocations), it keeps int32 slots with checked stores
// (checkedInt32Arrays), which widen it to doubles in place at the first
// store that is not an int32.
const arrays = arrayCandidates.get(fn.name);
if (arrays) {
let dropped = false;
const locals = arrayCandidates.get(fn.name);
let arrays = locals;
if (globalArrayFunctions.has(fn.name)) {
const both = new Set<string>();
if (locals) for (const id of locals) both.add(id);
for (const id of globalArrays) both.add(id);
arrays = both;
}
if (arrays && arrayGroups) {
const requeue = (name: string): void => {
if (queued.has(name)) return;
queued.add(name);
queue.push(name);
};
const params = new Set(fn.params.map((param) => param.localId));
for (const [id, values] of int32ArrayStores(fn, arrays)) {
const key = `l:${fn.name}\0${id}`;
if (!slots.hasInt32ElementSlot(key) || values.every(int32)) continue;
slots.dropInt32Elements(key);
const key = int32ArrayKey(fn.name, id);
if (!slots.hasInt32ElementSlot(int32ArraySlot(key)) || values.every(int32)) continue;
// The array's group drops together: every function reading a
// member is analyzed again without its facts.
const group = arrayGroups.group.get(key) ?? [key];
for (const member of group) {
slots.dropInt32Elements(int32ArraySlot(member));
if (member.startsWith("\0")) {
const names = arrayGroups.globalReaders.get(member);
if (names) for (const name of names) requeue(name);
} else requeue(member.slice(0, member.indexOf("\0")));
}
// Checked stores serve a lone local: its allocations are the only
// arrays it holds.
const checkable = (value: IrExpr): boolean => {
const range = ranges.get(value);
return int32(value) || range != null;
};
if (values.every(checkable) && checkableInt32Allocations(fn, id))
slots.checkedInt32Arrays.add(`${fn.name}\0${id}`);
else reason(`array:${fn.name}\0${id}`, "an element store is not int32");
dropped = true;
}
if (dropped && !queued.has(fn.name)) {
queued.add(fn.name);
queue.push(fn.name);
if (
group.length === 1 &&
!key.startsWith("\0") &&
!params.has(id) &&
values.every(checkable) &&
checkableInt32Allocations(fn, id)
)
slots.checkedInt32Arrays.add(key);
else for (const member of group) reason(`array:${member}`, "an element store is not int32");
}
}
// An int32 written into a slot still assumed non-negative: widen the
@@ -1298,10 +1342,8 @@ export function analyzeInt32Slots(mod: IrModule, inputs: Int32SlotInputs = {}):
}
for (const proven of provenByFunction.values())
for (const value of proven) slots.provenWrites.add(value);
for (const [fnName, ids] of arrayCandidates)
for (const id of ids)
if (slots.hasInt32ElementSlot(`l:${fnName}\0${id}`))
slots.int32Arrays.add(`${fnName}\0${id}`);
for (const key of arrayKeys)
if (slots.hasInt32ElementSlot(int32ArraySlot(key))) slots.int32Arrays.add(key);
const ranged = rangedParameters(
slots,
functions,
@@ -440,6 +440,37 @@ test("int32-closed locals keep the int32 range across loops and switches", () =>
expect(switched.get(afterSwitch)).toEqual(INT32);
});
test("loop-carried locals written from Int32[] element reads keep the int32 range", () => {
const arrType = { kind: "array", elem: F64 } as const;
const loop = (localId: string): { after: IrExpr; f: IrFunction } => {
const after = ref();
// `x = a[x]!` in a loop: a cursor that follows indices stored in the array.
const read: IrExpr = {
loc,
kind: "arrayGet",
arr: { loc, kind: "varRef", localId, type: arrType },
index: ref(),
type: F64,
};
const f = fn([
assign(num(0)),
{
loc,
kind: "while",
cond: condition("<", ref("unknown"), num(1)),
body: [assign(read)],
},
observe(after),
]);
return { after, f };
};
const elements = { elements: (arr: IrExpr) => arr.kind === "varRef" && arr.localId === "ints" };
const ints = loop("ints");
expect(analyzeIntegerRanges(ints.f, elements).get(ints.after)).toEqual(INT32);
const floats = loop("floats");
expect(analyzeIntegerRanges(floats.f, elements).get(floats.after) ?? null).toBeNull();
});
test("uint32-closed locals keep the uint32 range across loop back edges", () => {
// for (let i = 0; i < n; i++) { v = unknown >>> 0; acc = (acc + v) >>> 0; ... }
const loop = (start: IrExpr | null, writes: IrExpr[]) => {
@@ -480,6 +480,7 @@ export function analyzeIntegerRangeFacts(
return boxedRange(e.localId);
}
case "fieldGet":
case "arrayGet":
case "call":
return slotRange(e);
default:
+8
View File
@@ -1068,6 +1068,10 @@ export interface IrModule {
* each one's hierarchy carries a slot word (the slots.* intrinsics;
* backend/llvm/thread-slots.ts). */
threadSlotKeys?: string[];
/** typeKeys of the array types whose values the program may freeze
* (frontend frozen-records.ts): their writes test the array's frozen bit
* (SCR_ARR_FROZEN) and throw Node's TypeError when it is set. */
freezableArrays?: string[];
/** The entry module is CommonJS under Node's classification. Node runs a
* CommonJS main synchronously and drains process.nextTick callbacks
* before promise jobs at the first checkpoint; an ES-module main's
@@ -1709,6 +1713,10 @@ export interface IrRecordShape {
* dynCheck), and JSON serialization appends overflow entries after the
* declared fields in insertion order. */
indexValue?: IrType;
/** Values of this shape may be frozen (frozen-records.ts): their rc word
* can carry the frozen bit, which their release ignores, and every write
* to them checks it. */
freezable?: true;
/** Field names in FIRST-SEEN declaration order (the checker's property
* order of the first ts.Type interned to this shape) — metadata, NOT
* part of the interned identity: a later structurally-equal type with a
+16 -1
View File
@@ -16,7 +16,11 @@ import { createNativeTs7Api } from "../frontend/ts7/native-api.js";
import { resolveProvenanceSourcesWithParser } from "../frontend/provenance-core.js";
import { setCompilerReleaseVersion } from "../library/sidecar.js";
import { NativeCompiler } from "./compiler.js";
import { openNativeCache } from "./cache.js";
import { NATIVE_STRING_BYTES, openNativeCache } from "./cache.js";
import {
checkerExecutableIdentity,
setLibDiagnosticsStore,
} from "../frontend/ts7/lib-diagnostics.js";
import { loadNativeToolchain } from "./toolchain.js";
import { buildSanitizedRuntime } from "./sanitizer.js";
import { runCompilerTask } from "./task.js";
@@ -53,6 +57,17 @@ async function main(): Promise<number> {
if (toolchain.declarationsRoot !== undefined) setDeclarationRoot(toolchain.declarationsRoot);
setCompilerReleaseVersion(toolchain.compilerVersion);
cache = openNativeCache((path, harden) => compilerNativePrivateDirectory(path, harden));
if (cache !== null) {
const store = cache;
setLibDiagnosticsStore(
{
read: (key) =>
store.read("lib-diagnostics", key, NATIVE_STRING_BYTES)?.toString("utf8") ?? null,
write: (key, text) => store.write("lib-diagnostics", key, text),
},
checkerExecutableIdentity(toolchain.ts7Executable),
);
}
compiler = new NativeCompiler(toolchain, cache);
return compiler;
};
+1 -1
View File
@@ -1,6 +1,6 @@
{
"schemaVersion": 2,
"compilerVersion": "0.4.2",
"compilerVersion": "0.4.3",
"coverage": [
"Entries are projected mechanically from the compiler's own decision tables: the diagnostics registry, the unsupported-syntax dispatch tables, the stdlib and node-builtin lowering tables, and the supported-builtin-module list. Nothing is hand-maintained; the manifest regenerates byte-identically from the source tree at this version.",
"Absence from this manifest means 'not projected', never 'unsupported'. Surfaces lowered through dedicated code paths rather than tables are not yet projected: console, JSON, Promise/async and the timer surface, the net/http/tls/https/dgram/dns/assert/test/stream/readline module member surfaces, template literals, the regex slice, global functions (parseInt, parseFloat, isNaN, isFinite), and the process surface outside its ambient slice.",
@@ -77,6 +77,19 @@ function literalCounter(n: number): number {
for (let i = 0; i < n; i++) a[0] = a[0]! + 1;
return a[0]!;
}
function walk(n: number, steps: number): number {
const next: number[] = [];
for (let i = 0; i < n; i++) next.push(i + 1 < n ? i + 1 : 0);
let at = next[0]!;
let total = 0;
for (let s = 0; s < steps; s++) {
const t = next[at]!;
next[at] = at;
at = t;
total = (total + at) | 0;
}
return total;
}
function joined(n: number): string {
const a = [n | 0, 2];
return a.join(",");
@@ -91,6 +104,7 @@ console.log(
shortCounter(),
literalCounter(n),
joined(n),
walk(n | 0, 5),
);
`;
@@ -119,6 +133,16 @@ test("arrays storing only int32 elements get 4-byte slots", async () => {
expect(counts).not.toContain("popArray");
});
test("an array that stores a cursor read back from its own elements gets 4-byte slots", async () => {
const llvm = await emitLlvm(source);
const walk = functions(llvm, "walk");
expect(walk).not.toBe("");
// `at` is assigned `next[at]!` in the loop and stored back into `next`:
// it holds an int32 whenever the array's elements are int32s.
expect(walk).toMatch(/@scr_arr_new_small\(i32 7\)/);
expect(walk).toMatch(/getelementptr inbounds i32, ptr %\w+, i64 %\w+\n\s+%\w+ = load i32/);
});
test("arrays with a store that has no exact integer range, or that escape, keep doubles", async () => {
const llvm = await emitLlvm(source);
for (const name of ["fraction", "negativeZero", "joined"]) {
@@ -196,3 +220,42 @@ test("only the candidates the proof keeps are rewritten", async () => {
for (const fn of selected.functions)
if (fn.name !== "compose") expect(fn).toBe(mod.functions.find((f) => f.name === fn.name));
});
const sharedSource = `
const data: number[] = [];
for (let i = 0; i < 64; i++) data.push((i * 7) & 255);
function fnv(values: number[]): number {
let h = 2166136261;
for (let i = 0; i < values.length; i++) h = Math.imul(h ^ values[i]!, 16777619) >>> 0;
return h;
}
function mean(values: number[]): number {
let s = 0;
for (let i = 0; i < values.length; i++) s += values[i]!;
return s / values.length;
}
function first(values: number[]): number {
return values[0]!;
}
const ints = [1, 2, 3];
const halves = [0.5, 1.5];
const pick: (values: number[]) => number = first;
console.log(fnv(data), mean(ints), mean(halves), pick([4]));
`;
test("arrays passed to helpers and held in module globals get 4-byte slots", async () => {
const mod = await lowerIr(sharedSource);
const proposed = markInt32ArrayCandidates(mod).int32ArrayCandidates ?? [];
// A global and the parameter every call passes it to; a function used as
// a value keeps its parameter.
expect(proposed).toEqual(expect.arrayContaining(["\0%g.e.data", "fnv\0values.0"]));
expect(proposed.some((key) => key.startsWith("first\0"))).toBe(false);
const llvm = await emitLlvm(sharedSource);
// The helper reads int32 slots; `mean`, also called with fractions,
// reads doubles.
expect(functions(llvm, "fnv")).toMatch(
/getelementptr inbounds i32, ptr %\w+, i64 %\w+\n\s+%\w+ = load i32/,
);
expect(functions(llvm, "mean")).not.toMatch(/getelementptr inbounds i32, ptr/);
expect(llvm).toMatch(/@scr_arr_new(_small)?\(i32 7/);
});
@@ -101,6 +101,51 @@ delete loose.label;
expect(layout(deleted, "Shape").tracks).toBe(true);
});
test("freezing values other than class instances keeps JavaScript class layouts fixed", async () => {
const module = await lower(`${SHAPES}
/** @param {{ value: any }} data */
function settle(data) {
data.value = Object.freeze(data.value);
Object.seal(data);
return Object.preventExtensions(data.value);
}
settle({ value: { sides: 1 } });
const square = new Square();
Object.freeze(square);
`);
expect(layout(module, "Shape").tracks).toBe(false);
expect(layout(module, "Square").tracks).toBe(false);
});
test("accessors that share no field's name keep JavaScript class layouts fixed", async () => {
const module = await lower(`
class Path {
constructor(key) {
this.cache = [];
this.key = key;
}
get path() {
if (!this.cache.length) this.cache.push(this.key);
return this.cache;
}
}
class Named extends Path {
constructor(key) {
super(key);
this.label = "named";
}
/** @param {string} value */
set label2(value) {
this.label = value;
}
}
console.log(new Named("k").path.length);
`);
expect(layout(module, "Path")).toEqual({ fields: "cache:dyn key:dyn", tracks: false });
expect(layout(module, "Named").tracks).toBe(false);
expect(layout(module, "Named").fields).toBe("cache:dyn key:dyn label:string");
});
test("JavaScript subclass fields keep their types when the base constructor only assigns fields", async () => {
const module = await lower(`
class Base {
@@ -3,6 +3,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, expect, test } from "vitest";
import { compile, deserializeModule, validateModule } from "../src/index.js";
import { setLibDiagnosticsStore } from "../src/frontend/ts7/lib-diagnostics.js";
const dirs: string[] = [];
afterEach(async () => {
@@ -188,6 +189,38 @@ test("source outputs ignore invalid external compiler selection and create no ex
}
});
test("source outputs neither read nor create declaration diagnostics a previous build stored", async () => {
const { dir, entry, outDir } = await fixture();
const oldCache = process.env["SCRIPTC_CACHE_DIR"];
const cache = join(dir, "must-not-exist");
process.env["SCRIPTC_CACHE_DIR"] = cache;
// A store an earlier build of this process left configured.
const reads: string[] = [];
setLibDiagnosticsStore(
{ read: (key) => (reads.push(key), null), write: () => {} },
"earlier-checker",
);
try {
for (const [kind, name] of [
["ir", "main.ir.json"],
["llvm", "main.ll"],
] as const) {
const result = await compile(entry, {
outDir,
outPath: join(outDir, name),
outputKind: kind,
});
expect(result.ok).toBe(true);
}
expect(reads).toEqual([]);
await expect(readdir(cache)).rejects.toMatchObject({ code: "ENOENT" });
} finally {
setLibDiagnosticsStore(null);
if (oldCache === undefined) delete process.env["SCRIPTC_CACHE_DIR"];
else process.env["SCRIPTC_CACHE_DIR"] = oldCache;
}
});
test.each(["wasm64-wasi", "totally-invalid"])(
"source output rejects unsupported target %s without writing an artifact",
async (target) => {
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7260-teardown-segments.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7264-array-dense-paired-loops.ts"
],
"diags": []
}
@@ -0,0 +1,8 @@
{
"order": [
"<repo>/tests/corpus/7380-cycle-deferred-folded-key/util.ts",
"<repo>/tests/corpus/7380-cycle-deferred-folded-key/core.ts",
"<repo>/tests/corpus/7380-cycle-deferred-folded-key/main.ts"
],
"diags": []
}
@@ -0,0 +1,8 @@
{
"order": [
"<repo>/tests/corpus/7381-cycle-deferred-reached-tdz/beta.ts",
"<repo>/tests/corpus/7381-cycle-deferred-reached-tdz/alpha.ts",
"<repo>/tests/corpus/7381-cycle-deferred-reached-tdz/main.ts"
],
"diags": []
}
@@ -0,0 +1,8 @@
{
"order": [
"<repo>/tests/corpus/7382-cycle-static-this-tdz/beta.ts",
"<repo>/tests/corpus/7382-cycle-static-this-tdz/alpha.ts",
"<repo>/tests/corpus/7382-cycle-static-this-tdz/main.ts"
],
"diags": []
}
@@ -0,0 +1,8 @@
{
"order": [
"<repo>/tests/corpus/7383-cycle-class-expression-name-tdz/beta.ts",
"<repo>/tests/corpus/7383-cycle-class-expression-name-tdz/alpha.ts",
"<repo>/tests/corpus/7383-cycle-class-expression-name-tdz/main.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7390-record-key-read-presence.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7404-frozen-records.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7405-frozen-records-sloppy.cjs"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7406-frozen-containers.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7580-int32-arrays-across-calls.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7642-element-assignment-order.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7742-response-clone-tee.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7743-transform-stream-identity.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7748-npm-any-bytes-receiver/main.ts"
],
"diags": []
}

Some files were not shown because too many files have changed in this diff Show More