From 723975d01c5fc9804342a9838fdde69547cc652d Mon Sep 17 00:00:00 2001 From: Chris Tate Date: Tue, 27 Oct 2026 02:38:15 -0500 Subject: [PATCH] Prove that the arrays InlineArray fields hold are never written The constructor copies the array it is passed into the instance, and other reads of the field give a new array over the elements, while under Node.js the field holds that array itself. The contract check now proves the two agree, and reports SC7009 where it cannot: - The arrays fields hold are never written. A whole-program graph of the places arrays are held (locals, parameters, results, class and record fields, and the elements of arrays, Maps and Sets by where those are held) follows every array a field read may give, and reports a write to one, or one that reaches a dynamic value, a call through a function value or a library call other than printing and JSON.stringify without a replacer. Functions that return their argument give each caller its own argument back. The report names the field read. - The array passed to the constructor is one nothing else keeps: a new array (a literal, a copy, or the result of a function that returns only such arrays or its argument), a local array at its last use (after calls that only read it or write it in place, or in a loop that then assigns the local a new array), a parameter whose every caller passes such an array, an array a field already holds, or a const array that is never written, which is then followed like a field's. The review fixes from before stay: `new` evaluates its arguments into locals unless they are all locals and constants, so that the allocation reads the length after every argument has run; the constructor uses the array parameter only to store it; and the dynamic view of an instance reads the field as an array of its elements. --- docs/content/docs/contracts.mdx | 2 +- docs/content/docs/experimental.mdx | 6 +- packages/compiler/src/backend/llvm/classes.ts | 8 + .../src/backend/llvm/expr-stream-bridges.ts | 23 + packages/compiler/src/contracts/check.ts | 12 + .../compiler/src/ir/inline-array-sources.ts | 1716 +++++++++++++++++ packages/compiler/src/ir/inline-arrays.ts | 35 +- .../compiler/test/inline-array-fields.test.ts | 283 ++- packages/experimental/README.md | 4 +- packages/experimental/index.d.ts | 7 +- tests/corpus/contract-inline-array-fields.ts | 17 + 11 files changed, 2097 insertions(+), 16 deletions(-) create mode 100644 packages/compiler/src/ir/inline-array-sources.ts diff --git a/docs/content/docs/contracts.mdx b/docs/content/docs/contracts.mdx index 7815ccc40..1c612c0d9 100644 --- a/docs/content/docs/contracts.mdx +++ b/docs/content/docs/contracts.mdx @@ -368,7 +368,7 @@ A long-running program, such as a compiler in watch mode, can free program-lifet ### Inline array fields -An array field of a program-lifetime class whose elements are program-lifetime instances can keep its elements in the instance itself, after the other fields, with `InlineArray` from [`@scriptc/experimental`](/docs/experimental). The constructor writes the field once from one of its parameters; reads that index the field, read its `length` or iterate it load the elements from the instance, and other reads get an array of them, which is frozen in scriptc builds. scriptc reports a field that does not keep these rules (SC7009). +An array field of a program-lifetime class whose elements are program-lifetime instances can keep its elements in the instance itself, after the other fields, with `InlineArray` from [`@scriptc/experimental`](/docs/experimental). The constructor writes the field once from one of its parameters; reads that index the field, read its `length` or iterate it load the elements from the instance, and other reads get an array of them. The array passed to the constructor is one that nothing else keeps, and the arrays fields hold are never written. scriptc reports a field that does not keep these rules (SC7009). ## Diagnostics diff --git a/docs/content/docs/experimental.mdx b/docs/content/docs/experimental.mdx index 77ea52982..b023be2c9 100644 --- a/docs/content/docs/experimental.mdx +++ b/docs/content/docs/experimental.mdx @@ -388,14 +388,16 @@ scriptc requires: - `T` to be a `ProgramLifetime` class, and the field to be an instance field of a `ProgramLifetime` class; - the constructor of the class that declares the field to write it once, from one of its parameters. A subclass constructor passes one of its own parameters there through `super(...)`. Nothing else writes the field; +- the array passed to the constructor to be one that nothing else keeps: a new array (an array literal, a copy such as `slice()`, `[...items]` or `Array.from(items)`, or the result of a function that returns only such arrays or its own argument), a local array at its last use (calls that only read it or add to it may come first, and a loop may pass it and then assign a new array to the local), a parameter whose callers each pass such an array, an array that an InlineArray field already holds, or a `const` array that is never written, such as a shared empty array; +- the arrays fields hold never to be written, through a read of the field (which the `readonly` type allows only through a cast) or through any other reference. scriptc follows these arrays through locals, parameters, return values, fields, the elements of arrays, Maps and Sets, and closures, and reports one that reaches a write, a dynamic value, a call through a function value, or a library call other than printing and `JSON.stringify` without a replacer. The report names the field read the array may come from; - reads of the field not to be compared with `===` or `!==`: two reads need not give the same array. Compare a read kept in a local instead; -- instances of the class not to be constructed through a class value, and not to become dynamic values (for example through `console.log` or `JSON.stringify`). +- instances of the class not to be constructed through a class value, and not to become dynamic values (for example through `JSON.stringify` or a conversion to `unknown`). scriptc reports a field that does not keep these rules (SC7009). ### Reads -Indexing the field, reading its `length` and iterating it with `for...of` read the elements from the instance, directly or through a `const` local used only in those ways. Any other use, such as passing the field to a function or calling one of its methods, gets an array of the elements, which is frozen in scriptc builds. Under Node.js the field holds the array the constructor stored. +Indexing the field, reading its `length` and iterating it with `for...of` read the elements from the instance, directly or through a `const` local used only in those ways. Any other use, such as passing the field to a function or calling one of its methods, gets an array of the elements, which is frozen in scriptc builds. Under Node.js the field holds the array the constructor stored. The two agree because the field rules keep that array from being written. ### Inline array cost diff --git a/packages/compiler/src/backend/llvm/classes.ts b/packages/compiler/src/backend/llvm/classes.ts index d25233dc8..0cc1bae96 100644 --- a/packages/compiler/src/backend/llvm/classes.ts +++ b/packages/compiler/src/backend/llvm/classes.ts @@ -464,6 +464,12 @@ export function classFieldIndex( throw new InternalCompilerError( `llvm emitter bug: unknown field ${field} on class ${meta.def.name}`, ); + // An inline array field's slot holds its element count: its reads and + // writes go through inline-arrays.ts. + if (meta.inlineTail?.field === idx && meta.compact) + throw new InternalCompilerError( + `llvm emitter bug: field slot of the inline array field ${meta.def.name}.${field}`, + ); const compressed = meta.compressed[idx] === true; // Callers that read or write a field's storage as a pointer must not see // a compressed one; callers that decode and encode pass "compressed". @@ -940,6 +946,8 @@ function undefFieldInits( if (f.name === "%cause") return; // A nullable-pointer field's zeroed slot already is its unit arm. if (nullable?.get(meta.def.name, f.name)) return; + // An inline array field's slot holds its count, which `new` stores. + if (meta.inlineTail?.field === i && meta.compact) return; // Only union and dyn fields get a store here; a compressed field is a // class reference, which starts NULL. const { index, compressed } = classFieldIndex(meta, f.name, "compressed"); diff --git a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts index f711795d3..3f901e825 100644 --- a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts +++ b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts @@ -35,6 +35,7 @@ import { import { SCR_ARR_FIELD, llvmCommentText } from "./common.js"; import { FN_ATTRS, + arrOps, elemAccess, llFieldType, loadStructField, @@ -44,6 +45,7 @@ import { type ShapeHost, } from "./shapes.js"; import type { LlvmEmitterContext, LlStreamTypedRefAdapter } from "./expr-context.js"; +import { inlineFieldTail } from "./inline-arrays.js"; import type { NullableRefField } from "./nullable-fields.js"; import { emitUnionPeek, emitUnionTag } from "./union-repr.js"; import { inlineUnionCopyLines } from "./inline-unions.js"; @@ -1622,6 +1624,27 @@ function classFieldDynValue( t: Extract, field: LlClassMeta["def"]["fields"][number], ): { boxed: string; nullable: NullableRefField | null; fieldValue: string } { + // An inline array field reads as a frozen array over the instance's + // elements (inline-arrays.ts), which the box keeps. + const tail = inlineFieldTail(host, t.className, field.name); + if (tail !== undefined) { + const countp = B.tmp(), + count = B.tmp(), + at = B.tmp(), + view = B.tmp(); + B.line( + `${countp} = getelementptr inbounds %${classStructSym(t.className)}, ptr %p, i64 0, i32 ${meta.compactSlots[tail.field]!}`, + ); + B.line(`${count} = load i32, ptr ${countp}`); + B.line(`${at} = getelementptr inbounds i8, ptr %p, i64 ${tail.tailOffset}`); + const ops = arrOps(host.shapeHost, { kind: "object", className: tail.elemClass }); + host.declare(`declare ptr @scr_arr_view_cref(ptr, ptr, i32)`); + host.declare(`declare void @scr_arr_release(ptr)`); + B.line(`${view} = call ptr @scr_arr_view_cref(ptr ${ops}, ptr ${at}, i32 ${count})`); + const boxed = host.streamTypedRefBoxValue(B, field.type, view); + B.line(`call void @scr_arr_release(ptr ${view})`); + return { boxed, nullable: null, fieldValue: view }; + } const { index, compressed } = classFieldIndex(meta, field.name, "compressed"); const fieldPtr = B.tmp(); let fieldValue = B.tmp(); diff --git a/packages/compiler/src/contracts/check.ts b/packages/compiler/src/contracts/check.ts index 812306eea..76fd76a11 100644 --- a/packages/compiler/src/contracts/check.ts +++ b/packages/compiler/src/contracts/check.ts @@ -68,6 +68,7 @@ import { type ContractNote, } from "../frontend/contract-types.js"; import { planInlineArrays } from "../ir/inline-arrays.js"; +import { checkInlineArraySources } from "../ir/inline-array-sources.js"; /** The largest array index: 2^32 - 2 (ECMA-262 array index). */ const MAX_ARRAY_INDEX = 2 ** 32 - 2; @@ -379,6 +380,17 @@ class ContractChecker { "write the field once in the constructor from one of its parameters (subclasses pass theirs through super(...)), never again, and keep the class's instances out of class values and dynamic values", ); } + // Where the arrays come from and who writes them (ir/inline-array-sources.ts). + for (const issue of checkInlineArraySources(this.mod, plan)) { + const note = notes.get(`${issue.declaringClass}\0${issue.field}`)!; + const from = issue.origin === undefined ? "" : ` (from ${this.where(issue.origin)})`; + this.report( + "SC7009", + issue.loc, + `'${note.name}' (InlineArray): ${issue.reason}${from}`, + "an InlineArray field holds its array for good: build a new array for each constructor call, and never write to an array a field holds", + ); + } const kept = new Set(); for (const cls of plan.classes.values()) kept.add(`${cls.declaringClass}\0${cls.field}`); this.record("inlineArrays", [...kept]); diff --git a/packages/compiler/src/ir/inline-array-sources.ts b/packages/compiler/src/ir/inline-array-sources.ts new file mode 100644 index 000000000..e3cf26116 --- /dev/null +++ b/packages/compiler/src/ir/inline-array-sources.ts @@ -0,0 +1,1716 @@ +/* Where the arrays of InlineArray fields come from, and who writes them + * (ir/inline-arrays.ts plans the layout; contracts/check.ts reports). + * + * The constructor copies the elements of the array it is passed into the + * instance, and other reads of the field give a new array over them. Under + * Node.js the field holds that array itself. The two agree as long as no + * write ever reaches an array a field holds, either through a read of the + * field or through another reference to the array the constructor stored: + * + * - every array that may be (a read of) an inline field is never written, + * and never reaches code whose writes this analysis cannot see (dynamic + * values, calls through function values, the runtime library). A + * whole-program graph of where arrays are held (see "Arrays fields + * hold") follows them from the field reads; + * - the array passed to a constructor where the field takes its length is + * one nothing else keeps: a fresh array (a literal, a copy, the result of + * a function that returns only such arrays or its arguments), a local + * array at its last use (a local built here, moved into the constructor; + * calls that only read or write it in place may come first), a parameter + * that is itself moved (its every caller passes such an array), an array + * that is already a field's, or a `const` array that is never written, + * which is then followed like a field's array. + * + * Arrays are followed shallowly: only the array objects matter, not the + * elements, which stay ordinary mutable instances. */ + +import type { IrExpr, IrFunction, IrGlobal, IrModule, IrStmt, IrType, SrcLoc } from "./ir.js"; +import { typeKey } from "./ir.js"; +import type { InlineArrayPlan } from "./inline-arrays.js"; +import { everyExprChild, everyStmtChild } from "./traverse.js"; + +/** A use that breaks an InlineArray field's contract. */ +export interface InlineArraySourceIssue extends Issue { + readonly declaringClass: string; + readonly field: string; +} + +interface Issue { + readonly loc: SrcLoc; + readonly reason: string; + /** The read of the field whose array the use may reach. */ + readonly origin?: SrcLoc; +} + +/** Array methods that leave their receiver unchanged. */ +const ARR_READS: ReadonlySet = new Set([ + "length", + "getNumber", + "indexEq", + "indexOf", + "includes", + "nextPresent", + "join", + "slice", + "flatCopy", + "flatOne", + "toReversed", + "toSpliced", + "toSortedPrimitive", + "concatSpread", + "with", + "withUndefined", +]); + +/** Array methods whose arguments are arrays of the receiver's type, whose + * elements they add. */ +const SPREADS: ReadonlySet = new Set(["pushSpread", "unshiftSpread", "concatSpread"]); + +/** Array methods whose result is a new array. */ +const ARR_COPIES: ReadonlySet = new Set([ + "slice", + "flatCopy", + "flatOne", + "toReversed", + "toSpliced", + "toSortedPrimitive", + "concatSpread", + "with", + "withUndefined", +]); + +/** Array methods that write their receiver and return it. */ +const RETURNS_RECEIVER: ReadonlySet = new Set([ + "reverse", + "sortPrimitive", + "fill", + "fillUndefined", + "copyWithin", +]); + +/** Library calls that only read their arguments: printing, inspection, + * and JSON.stringify without a replacer function. */ +function readsArguments(e: IrExpr & { kind: "libCall" }): boolean { + if (/^(console|insp|inspect)\./.test(e.fn)) return true; + switch (e.fn) { + case "array.hasProperties": + case "array.hasNamedKey": + case "array.namedKeys": + case "json.stringifyValue": + return true; + case "json.stringifyReplacer": { + const replacer = e.args[1]; + return replacer?.kind === "dynFrom" && replacer.value.kind === "unitLit"; + } + default: + return false; + } +} + +/** Expressions whose value is one of their children's. */ +const PASS_THROUGH: ReadonlySet = new Set([ + "upcast", + "downcast", + "unionWrap", + "unionRetype", + "unionNarrow", + "ternary", + "logical", + "nullish", + "orDefault", + "seqExpr", + "chainRecv", + "optChain", +]); + +/** Expressions that only look at their children. */ +const OBSERVE: ReadonlySet = new Set([ + "toBool", + "strEq", + "unionEq", + "unionFuncEq", + "unionIsTag", + "unionDisc", + "instanceOf", + "instanceOfValue", + "dynScalarEq", + "fieldAbsent", + "recordHas", + "recordOvfHas", + "bin", + "unary", +]); + +/** Expressions this analysis follows: reads, the stores and calls it + * tracks, and the escapes it reports. */ +const MODELLED: ReadonlySet = new Set([ + ...PASS_THROUGH, + ...OBSERVE, + "varRef", + "assignExpr", + "call", + "new", + "virtualCall", + "arrIntrinsic", + "arrayGet", + "arrayHas", + "arrayState", + "fieldGet", + "recordGet", + "mapIntrinsic", + "setIntrinsic", + "setNew", + "mapNew", + "arrayLit", + "recordLit", + "closure", + "dynFrom", + "caughtToDyn", + "callValue", + "libCall", + "intrinsic", +]); + +interface Occurrence { + readonly index: FunctionIndex; + readonly ref: IrExpr & { kind: "varRef" }; + readonly parent: IrExpr | IrStmt; + /** Evaluation order in the function. */ + readonly order: number; + /** The loops around the occurrence, innermost last. */ + readonly loops: readonly IrStmt[]; + /** The arms of conditionals the occurrence is in (`id:arm`), outermost + * first: occurrences in different arms of one conditional never both + * run. */ + readonly branches: readonly string[]; +} + +/** Whether two occurrences sit in different arms of one conditional. */ +function exclusive(a: Occurrence, b: Occurrence): boolean { + const n = Math.min(a.branches.length, b.branches.length); + for (let i = 0; i < n; i++) { + const x = a.branches[i]!, + y = b.branches[i]!; + if (x === y) continue; + return x.slice(0, x.indexOf(":")) === y.slice(0, y.indexOf(":")); + } + return false; +} + +/** Whether a statement list always leaves the block it ends (return, throw, + * break or continue on every path). */ +function exits(stmts: readonly IrStmt[]): boolean { + const last = stmts[stmts.length - 1]; + if (last === undefined) return false; + switch (last.kind) { + case "return": + case "throw": + case "break": + case "continue": + return true; + case "if": + return last.else_ !== null && exits(last.then) && exits(last.else_); + case "block": + return exits(last.body); + default: + return false; + } +} + +type Definition = { stmt: IrStmt; loops: readonly IrStmt[] }; + +/** Every local's occurrences in a function, with each node's parent. */ +class FunctionIndex { + readonly occurrences = new Map(); + readonly declarations = new Map(); + /** Assignment statements, by local (globals are defined by these). */ + readonly assignments = new Map(); + readonly assigned = new Set(); + readonly assignedByExpr = new Set(); + /** The parent of every expression. */ + readonly parents = new Map(); + /** The statement after each statement of a block. */ + readonly next = new Map(); + /** The position of each statement among the occurrences. */ + readonly positions = new Map(); + readonly captured = new Set(); + + constructor(readonly fn: IrFunction) { + let order = 0; + let conditionals = 0; + const loops: IrStmt[] = []; + const branches: string[] = []; + const define = (map: Map, id: string, stmt: IrStmt): void => { + let list = map.get(id); + if (!list) map.set(id, (list = [])); + list.push({ stmt, loops: [...loops] }); + }; + const arm = (id: number, which: string, visit: () => void): void => { + branches.push(`${id}:${which}`); + visit(); + branches.pop(); + }; + const visitExpr = (e: IrExpr, parent: IrExpr | IrStmt): void => { + this.parents.set(e, parent); + if (e.kind === "varRef") { + let list = this.occurrences.get(e.localId); + if (!list) this.occurrences.set(e.localId, (list = [])); + list.push({ + index: this, + ref: e, + parent, + order: order++, + loops: [...loops], + branches: [...branches], + }); + return; + } + if (e.kind === "assignExpr" || e.kind === "incDec") { + this.assigned.add(e.localId); + this.assignedByExpr.add(e.localId); + } + if (e.kind === "closure") for (const id of e.captures) this.captured.add(id); + if (e.kind === "ternary") { + const id = conditionals++; + visitExpr(e.cond, e); + arm(id, "t", () => visitExpr(e.then, e)); + arm(id, "e", () => visitExpr(e.else_, e)); + return; + } + everyExprChild( + e, + (child) => { + visitExpr(child, e); + return true; + }, + (child) => { + visitStmt(child); + return true; + }, + ); + }; + // The statements of a block after an `if` whose one arm always leaves + // the block run only when the other arm ran. + const visitBlock = (stmts: readonly IrStmt[]): void => { + const depth = branches.length; + for (let i = 0; i + 1 < stmts.length; i++) this.next.set(stmts[i]!, stmts[i + 1]!); + for (const s of stmts) { + const id = visitStmt(s); + if (s.kind !== "if" || id === null) continue; + if (exits(s.then)) branches.push(`${id}:e`); + else if (s.else_ !== null && exits(s.else_)) branches.push(`${id}:t`); + } + branches.length = depth; + }; + const inLoop = (s: IrStmt, visit: () => void): void => { + loops.push(s); + visit(); + loops.pop(); + }; + /** Visits `s`; returns the conditional id of an `if`. */ + const visitStmt = (s: IrStmt): number | null => { + this.positions.set(s, order); + switch (s.kind) { + case "varDecl": + define(this.declarations, s.localId, s); + break; + case "assign": + this.assigned.add(s.localId); + define(this.assignments, s.localId, s); + break; + case "if": { + visitExpr(s.cond, s); + const id = conditionals++; + arm(id, "t", () => visitBlock(s.then)); + arm(id, "e", () => visitBlock(s.else_ ?? [])); + return id; + } + case "switch": { + // Case tests run in order until one matches; a case's body runs + // after it, or after the case before falls through into it. + visitExpr(s.disc, s); + for (const c of s.cases) if (c.test) visitExpr(c.test, s); + const id = conditionals++; + let entry = 0; + s.cases.forEach((c, i) => { + if (i > 0 && exits(s.cases[i - 1]!.body)) entry = i; + arm(id, String(entry), () => visitBlock(c.body)); + }); + return null; + } + case "block": + visitBlock(s.body); + return null; + case "while": + inLoop(s, () => { + visitExpr(s.cond, s); + visitBlock(s.body); + }); + return null; + case "doWhile": + inLoop(s, () => { + visitBlock(s.body); + visitExpr(s.cond, s); + }); + return null; + case "for": + if (s.init) visitStmt(s.init); + inLoop(s, () => { + if (s.cond) visitExpr(s.cond, s); + visitBlock(s.body); + if (s.update) visitStmt(s.update); + }); + return null; + case "forOf": + this.assigned.add(s.localId); + visitExpr(s.iterable, s); + inLoop(s, () => visitBlock(s.body)); + return null; + } + everyStmtChild( + s, + (child) => { + visitExpr(child, s); + return true; + }, + (child) => { + visitStmt(child); + return true; + }, + ); + return null; + }; + visitBlock(fn.body); + } +} + +/** Where the value of an occurrence goes: up through the expressions whose + * value it may be. */ +function valueUse(o: Occurrence): { parent: IrExpr | IrStmt | undefined; child: IrExpr } { + let child: IrExpr = o.ref; + let parent: IrExpr | IrStmt | undefined = o.parent; + while (parent !== undefined && givesValueOf(parent, child)) { + child = parent as IrExpr; + parent = o.index.parents.get(child); + } + return { parent, child }; +} + +/** Whether `e` is `undefined` or `null`, as itself or as a union arm. */ +function isUndefined(e: IrExpr): boolean { + while (e.kind === "unionWrap" || e.kind === "upcast") e = e.value; + return e.kind === "unitLit"; +} + +/** Whether an occurrence is (the value of) a returned expression. */ +function returned(o: Occurrence): boolean { + return valueUse(o).parent?.kind === "return"; +} + +/** Whether the value of `parent` may be that of its child `child`. */ +function givesValueOf(parent: IrExpr | IrStmt, child: IrExpr): boolean { + switch (parent.kind) { + case "ternary": + return parent.then === child || parent.else_ === child; + case "seqExpr": + return parent.result === child; + case "upcast": + case "downcast": + case "unionWrap": + case "unionRetype": + case "unionNarrow": + return parent.value === child; + case "logical": + case "nullish": + case "orDefault": + return true; + default: + return false; + } +} + +/** Whether `parent` only reads its array child `ref` without keeping or + * writing it. */ +function readsOnly(parent: IrExpr | IrStmt, ref: IrExpr): boolean { + switch (parent.kind) { + case "arrayGet": + case "arrayHas": + case "arrayState": + return parent.arr === ref; + case "arrIntrinsic": + if (parent.receiver === ref) return ARR_READS.has(parent.method); + // A spread argument gives its elements. + return SPREADS.has(parent.method) && typeKey(ref.type) === typeKey(parent.receiver.type); + case "forOf": + return parent.iterable === ref; + default: + return OBSERVE.has(parent.kind as IrExpr["kind"]); + } +} + +/** Whether `parent` writes its array child `ref` in place (and keeps no + * reference to it). */ +function writesInPlace(parent: IrExpr | IrStmt, ref: IrExpr): boolean { + switch (parent.kind) { + case "arraySet": + case "arraySetLength": + case "arraySetUndefined": + case "arrayDelete": + return parent.arr === ref; + case "arrIntrinsic": + return parent.receiver === ref && !ARR_READS.has(parent.method); + default: + return false; + } +} + +export function checkInlineArraySources( + mod: IrModule, + plan: InlineArrayPlan, +): InlineArraySourceIssue[] { + if (plan.classes.size === 0) return []; + return new SourceAnalysis(mod, plan).run(); +} + +class SourceAnalysis { + private readonly functions = new Map(); + private readonly indexes = new Map(); + /** Functions that are called other than directly (function values, + * library exports): their callers are not all known. */ + private readonly escapedFunctions = new Set(); + private readonly methodImpls = new Map(); + /** The hierarchy roots of the planned fields' element classes: arrays of + * any class in those hierarchies may be a field's (a cast can retype + * one). */ + private readonly elementRoots = new Set(); + private readonly bases = new Map(); + private readonly globals = new Map(); + /** The slot graph (see "Arrays fields hold"). */ + private readonly edges = new Map>(); + private readonly sites = new Map(); + /** The slots of inline field reads, with the reads' locations. */ + private readonly seeds = new Map(); + /** The field each read reads. */ + private readonly origins = new Map(); + /** Slots that may hold an inline field's array, with the field read it + * may come from. */ + private readonly pinned = new Map(); + /** `followed` by type key. */ + private readonly followedTypes = new Map(); + /** Parameters (`fn\0index`) whose array moves into an inline field. */ + private readonly moved = new Set(); + /** Functions whose every result is an array nothing else keeps, a + * field's, or one of the `passed` parameters. */ + private readonly factories = new Set(); + private readonly passed = new Map>(); + /** The locals `freshLocal` is answering for. */ + private readonly inProgress = new Set(); + /** Where `share` records the slots it shares, with their definitions; + * null when sharing is not allowed. */ + private shareSink: Map | null = null; + /** The slots each factory's results may share. */ + private readonly factoryShares = new Map>(); + /** Parameters (by index) a function may return as they came: a call's + * value may be the array passed there. */ + private readonly returnedParams = new Map>(); + /** The slots that carry a returned parameter to a function's returns: + * the parameter's own and the locals between. Their other values reach + * the function's result slot directly. */ + private readonly returnCarriers = new Map>(); + private readonly issues: InlineArraySourceIssue[] = []; + private readonly reported = new Set(); + + constructor( + private readonly mod: IrModule, + private readonly plan: InlineArrayPlan, + ) { + for (const fn of mod.functions) this.functions.set(fn.name, fn); + for (const cls of mod.classes ?? []) + for (const m of cls.methods ?? []) { + let list = this.methodImpls.get(m); + if (!list) this.methodImpls.set(m, (list = [])); + list.push(`%${cls.name}.${m}`); + } + for (const cls of mod.classes ?? []) this.bases.set(cls.name, cls.base); + for (const g of mod.globals ?? []) this.globals.set(g.id, g); + for (const cls of plan.classes.values()) this.elementRoots.add(this.rootOf(cls.elemClass)); + } + + private rootOf(className: string): string { + let name = className; + for (let base = this.bases.get(name); base !== undefined; base = this.bases.get(name)) + name = base; + return name; + } + + private index(fn: IrFunction): FunctionIndex { + let index = this.indexes.get(fn.name); + if (!index) this.indexes.set(fn.name, (index = new FunctionIndex(fn))); + return index; + } + + run(): InlineArraySourceIssue[] { + this.collectCalls(); + this.computeLent(); + this.computeMoves(); + this.computeFactories(); + this.computePinned(); + this.checkSources(); + this.checkWrites(); + return this.issues; + } + + /** The planned class of the first hierarchy, for issues not tied to one. */ + private anyField(): { declaringClass: string; field: string } { + const cls = this.plan.classes.values().next().value!; + return { declaringClass: cls.declaringClass, field: cls.field }; + } + + private report( + issue: Issue, + owner = (issue.origin && this.origins.get(issue.origin)) ?? this.anyField(), + ): void { + // One issue per site. + const key = `${issue.loc.file}:${issue.loc.start}`; + if (this.reported.has(key)) return; + this.reported.add(key); + this.issues.push({ ...owner, ...issue }); + } + + // ── Calls ── + + private collectCalls(): void { + for (const e of this.mod.lib?.exports ?? []) this.escapedFunctions.add(e.fnName); + for (const fn of this.mod.functions) + this.eachExpr(fn, (e) => { + if (e.kind === "closure") this.escapedFunctions.add(e.fnName); + }); + } + + /** The functions a call, construction or method call may run, with the + * offset of its first argument in their parameters (a construction's + * arguments follow `this`). */ + private callees(e: IrExpr): { callee: string; offset: number }[] { + switch (e.kind) { + case "call": + return [{ callee: e.callee, offset: 0 }]; + case "new": + return [{ callee: `%${e.className}.constructor`, offset: 1 }]; + case "virtualCall": { + // `targets` are the classes whose own declaration may run. + const impls = e.targets?.map((c) => `%${c}.${e.method}`) ?? this.methodImpls.get(e.method); + return (impls ?? []).map((callee) => ({ callee, offset: 0 })); + } + default: + return []; + } + } + + private eachExpr(fn: IrFunction, visit: (e: IrExpr) => void): void { + const expr = (e: IrExpr): boolean => { + visit(e); + return everyExprChild(e, expr, stmt); + }; + const stmt = (s: IrStmt): boolean => everyStmtChild(s, expr, stmt); + for (const s of fn.body) stmt(s); + } + + /** The planned class an inline `new` constructs, with the argument the + * field takes its length from. */ + private inlineNew(e: IrExpr): { arg: number } | null { + if (e.kind !== "new") return null; + const cls = this.plan.classes.get(e.className); + return cls === undefined ? null : { arg: cls.lengthArg }; + } + + /** Whether passing `arg` in `call` hands it to a parameter that moves it + * into an inline field. */ + private consumes(call: IrExpr, index: number): boolean { + const inline = this.inlineNew(call); + if (inline !== null) return index === inline.arg; + return this.callees(call).some(({ callee, offset }) => + this.moved.has(`${callee}\0${index + offset}`), + ); + } + + // ── Lent arrays ── + + /** Array parameters (`fn\0index`) whose function reads the array or + * writes it in place and keeps no reference to it: passing an array there + * lends it for the call. */ + private readonly lent = new Set(); + /** The lent parameters whose function only reads the array. */ + private readonly readParams = new Set(); + + /** How an occurrence uses its array: it only reads it, writes it in + * place, or does something else (keeps it, hands it on). */ + private use(o: Occurrence, depth = 0): "read" | "write" | "other" { + // The expression whose value it is, such as a narrowing of it. + const { parent: p, child } = valueUse(o); + if (p === undefined) return "other"; + if (readsOnly(p, child)) return "read"; + if (writesInPlace(p, child)) return "write"; + // A `const` copy of the reference (as `for...of` makes) uses the array + // as its own uses do. + const copy = this.copyUses(o, depth); + if (copy !== null) { + const uses = copy.map((u) => this.use(u, depth + 1)); + if (uses.includes("other")) return "other"; + return uses.includes("write") ? "write" : "read"; + } + if (p.kind !== "call" && p.kind !== "new" && p.kind !== "virtualCall") return "other"; + const i = p.args.indexOf(child); + const params = this.callees(p).map(({ callee, offset }) => `${callee}\0${i + offset}`); + if (i < 0 || params.length === 0) return "other"; + if (params.every((k) => this.readParams.has(k))) return "read"; + if (params.every((k) => this.lent.has(k))) return "write"; + return "other"; + } + + /** The occurrences of the `const` local an occurrence initializes, or + * null when it initializes none (or one written again). */ + private copyUses(o: Occurrence, depth: number): Occurrence[] | null { + const { parent: p, child } = valueUse(o); + if (p?.kind !== "varDecl" || p.init !== child || depth >= 4) return null; + const index = o.index; + if (index.assigned.has(p.localId) || index.captured.has(p.localId)) return null; + if ((index.declarations.get(p.localId) ?? []).length !== 1) return null; + return index.occurrences.get(p.localId) ?? []; + } + + /** The position of the last use an occurrence makes, through `const` + * copies. */ + private lastPosition(o: Occurrence, depth = 0): number { + const copy = this.copyUses(o, depth); + if (copy === null) return o.order; + return Math.max(o.order, ...copy.map((u) => this.lastPosition(u, depth + 1))); + } + + /** Whether `last` hands the array on for good: every other occurrence that + * can run with it only reads the array, or writes it in place before. */ + private lastUse(occurrences: readonly Occurrence[], last: Occurrence): boolean { + for (const o of occurrences) { + if (o === last || exclusive(o, last)) continue; + const use = this.use(o); + if (use === "other" || (use === "write" && this.lastPosition(o) > last.order)) return false; + } + return true; + } + + /** The largest sets of lent and read-only parameters: start from every + * array parameter and drop those with another use, to a fixpoint. */ + private computeLent(): void { + for (const fn of this.mod.functions) + fn.params.forEach((p, i) => { + if (!this.holdsFieldArray(p.type)) return; + const index = this.index(fn); + const local = fn.locals.find((l) => l.id === p.localId); + if (index.assigned.has(p.localId) || index.captured.has(p.localId) || local?.boxed === true) + return; + this.lent.add(`${fn.name}\0${i}`); + this.readParams.add(`${fn.name}\0${i}`); + }); + for (let changed = true; changed;) { + changed = false; + for (const fn of this.mod.functions) + fn.params.forEach((p, i) => { + const key = `${fn.name}\0${i}`; + if (!this.lent.has(key)) return; + const uses = (this.index(fn).occurrences.get(p.localId) ?? []).map((o) => this.use(o)); + if (uses.includes("other")) { + this.lent.delete(key); + this.readParams.delete(key); + changed = true; + } else if (uses.includes("write") && this.readParams.delete(key)) changed = true; + }); + } + } + + // ── Moves ── + + /** Parameters that only reach an inline field: each is used only to be + * read, written in place, and finally passed where an inline field takes + * its array; every caller then passes an array nothing else keeps. */ + private computeMoves(): void { + // Subclass constructors forward their parameter to the base's: start + // from the constructors' own length parameters. + for (const [name, cls] of this.plan.classes) + this.moved.add(`%${name}.constructor\0${cls.lengthArg + 1}`); + // Grow: parameters whose last use moves them. + for (let changed = true; changed;) { + changed = false; + for (const fn of this.mod.functions) { + if (this.escapedFunctions.has(fn.name)) continue; + fn.params.forEach((p, i) => { + const key = `${fn.name}\0${i}`; + if (this.moved.has(key)) return; + if (this.movedAtLastUse(fn, p.localId)) { + this.moved.add(key); + changed = true; + } + }); + } + } + } + + /** Whether the parameter `localId` moves into an inline field: every use + * reads it, writes it in place, or passes it on where an inline field + * takes its array as its last use on that path, outside any loop. */ + private movedAtLastUse(fn: IrFunction, localId: string): boolean { + const index = this.index(fn); + if (index.assigned.has(localId) || index.captured.has(localId)) return false; + const local = fn.locals.find((l) => l.id === localId); + if (local === undefined || local.boxed === true) return false; + const occurrences = index.occurrences.get(localId) ?? []; + const moves = occurrences.filter((o) => this.isMove(o)); + if (moves.length === 0) return false; + return moves.every((m) => m.loops.length === 0 && this.lastUse(occurrences, m)); + } + + /** Whether the occurrence hands the array on to an inline field. */ + private isMove(o: Occurrence): boolean { + const { parent: p, child } = valueUse(o); + if (p === undefined || (p.kind !== "new" && p.kind !== "call" && p.kind !== "virtualCall")) + return false; + const i = p.args.indexOf(child); + return i >= 0 && this.consumes(p, i); + } + + // ── Fresh values ── + + /** Whether `e` (in `fn`) is an array nothing else keeps, or one an + * inline field already holds. */ + private fresh(fn: IrFunction, e: IrExpr): boolean { + switch (e.kind) { + case "arrayLit": + return true; + case "arrIntrinsic": + if (RETURNS_RECEIVER.has(e.method)) return this.fresh(fn, e.receiver); + return ARR_COPIES.has(e.method); + case "call": + case "virtualCall": { + // A factory's result, and the arrays it may give back as they came. + const callees = this.callees(e); + return ( + callees.length > 0 && + callees.every( + ({ callee, offset }) => + this.factories.has(callee) && + this.shareAll(this.factoryShares.get(callee)) && + [...(this.passed.get(callee) ?? [])].every((i) => { + const arg = e.args[i - offset]; + return arg !== undefined && this.fresh(fn, arg); + }), + ) + ); + } + case "varRef": { + if (e.localId.startsWith("%g.")) + return this.fieldArray(fn, e) || this.freshGlobal(fn, e) || this.share(fn, e); + const param = fn.params.findIndex((p) => p.localId === e.localId); + if (param >= 0) return this.moved.has(`${fn.name}\0${param}`); + return this.fieldArray(fn, e) || this.freshLocal(fn, e) || this.share(fn, e); + } + case "fieldGet": + return this.isFieldRead(e); + case "seqExpr": + return this.fresh(fn, e.result); + case "ternary": + return this.fresh(fn, e.then) && this.fresh(fn, e.else_); + case "logical": + case "nullish": + case "orDefault": + return everyExprChild( + e, + (child) => this.fresh(fn, child), + () => false, + ); + case "upcast": + case "downcast": + case "unionWrap": + case "unionRetype": + case "unionNarrow": + return this.fresh(fn, e.value); + default: + return false; + } + } + + /** A local whose only definition is a fresh array and whose occurrence + * `ref` is its last use on every path, outside any loop the declaration + * is not in. */ + private freshLocal(fn: IrFunction, ref: IrExpr & { kind: "varRef" }): boolean { + const id = ref.localId; + const index = this.index(fn); + if (index.assignedByExpr.has(id) || index.captured.has(id)) return false; + const declarations = index.declarations.get(id) ?? []; + if (declarations.length !== 1 || !this.movedAt(index, ref, declarations[0]!)) return false; + // Every value the local takes (a `let` may be assigned again) is fresh, + // or undefined. + const key = `${fn.name}\0${id}`; + if (this.inProgress.has(key)) return false; + this.inProgress.add(key); + try { + const init = (declarations[0]!.stmt as IrStmt & { kind: "varDecl" }).init; + const values = [ + init, + ...(index.assignments.get(id) ?? []).map( + (d) => (d.stmt as IrStmt & { kind: "assign" }).value, + ), + ]; + return values.every((v) => + v === null ? index.assignments.has(id) : isUndefined(v) || this.fresh(fn, v), + ); + } finally { + this.inProgress.delete(key); + } + } + + /** Whether `ref` is the last use of its local or global on every path, + * outside any loop its definition is not in. */ + private movedAt( + index: FunctionIndex, + ref: IrExpr & { kind: "varRef" }, + def: Definition, + ): boolean { + const occurrences = index.occurrences.get(ref.localId) ?? []; + const at = occurrences.find((o) => o.ref === ref); + if (at === undefined) return false; + // A move followed by a fresh value for the local (in a loop that builds + // one array per pass): later uses see the new array. + const kill = this.killAfter(at); + if (kill !== null) + return this.lastUse( + occurrences.filter((o) => o.order < kill), + at, + ); + return at.loops.length <= def.loops.length && this.lastUse(occurrences, at); + } + + /** The position of an assignment of a fresh array (or undefined) to the + * occurrence's local by the statement right after the occurrence's. */ + private killAfter(at: Occurrence): number | null { + const index = at.index; + let node: IrExpr | IrStmt | undefined = at.parent; + while (node !== undefined && !index.positions.has(node as IrStmt)) + node = index.parents.get(node as IrExpr); + const next = node === undefined ? undefined : index.next.get(node as IrStmt); + if (next?.kind !== "assign" || next.localId !== at.ref.localId) return null; + if (!isUndefined(next.value) && !this.fresh(index.fn, next.value)) return null; + return index.positions.get(next) ?? null; + } + + /** Whether `ref` is a `const` local or global from a fresh array that is + * only read or passed where an inline field takes it: such an array may + * be shared, and is followed like a field's, so that writes to it through + * other references are reported. Only while `shareSink` collects them. */ + private share(fn: IrFunction, ref: IrExpr & { kind: "varRef" }): boolean { + if (this.shareSink === null) return false; + const id = ref.localId; + let def: { fn: IrFunction; value: IrExpr; loc: SrcLoc }; + if (id.startsWith("%g.")) { + const source = this.globalSource(id); + if (source === null) return false; + def = { + fn: source.fn, + value: (source.def.stmt as IrStmt & { kind: "assign" }).value, + loc: source.def.stmt.loc, + }; + } else { + const index = this.index(fn); + if (index.assigned.has(id) || index.captured.has(id)) return false; + const declarations = index.declarations.get(id) ?? []; + const decl = declarations[0]?.stmt as (IrStmt & { kind: "varDecl" }) | undefined; + if (declarations.length !== 1 || !decl?.init) return false; + def = { fn, value: decl.init, loc: decl.loc }; + } + const slot = this.localSlot(fn, id); + if (!this.shareSink.has(slot)) { + const functions = id.startsWith("%g.") ? this.mod.functions : [fn]; + const constant = functions.every((f) => + (this.index(f).occurrences.get(id) ?? []).every( + (o) => this.use(o) === "read" || this.isMove(o) || returned(o), + ), + ); + if (!constant || !this.fresh(def.fn, def.value)) return false; + } + this.shareSink.set(slot, def.loc); + return true; + } + + /** Shares the arrays a factory's results may be (see `share`). */ + private shareAll(shares: ReadonlyMap | undefined): boolean { + if (shares === undefined) return true; + if (this.shareSink === null) return false; + for (const [slot, loc] of shares) this.shareSink.set(slot, loc); + return true; + } + + /** Whether `e` is always an array a field already holds: a read of an + * inline field, or a `const` local or global initialized from one. Such + * an array may be shared: every reference to it is followed. */ + private fieldArray(fn: IrFunction, e: IrExpr, depth = 0): boolean { + if (depth > 16) return false; + switch (e.kind) { + case "fieldGet": + return this.isFieldRead(e); + case "varRef": { + if (e.localId.startsWith("%g.")) { + const source = this.globalSource(e.localId); + if (source === null) return false; + const value = (source.def.stmt as IrStmt & { kind: "assign" }).value; + return this.fieldArray(source.fn, value, depth + 1); + } + if (fn.params.some((p) => p.localId === e.localId)) return false; + const index = this.index(fn); + if (index.assigned.has(e.localId) || index.captured.has(e.localId)) return false; + const declarations = index.declarations.get(e.localId) ?? []; + if (declarations.length !== 1) return false; + const init = (declarations[0]!.stmt as IrStmt & { kind: "varDecl" }).init; + return init !== null && this.fieldArray(fn, init, depth + 1); + } + case "seqExpr": + return this.fieldArray(fn, e.result, depth + 1); + case "ternary": + return this.fieldArray(fn, e.then, depth + 1) && this.fieldArray(fn, e.else_, depth + 1); + case "upcast": + case "downcast": + case "unionWrap": + case "unionRetype": + case "unionNarrow": + return this.fieldArray(fn, e.value, depth + 1); + default: + return false; + } + } + + /** A `const` global from a fresh array, read here for the last time, in + * the function that defines it, and only read elsewhere. */ + private freshGlobal(fn: IrFunction, ref: IrExpr & { kind: "varRef" }): boolean { + const source = this.globalSource(ref.localId); + if (source === null) return false; + const value = (source.def.stmt as IrStmt & { kind: "assign" }).value; + if (!this.fresh(source.fn, value)) return false; + if (fn !== source.fn || !this.movedAt(this.index(fn), ref, source.def)) return false; + return this.mod.functions.every( + (other) => + other === fn || + (this.index(other).occurrences.get(ref.localId) ?? []).every((o) => this.use(o) === "read"), + ); + } + + /** The one assignment of an immutable global. */ + private globalSource(id: string): { fn: IrFunction; def: Definition } | null { + if (this.globals.get(id)?.mutable !== false) return null; + let source: { fn: IrFunction; def: Definition } | null = null; + for (const fn of this.mod.functions) { + const index = this.index(fn); + if (index.assignedByExpr.has(id)) return null; + for (const def of index.assignments.get(id) ?? []) { + if (source !== null) return null; + source = { fn, def }; + } + } + return source; + } + + private isFieldRead(e: IrExpr): boolean { + if (e.kind !== "fieldGet") return false; + const cls = this.plan.classes.get(e.className); + return cls !== undefined && cls.field === e.field; + } + + // ── Arrays fields hold ── + // + // A whole-program graph of slots: locals and parameters (`l:fn\0local`), + // globals (`g:global`), results (`r:fn`), the fields of a class hierarchy + // (`f:root\0field`), record fields (`s:shape\0field`), arrays, Maps and + // Sets made at one expression (`a:fn\0n`), and reads of an inline field + // (`v:n`). An edge says that a value in one slot may reach another. + // `c:` is the elements (keys and values) of the arrays, Maps and + // Sets in a slot: an edge from one slot to another is one between their + // elements too. The slots a field read reaches may hold a field's array. + + /** A field of every class in a hierarchy. */ + private fieldSlot(className: string, field: string): string { + return `f:${this.rootOf(className)}\0${field}`; + } + + private localSlot(fn: IrFunction, id: string): string { + return id.startsWith("%g.") ? `g:${id}` : `l:${fn.name}\0${id}`; + } + + /** The slot of the array, Map or Set an expression makes. */ + private site(fn: IrFunction, e: IrExpr): string { + let slot = this.sites.get(e); + if (slot === undefined) this.sites.set(e, (slot = `a:${fn.name}\0${this.sites.size}`)); + return slot; + } + + /** Whether values of `t` can be arrays a planned field holds: arrays of + * a class in the element classes' hierarchies, directly or as a union + * arm. */ + private holdsFieldArray(t: IrType): boolean { + if (t.kind === "array") + return t.elem.kind === "object" && this.elementRoots.has(this.rootOf(t.elem.className)); + if (t.kind === "union") return this.arms(t).some((arm) => this.holdsFieldArray(arm)); + return false; + } + + /** Whether values of `t` can be, or be arrays, Maps or Sets that contain, + * arrays a planned field holds: the values the graph follows. */ + private followed(t: IrType, depth = 0): boolean { + if (depth > 4) return false; + const key = typeKey(t); + const known = this.followedTypes.get(key); + if (known !== undefined) return known; + let result: boolean; + switch (t.kind) { + case "array": + result = this.holdsFieldArray(t) || this.followed(t.elem, depth + 1); + break; + case "set": + result = this.followed(t.elem, depth + 1); + break; + case "map": + result = this.followed(t.key, depth + 1) || this.followed(t.value, depth + 1); + break; + case "union": + result = this.arms(t).some((arm) => this.followed(arm, depth + 1)); + break; + default: + result = false; + } + this.followedTypes.set(key, result); + return result; + } + + private arms(t: IrType & { kind: "union" }): readonly IrType[] { + return (this.mod.unions ?? []).find((u) => u.id === t.unionId)?.arms ?? []; + } + + /** The slots the value of `e` may come from. */ + private slotsOf(fn: IrFunction, e: IrExpr): string[] { + const contents = (inner: IrExpr): string[] => this.slotsOf(fn, inner).map((s) => `c:${s}`); + const children = (): string[] => { + const out: string[] = []; + everyExprChild( + e, + (child) => { + out.push(...this.slotsOf(fn, child)); + return true; + }, + () => true, + ); + return out; + }; + switch (e.kind) { + case "fieldGet": + if (!this.isFieldRead(e)) return [this.fieldSlot(e.className, e.field)]; + return [this.seed(fn, e)]; + case "varRef": + return [this.localSlot(fn, e.localId)]; + case "call": + case "virtualCall": { + const slots: string[] = []; + for (const { callee, offset } of this.callees(e)) { + slots.push(`r:${callee}`); + for (const i of this.returnedParams.get(callee) ?? []) { + const arg = e.args[i - offset]; + if (arg !== undefined) slots.push(...this.slotsOf(fn, arg)); + } + } + return slots; + } + case "recordGet": + return e.obj.type.kind === "record" ? [`s:${e.obj.type.shapeId}\0${e.field}`] : []; + case "arrayGet": + return contents(e.arr); + case "mapIntrinsic": + case "setIntrinsic": + if (e.method === "set" || e.method === "add") return this.slotsOf(fn, e.receiver); + if (e.method === "clone" || e.method === "keySet" || e.method === "valueSet") + return [this.site(fn, e)]; + return contents(e.receiver); + case "arrIntrinsic": + if (RETURNS_RECEIVER.has(e.method)) return this.slotsOf(fn, e.receiver); + if (e.method === "pop" || e.method === "shift") return contents(e.receiver); + if (ARR_COPIES.has(e.method) || e.method === "splice" || e.method === "spliceInsert") + return [this.site(fn, e)]; + return []; + case "arrayLit": + case "setNew": + case "mapNew": + return [this.site(fn, e)]; + case "callValue": + return ["r:*"]; + case "seqExpr": + return this.slotsOf(fn, e.result); + case "ternary": + return [...this.slotsOf(fn, e.then), ...this.slotsOf(fn, e.else_)]; + case "logical": + case "nullish": + case "orDefault": + return children(); + default: + return PASS_THROUGH.has(e.kind) ? children() : []; + } + } + + /** The slot of a read of an inline field. */ + private seed(fn: IrFunction, e: IrExpr): string { + const slot = this.site(fn, e).replace(/^a:/, "v:"); + if (!this.seeds.has(slot)) { + this.seeds.set(slot, e.loc); + this.pinned.set(slot, e.loc); + if (e.kind === "fieldGet") this.origins.set(e.loc, this.ownerOf(e.className)); + } + return slot; + } + + private edge(from: string, to: string): void { + let list = this.edges.get(from); + if (!list) this.edges.set(from, (list = new Set())); + list.add(to); + } + + /** Whether `e` may evaluate to an array an inline field holds. */ + private pinnedExpr(fn: IrFunction, e: IrExpr): boolean { + return this.origin(fn, e) !== undefined; + } + + /** The field read whose array `e` may be. */ + private origin(fn: IrFunction, e: IrExpr): SrcLoc | undefined { + if (!this.holdsFieldArray(e.type)) return undefined; + for (const slot of this.slotsOf(fn, e)) { + const origin = this.pinned.get(slot); + if (origin !== undefined) return origin; + } + return undefined; + } + + /** The field read whose array values of type `t` in `slots`, or the + * arrays, Maps, Sets and records inside them, may hold. */ + private originWithin(slots: readonly string[], t: IrType, depth = 0): SrcLoc | undefined { + if (depth > 4) return undefined; + if (depth > 0 && this.holdsFieldArray(t)) + for (const slot of slots) { + const origin = this.pinned.get(slot); + if (origin !== undefined) return origin; + } + const elements = slots.map((s) => `c:${s}`); + switch (t.kind) { + case "array": + case "set": + return this.originWithin(elements, t.elem, depth + 1); + case "map": + return ( + this.originWithin(elements, t.key, depth + 1) ?? + this.originWithin(elements, t.value, depth + 1) + ); + case "union": + for (const arm of this.arms(t)) { + const origin = this.originWithin(slots, arm, depth + 1); + if (origin !== undefined) return origin; + } + return undefined; + case "record": + for (const f of (this.mod.records ?? []).find((r) => r.id === t.shapeId)?.fields ?? []) { + const origin = this.originWithin([`s:${t.shapeId}\0${f.name}`], f.type, depth + 1); + if (origin !== undefined) return origin; + } + return undefined; + default: + return undefined; + } + } + + /** The slots of the parameters of `fn` that only ever hold the caller's + * arguments (never assigned or captured), by index. */ + private ownParams(fn: IrFunction): Map { + const index = this.index(fn); + const own = new Map(); + if (fn.async || fn.generator) return own; + fn.params.forEach((p, i) => { + if (!index.assigned.has(p.localId) && !index.captured.has(p.localId)) + own.set(this.localSlot(fn, p.localId), i); + }); + return own; + } + + /** The parameters each function may return as they came, directly or + * through its locals, to a fixpoint (a returned call gives back what its + * callee does). A call's value is then the arguments passed there, not a + * result every caller shares. */ + private computeReturnedParams(): void { + for (let changed = true; changed;) { + changed = false; + for (const fn of this.mod.functions) { + const own = this.ownParams(fn); + if (own.size === 0 || !this.followed(fn.returnType)) continue; + // The function's locals that take a value from another of them. + const local = `l:${fn.name}\0`; + const aliases = new Map>(); + const returned = new Set(); + const alias = (to: string, value: IrExpr): void => { + for (const from of this.slotsOf(fn, value)) { + if (!from.startsWith(local)) continue; + let set = aliases.get(from); + if (!set) aliases.set(from, (set = new Set())); + set.add(to); + } + }; + const expr = (e: IrExpr): boolean => { + if (e.kind === "assignExpr") alias(this.localSlot(fn, e.localId), e.value); + return everyExprChild(e, expr, stmt); + }; + const stmt = (s: IrStmt): boolean => { + if (s.kind === "varDecl" && s.init) alias(this.localSlot(fn, s.localId), s.init); + else if (s.kind === "assign") alias(this.localSlot(fn, s.localId), s.value); + else if (s.kind === "return" && s.value) + for (const slot of this.slotsOf(fn, s.value)) returned.add(slot); + return everyStmtChild(s, expr, stmt); + }; + for (const s of fn.body) stmt(s); + // From each parameter, the locals it reaches; those that reach a + // return carry it back to the caller. + const carriers = new Set(); + let set = this.returnedParams.get(fn.name); + for (const [param, i] of own) { + const reached = new Map([[param, null]]); + const queue = [param]; + while (queue.length > 0) + for (const to of aliases.get(queue.pop()!) ?? []) + if (!reached.has(to)) { + reached.set(to, null); + queue.push(to); + } + const back = [...reached.keys()].filter((slot) => returned.has(slot)); + if (back.length === 0) continue; + if (!set?.has(i)) { + if (!set) this.returnedParams.set(fn.name, (set = new Set())); + set.add(i); + changed = true; + } + // The locals on the way from the parameter to a return. + for (const slot of reached.keys()) + if (slot === param || this.reaches(aliases, slot, back)) carriers.add(slot); + } + if (carriers.size > 0) this.returnCarriers.set(fn.name, carriers); + } + } + } + + private reaches( + aliases: ReadonlyMap>, + from: string, + targets: readonly string[], + ): boolean { + const seen = new Set([from]); + const queue = [from]; + while (queue.length > 0) { + const slot = queue.pop()!; + if (targets.includes(slot)) return true; + for (const to of aliases.get(slot) ?? []) + if (!seen.has(to)) { + seen.add(to); + queue.push(to); + } + } + return false; + } + + /** Builds the graph and marks every slot a field read reaches. */ + private computePinned(): void { + this.computeReturnedParams(); + for (const fn of this.mod.functions) this.flow(fn, null); + this.propagate([...this.seeds.keys()]); + } + + /** Marks the slots the queued pinned slots reach. */ + private propagate(queue: string[]): void { + const pin = (slot: string, origin: SrcLoc): void => { + if (this.pinned.has(slot)) return; + this.pinned.set(slot, origin); + queue.push(slot); + }; + while (queue.length > 0) { + const slot = queue.pop()!; + const origin = this.pinned.get(slot)!; + // The edges of the slot, and those of the slots whose elements it is. + let prefix = ""; + for (let inner = slot; ; inner = inner.slice(2)) { + for (const to of this.edges.get(inner) ?? []) pin(prefix + to, origin); + if (!inner.startsWith("c:")) break; + prefix += "c:"; + } + } + } + + /** Functions whose every result is an array nothing else keeps, or one + * of their parameters (`passed`) that they only read, write in place and + * return. */ + private computeFactories(): void { + for (let changed = true; changed;) { + changed = false; + for (const fn of this.mod.functions) { + if (this.factories.has(fn.name) || fn.async || fn.generator) continue; + if (!this.holdsFieldArray(fn.returnType)) continue; + let returns = 0; + let ok = true; + const passed = new Set(); + const shares = new Map(); + this.shareSink = shares; + const visit = (s: IrStmt): boolean => { + if (s.kind === "return") { + returns++; + const local = s.value?.kind === "varRef" ? s.value.localId : null; + const param = local !== null ? this.passedParam(fn, local) : -1; + const carried = local !== null && param < 0 ? this.carrier(fn, local) : null; + if (param >= 0) passed.add(param); + else if (carried !== null && carried.size > 0) + for (const i of carried) { + if (this.passedParam(fn, fn.params[i]!.localId) < 0) ok = false; + passed.add(i); + } + else if (!s.value || !this.fresh(fn, s.value)) ok = false; + } + return ok && everyStmtChild(s, () => true, visit); + }; + for (const s of fn.body) visit(s); + this.shareSink = null; + if (ok && returns > 0) { + this.factories.add(fn.name); + if (passed.size > 0) this.passed.set(fn.name, passed); + if (shares.size > 0) this.factoryShares.set(fn.name, shares); + changed = true; + } + } + } + } + + /** The index of the parameter `localId` when `fn` only reads it, writes + * it in place and returns it, directly or through `carrier` locals; -1 + * otherwise. */ + private passedParam(fn: IrFunction, localId: string): number { + const i = fn.params.findIndex((p) => p.localId === localId); + if (i < 0) return -1; + const index = this.index(fn); + if (index.assigned.has(localId) || index.captured.has(localId)) return -1; + const uses = index.occurrences.get(localId) ?? []; + const carried = (o: Occurrence): boolean => { + const { parent: p, child } = valueUse(o); + if (p?.kind === "varDecl" && p.init === child) return this.carrier(fn, p.localId) !== null; + if (p?.kind === "assign" && p.value === child) return this.carrier(fn, p.localId) !== null; + return false; + }; + return uses.every((o) => returned(o) || this.use(o) !== "other" || carried(o)) ? i : -1; + } + + /** The parameters a local carries to a return when its values are fresh + * arrays and parameters, and it is only read, written in place and + * returned; null otherwise. */ + private carrier(fn: IrFunction, localId: string): Set | null { + if (fn.params.some((p) => p.localId === localId)) return null; + const index = this.index(fn); + if (index.assignedByExpr.has(localId) || index.captured.has(localId)) return null; + const declarations = index.declarations.get(localId) ?? []; + if (declarations.length !== 1) return null; + const key = `carrier:${fn.name}\0${localId}`; + if (this.inProgress.has(key)) return null; + this.inProgress.add(key); + try { + const init = (declarations[0]!.stmt as IrStmt & { kind: "varDecl" }).init; + const values = [ + init, + ...(index.assignments.get(localId) ?? []).map( + (d) => (d.stmt as IrStmt & { kind: "assign" }).value, + ), + ]; + const params = new Set(); + for (const v of values) { + if (v === null || isUndefined(v)) continue; + const param = + v.kind === "varRef" ? fn.params.findIndex((p) => p.localId === v.localId) : -1; + if (param >= 0) params.add(param); + else if (!this.fresh(fn, v)) return null; + } + const uses = index.occurrences.get(localId) ?? []; + return uses.every((o) => returned(o) || this.use(o) !== "other") ? params : null; + } finally { + this.inProgress.delete(key); + } + } + + /** One pass over `fn`: without `check`, the graph's edges; with it, the + * writes and escapes of arrays fields may hold. */ + private flow(fn: IrFunction, check: ((issue: Issue) => void) | null): void { + const carriers = this.returnCarriers.get(fn.name); + const hold = (to: string, value: IrExpr): void => { + if (check !== null || !this.followed(value.type)) return; + for (const from of this.slotsOf(fn, value)) { + this.edge(from, to); + if (carriers?.has(to) && !carriers.has(from)) this.edge(from, `r:${fn.name}`); + } + }; + // Into the elements of the arrays, Maps or Sets `container` may be. + const store = (container: IrExpr | string, value: IrExpr): void => { + const slots = typeof container === "string" ? [container] : this.slotsOf(fn, container); + for (const slot of slots) hold(`c:${slot}`, value); + }; + // The elements of `from` join those of `to`. + const spread = (from: IrExpr, to: IrExpr | string): void => { + if (check !== null) return; + const targets = typeof to === "string" ? [to] : this.slotsOf(fn, to); + for (const a of this.slotsOf(fn, from)) + for (const b of targets) this.edge(`c:${a}`, `c:${b}`); + }; + const write = (target: IrExpr, loc: SrcLoc, what: string): void => { + if (check === null) return; + const origin = this.origin(fn, target); + if (origin !== undefined) + check({ + loc, + origin, + reason: `${what} writes to an array that may be an InlineArray field's`, + }); + }; + // `deep`: arrays inside the value escape too (dynamic and library code + // reads them other than through the reads this pass follows). + const escape = (value: IrExpr, loc: SrcLoc, what: string, deep = true): void => { + if (check === null) return; + const origin = + this.origin(fn, value) ?? + (deep ? this.originWithin(this.slotsOf(fn, value), value.type) : undefined); + if (origin !== undefined) + check({ + loc, + origin, + reason: `an array that may be an InlineArray field's reaches ${what}, where writes are not checked`, + }); + }; + const args = (call: IrExpr & { args: IrExpr[] }): void => { + const callees = this.callees(call); + for (const { callee, offset } of callees) { + const target = this.functions.get(callee); + call.args.forEach((arg, i) => { + const param = target?.params[i + offset]; + if (!target || !param) return escape(arg, call.loc, `a call to ${callee}`); + hold(`l:${callee}\0${param.localId}`, arg); + }); + } + if (callees.length === 0) for (const arg of call.args) escape(arg, call.loc, "a method call"); + }; + // Dynamic values a library call that only reads them makes for it. + const readSinks = new Set(); + const expr = (e: IrExpr): boolean => { + switch (e.kind) { + case "assignExpr": + hold(this.localSlot(fn, e.localId), e.value); + break; + case "call": + case "new": + case "virtualCall": + args(e); + break; + case "arrIntrinsic": { + if (!ARR_READS.has(e.method)) write(e.receiver, e.loc, `'${e.method}'`); + // Copies hold the receiver's elements, and the arguments they add. + const copy = + ARR_COPIES.has(e.method) || e.method === "splice" || e.method === "spliceInsert"; + const into = copy ? this.site(fn, e) : e.receiver; + if (copy) spread(e.receiver, into); + for (const a of e.args) + if (SPREADS.has(e.method) && typeKey(a.type) === typeKey(e.receiver.type)) + spread(a, into); + else store(into, a); + break; + } + case "mapIntrinsic": + case "setIntrinsic": + if (e.method === "set" || e.method === "add") + for (const a of e.args) store(e.receiver, a); + else if (e.method === "clone" || e.method === "keySet" || e.method === "valueSet") + spread(e.receiver, this.site(fn, e)); + break; + case "arrayLit": + e.elems.forEach((x, i) => { + if (e.spreads?.includes(i)) spread(x, this.site(fn, e)); + else store(this.site(fn, e), x); + }); + break; + case "setNew": + if (e.seed) spread(e.seed, this.site(fn, e)); + break; + case "mapNew": + for (const { key, value } of e.seed ?? []) { + store(this.site(fn, e), key); + store(this.site(fn, e), value); + } + break; + case "recordLit": + if (e.type.kind === "record") { + const shape = e.type.shapeId; + for (const f of e.fields) hold(`s:${shape}\0${f.name}`, f.value); + } + break; + case "closure": { + const lifted = this.functions.get(e.fnName); + // A captured local is one binding on both sides. + if (check === null) + e.captures.forEach((id, i) => { + const inner = lifted?.captures?.[i]; + if (inner === undefined) return; + const outer = this.localSlot(fn, id); + const captured = `l:${lifted!.name}\0${inner.localId}`; + this.edge(outer, captured); + this.edge(captured, outer); + }); + break; + } + case "dynFrom": + case "caughtToDyn": + if (!readSinks.has(e)) escape(e.value, e.loc, "a dynamic value"); + break; + case "callValue": + for (const a of e.args) escape(a, e.loc, "a call through a function value", false); + break; + case "intrinsic": + // Printing reads its arguments; publishing freezes them. + if (e.name.startsWith("console.") || e.name === "threads.publish") { + for (const a of e.args) if (a.kind === "dynFrom") readSinks.add(a); + } else for (const a of e.args) escape(a, e.loc, `'${e.name}'`); + break; + case "libCall": + if (readsArguments(e)) { + for (const a of e.args) if (a.kind === "dynFrom") readSinks.add(a); + } else for (const a of e.args) escape(a, e.loc, `the library call ${e.fn}`); + break; + default: + // Any other parent takes its child somewhere this pass does not + // follow. + if (!MODELLED.has(e.kind)) + everyExprChild( + e, + (child) => { + escape(child, e.loc, `a '${e.kind}' expression`); + return true; + }, + () => true, + ); + } + return everyExprChild(e, expr, stmt); + }; + const stmt = (s: IrStmt): boolean => { + switch (s.kind) { + case "varDecl": + if (s.init) hold(this.localSlot(fn, s.localId), s.init); + break; + case "assign": + hold(this.localSlot(fn, s.localId), s.value); + break; + case "return": + if (s.value === null) break; + // A promise or iterator holds the value. + if (fn.async || fn.generator) + escape(s.value, s.loc, "the result of an async function or generator"); + // A parameter returned as it came is the caller's argument. The + // result of a function value may be any such function's. + else if (check === null && this.followed(s.value.type)) + for (const from of this.slotsOf(fn, s.value)) { + if (carriers?.has(from)) continue; + this.edge(from, `r:${fn.name}`); + if (this.escapedFunctions.has(fn.name)) this.edge(from, "r:*"); + } + break; + case "fieldSet": + hold(this.fieldSlot(s.className, s.field), s.value); + break; + case "recordSet": + hold(`s:${s.shapeId}\0${s.field}`, s.value); + break; + case "arraySet": + write(s.arr, s.loc, "an element assignment"); + store(s.arr, s.value); + break; + case "arraySetLength": + write(s.arr, s.loc, "a 'length' assignment"); + break; + case "arraySetUndefined": + case "arrayDelete": + write(s.arr, s.loc, "an element assignment"); + break; + case "forOf": + // The binding takes the elements of an array, Map or Set. + if (check === null) + for (const from of this.slotsOf(fn, s.iterable)) + this.edge(`c:${from}`, this.localSlot(fn, s.localId)); + break; + case "throw": + escape(s.value, s.loc, "a throw"); + break; + case "recordKeySet": + escape(s.value, s.loc, "a dynamically keyed property"); + break; + } + return everyStmtChild(s, expr, stmt); + }; + for (const s of fn.body) stmt(s); + } + + // ── Checks ── + + /** Every inline `new` passes an array nothing else keeps (or a field's). */ + private checkSources(): void { + const shared = new Map(); + this.shareSink = shared; + this.checkConstructions(); + this.shareSink = null; + // The shared arrays are followed from their definitions. + const queue: string[] = []; + for (const [slot, loc] of shared) + if (!this.pinned.has(slot)) { + this.pinned.set(slot, loc); + queue.push(slot); + } + this.propagate(queue); + } + + private checkConstructions(): void { + for (const fn of this.mod.functions) + this.eachExpr(fn, (e) => { + const inline = this.inlineNew(e); + if (inline !== null && e.kind === "new") { + const arg = e.args[inline.arg]; + if (arg !== undefined && !this.fresh(fn, arg)) + this.report( + { + loc: arg.loc, + reason: + "the array passed to the constructor may be kept elsewhere (pass a new array, a copy, or a local array's last use)", + }, + this.ownerOf(e.className), + ); + } + if (inline !== null || (e.kind !== "call" && e.kind !== "new" && e.kind !== "virtualCall")) + return; + e.args.forEach((arg, i) => { + if (!this.consumes(e, i) || this.fresh(fn, arg)) return; + this.report({ + loc: arg.loc, + reason: + "the array passed here moves into an InlineArray field but may be kept elsewhere (pass a new array, a copy, or a local array's last use)", + }); + }); + }); + } + + private checkWrites(): void { + for (const fn of this.mod.functions) this.flow(fn, (issue) => this.report(issue)); + } + + private ownerOf(className: string): { declaringClass: string; field: string } { + const cls = this.plan.classes.get(className)!; + return { declaringClass: cls.declaringClass, field: cls.field }; + } +} diff --git a/packages/compiler/src/ir/inline-arrays.ts b/packages/compiler/src/ir/inline-arrays.ts index 4cb929e7a..a3fc37a81 100644 --- a/packages/compiler/src/ir/inline-arrays.ts +++ b/packages/compiler/src/ir/inline-arrays.ts @@ -14,8 +14,9 @@ * * `inlineArrayReads` then prepares the IR for the layout: * - a `new` of such a class evaluates its arguments into locals first, in - * order, when the length argument is not already a local, so that the - * backend can read the length before it allocates; + * order, unless every argument is a local or a constant, so that the + * backend reads the length after every argument has run and before it + * allocates; * - an immutable local initialized from the field (`const nodes = * list.nodes`) whose every use indexes it, reads its length or iterates * it with `for...of` is replaced by reads of the field, through a local @@ -220,9 +221,26 @@ function parameterOf(fn: IrFunction, value: IrExpr): number | string { const local = fn.locals.find((l) => l.id === value.localId); if (local === undefined || local.boxed === true || assigns(fn, value.localId)) return "the constructor parameter is written"; + // The allocation took its length from the argument, so nothing in the + // constructor may change the array first: the field's value is the + // parameter's only use. + if (reads(fn, value.localId) !== 1) + return "the constructor uses the array parameter besides storing it in the field"; return index; } +function reads(fn: IrFunction, localId: string): number { + let n = 0; + everyStmtList(fn.body, { + expr: (e) => { + if (e.kind === "varRef" && e.localId === localId) n++; + return true; + }, + stmt: () => true, + }); + return n; +} + function assigns(fn: IrFunction, localId: string): boolean { return !everyStmtList(fn.body, { expr: () => true, @@ -435,10 +453,15 @@ function rewriteFunction(fn: IrFunction, plan: InlineArrayPlan): IrFunction { /** A `new` of a planned class whose length argument is not a local read. */ function newNeedsLocals(e: IrExpr & { kind: "new" }, plan: InlineArrayPlan): boolean { - const cls = plan.classes.get(e.className); - if (cls === undefined) return false; - const arg = e.args[cls.lengthArg]; - return arg !== undefined && arg.kind !== "varRef"; + if (!plan.classes.has(e.className)) return false; + // The allocation reads the length once every argument has run: unless + // they are all locals and constants, which run no code, they are + // evaluated into locals first. + return !e.args.every((a) => a.kind === "varRef" || isConstant(a)); +} + +function isConstant(e: IrExpr): boolean { + return e.kind === "numLit" || e.kind === "strLit" || e.kind === "boolLit" || e.kind === "unitLit"; } /** `new C(a, b, ...)` as `(t0 = a, t1 = b, ..., new C(t0, t1, ...))`: the diff --git a/packages/compiler/test/inline-array-fields.test.ts b/packages/compiler/test/inline-array-fields.test.ts index f948bb5fe..aa4240fe1 100644 --- a/packages/compiler/test/inline-array-fields.test.ts +++ b/packages/compiler/test/inline-array-fields.test.ts @@ -1,4 +1,5 @@ -import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; import { join } from "node:path"; import { expect, test } from "vitest"; import { compile } from "../src/index.js"; @@ -7,6 +8,8 @@ import { compile } from "../src/index.js"; // elements of a ProgramLifetime class's array field written once from a // constructor parameter live in the instance, after its fixed fields. +const repoRoot = join(import.meta.dirname, "../../.."); + const header = ` import type { ProgramLifetime } from "@scriptc/types"; import type { InlineArray } from "@scriptc/experimental"; @@ -15,9 +18,10 @@ import type { InlineArray } from "@scriptc/experimental"; async function build( source: string, ): Promise<{ ok: boolean; codes: string[]; llvm: string; messages: string[] }> { - // Inside the workspace, so that the program resolves both packages. - const dir = await mkdtemp(join(import.meta.dirname, ".inline-array-")); + const dir = await mkdtemp(join(tmpdir(), "scriptc-inline-array-")); try { + // The workspace's packages, so that the program resolves both imports. + await symlink(join(repoRoot, "node_modules"), join(dir, "node_modules")); const entry = join(dir, "main.ts"); const outPath = join(dir, "main.ll"); await writeFile(entry, header + source); @@ -159,6 +163,235 @@ console.log(use(new List(0, [new Item(1)]))); expect(body("use", llvm)).toContain("call ptr @scr_arr_view_cref("); }); +test("writes to the arrays fields hold are refused", async () => { + for (const write of [ + // Through a read of the field. + `function grow(list: List): void { + (list.items as Item[]).push(new Item(5)); +} +grow(new List(0, [new Item(1)]));`, + // Through a parameter and an array element. + `function clear(items: readonly Item[]): void { + (items as Item[]).length = 0; +} +const all = [new List(0, [new Item(1)]).items]; +clear(all[0]!);`, + // Through a Map value. + `const byPos = new Map(); +const list = new List(0, [new Item(1)]); +byPos.set(list.pos, list.items); +(byPos.get(0)! as Item[])[0] = new Item(2);`, + ]) { + const result = await build(classes + write); + expect(result.ok).toBe(false); + expect(result.codes).toContain("SC7009"); + expect(result.messages.join("\n")).toContain( + "writes to an array that may be an InlineArray field's", + ); + } +}); + +test("a write names the field read whose array it may reach", async () => { + const result = await build( + classes + + ` +function same(items: readonly Item[]): readonly Item[] { + return items; +} +const list = new List(0, [new Item(1)]); +const all: (readonly Item[])[] = [same(list.items)]; +(all[0]! as Item[]).pop(); +`, + ); + expect(result.ok).toBe(false); + expect(result.messages.join("\n")).toMatch( + /writes to an array that may be an InlineArray field's \(from main\.ts:\d+:\d+\)/, + ); +}); + +test("other arrays of the element type stay writable", async () => { + // Arrays are followed by where they are held, not by type: a helper that + // returns its argument gives each caller its own array back. + const result = await build( + classes + + ` +function same(items: readonly Item[]): readonly Item[] { + return items; +} +const list = new List(0, [new Item(1)]); +const views: (readonly Item[])[] = [same(list.items)]; +const scratch: Item[][] = [[]]; +const own = same([new Item(2)]) as Item[]; +scratch[0]!.push(new Item(3)); +own.push(new Item(4)); +console.log(views[0]!.length, scratch[0]!.length, own.length); +`, + ); + expect(result.messages).toEqual([]); +}); + +test("a read of the field may reach printing and copies, but no other library call", async () => { + const reads = await build( + classes + + ` +const list = new List(0, [new Item(1)]); +console.log(list.items, JSON.stringify(list.items), [...list.items].length); +console.log(new List(1, list.items.slice()).items.length); +`, + ); + expect(reads.ok).toBe(true); + const frozen = await build( + classes + + ` +console.log(Object.isFrozen(new List(0, [new Item(1)]).items)); +`, + ); + expect(frozen.ok).toBe(false); + expect(frozen.messages.join("\n")).toContain("reaches the library call"); +}); + +test("the constructor is passed an array nothing else keeps", async () => { + const accepted = [ + // A literal, a copy, and a factory's result. + `function fresh(n: number): Item[] { + const out: Item[] = []; + for (let i = 0; i < n; i++) out.push(new Item(i)); + return out; +} +const a = new List(0, [new Item(1)]); +const b = new List(1, a.items.concat([new Item(2)])); +console.log(a.items.length, b.items.length, new List(2, fresh(3)).items.length);`, + // A local's last use, after writes in place, in either arm of a + // conditional, and after an early return. + `function make(c: boolean): List { + const items = [new Item(1)]; + items.push(new Item(2)); + if (items.length > 5) return new List(-1, items); + return c ? new List(0, items) : new List(1, items); +} +console.log(make(true).items.length, make(false).pos);`, + // A parameter its callers move. + `function wrap(pos: number, items: Item[]): List { + items.reverse(); + return new List(pos, items); +} +function build(n: number): List { + return wrap(n, Array.from({ length: n }, (_, i) => new Item(i))); +} +console.log(build(2).items[0]!.kind);`, + // An array a field already holds. + `const a = new List(0, [new Item(1)]); +const shared = a.items; +console.log(new List(1, shared).items.length, new List(2, shared).pos);`, + // An array that is never written, shared by several constructors. + `const none: Item[] = []; +function make(items: Item[]): List { + return new List(0, items.length === 0 ? none : items); +} +const lists: List[] = []; +for (let i = 0; i < 2; i++) lists.push(new List(i, none)); +console.log(lists.length, make([]).items.length, make([new Item(1)]).items.length);`, + // A local that each pass of a loop moves and then replaces. + `function pairs(n: number): List[] { + const lists: List[] = []; + let items: Item[] = []; + for (let i = 0; i < n; i++) { + items.push(new Item(i)); + if (i % 2 === 1) { + lists.push(new List(i, items)); + items = []; + } + } + return lists; +} +console.log(pairs(4).length);`, + // A top-level array, read elsewhere. + `const items = [new Item(1), new Item(2)]; +function count(): number { + return items.length; +} +console.log(count()); +const list = new List(0, items); +console.log(list.items.length);`, + ]; + for (const source of accepted) { + const result = await build(classes + source); + expect(result.messages).toEqual([]); + } + const refused = [ + // Written after the move. + `const items = [new Item(1)]; +const list = new List(0, items); +items.push(new Item(2)); +console.log(list.items.length);`, + // Kept in another variable. + `const items = [new Item(1)]; +const kept = items; +const list = new List(0, items); +kept.push(new Item(2)); +console.log(list.items.length);`, + // A parameter whose caller keeps the array. + `function wrap(items: Item[]): List { + return new List(0, items); +} +const items = [new Item(1)]; +const list = wrap(items); +items.push(new Item(2)); +console.log(list.items.length);`, + // A parameter of another class's constructor, and of a method. + `class Section { + readonly list: List; + constructor(items: Item[]) { + this.list = new List(0, items); + } +} +const items = [new Item(1)]; +const section = new Section(items); +items.push(new Item(2)); +console.log(section.list.items.length);`, + `class Maker { + make(items: Item[]): List { + return new List(0, items); + } +} +class FlagMaker extends Maker { + override make(items: Item[]): List { + return new Flagged(0, items, 1); + } +} +function run(maker: Maker): number { + const items = [new Item(1)]; + const list = maker.make(items); + items.push(new Item(2)); + return list.items.length; +} +console.log(run(new Maker()), run(new FlagMaker()));`, + // A shared array written elsewhere. + `const none: Item[] = []; +const a = new List(0, none); +const b = new List(1, none); +function add(items: Item[]): void { + items.push(new Item(2)); +} +add(none); +console.log(a.items.length, b.items.length);`, + // A function returning a shared array. + `const all: Item[] = [new Item(1)]; +function current(): Item[] { + return all; +} +const list = new List(0, current()); +all.push(new Item(2)); +console.log(list.items.length);`, + ]; + for (const source of refused) { + const result = await build(classes + source); + expect(result.ok).toBe(false); + expect(result.codes).toContain("SC7009"); + expect(result.messages.join("\n")).toContain("may be kept elsewhere"); + } +}); + test("an InlineArray field must keep its contract", async () => { const twice = await build(` class Item implements ProgramLifetime {} @@ -205,6 +438,50 @@ console.log(new List(2).items.length); expect(computed.messages.join("\n")).toContain("constructor parameter"); }); +test.runIf(lowRegion)( + "an instance inside a dynamic value shows the field as an array", + async () => { + const { ok, llvm } = await build( + classes + + ` +class Holder { + readonly list: List; + constructor(list: List) { + this.list = list; + } +} +const list = new List(3, [new Item(1), new Item(2)]); +console.log(JSON.stringify({ list }), JSON.stringify(new Holder(list))); +`, + ); + expect(ok).toBe(true); + // The instance's dynamic view reads the field as an array of the elements. + const views = llvm + .split(/\n(?=define )/) + .filter((f) => f.startsWith("define") && f.includes("_materialize(")); + expect(views.some((f) => f.includes("call ptr @scr_arr_view_cref("))).toBe(true); + }, +); + +test("the constructor stores the array parameter and does nothing else with it", async () => { + const result = await build(` +class Item implements ProgramLifetime {} +class List implements ProgramLifetime { + readonly items: InlineArray; + readonly count: number; + constructor(items: Item[]) { + items.push(new Item()); + this.items = items; + this.count = items.length; + } +} +console.log(new List([new Item()]).count); +`); + expect(result.ok).toBe(false); + expect(result.codes).toContain("SC7009"); + expect(result.messages.join("\n")).toContain("besides storing it in the field"); +}); + test("a read of an InlineArray field is not compared by identity", async () => { const result = await build(` class Item implements ProgramLifetime {} diff --git a/packages/experimental/README.md b/packages/experimental/README.md index 6c2e0649b..99261b6c9 100644 --- a/packages/experimental/README.md +++ b/packages/experimental/README.md @@ -143,7 +143,9 @@ function visit(list: NodeList): void { - `T` is a `ProgramLifetime` class, and the field is an instance field of a `ProgramLifetime` class. - The constructor of the class that declares the field writes it once, from one of its parameters. A subclass constructor passes one of its own parameters there through `super(...)`. Nothing else writes the field. - Indexing the field, reading its `length` and iterating it with `for...of` read the elements from the instance, directly or through a `const` local used only in those ways. Any other use, such as passing the field to a function or calling one of its methods, gets an array of the elements, which is frozen in scriptc builds. +- The array passed to the constructor is one that nothing else keeps: a new array (a literal, a copy, or the result of a function that returns only such arrays or its own argument), a local array at its last use, a parameter whose callers each pass such an array, an array that an InlineArray field already holds, or a `const` array that is never written. +- The arrays fields hold are never written, through a read of the field (a cast) or through any other reference. scriptc follows them through locals, parameters, return values, fields, the elements of arrays, Maps and Sets, and closures, and reports one that reaches a write, a dynamic value, a call through a function value, or a library call other than printing and `JSON.stringify` without a replacer. - Two reads of the field need not give the same array, so a read of the field is never compared with `===` or `!==`. Compare a read kept in a local instead. -- Instances of the class are not constructed through a class value, and do not become dynamic values (for example through `console.log` or `JSON.stringify`). +- Instances of the class are not constructed through a class value, and do not become dynamic values (for example through `JSON.stringify` or a conversion to `unknown`). scriptc reports a field that does not keep these rules (SC7009). Under Node.js the field holds the array the constructor stored. In builds with sanitizers, in programs that use `LifetimeRegion`, and on platforms other than 64-bit Linux, the elements stay in an ordinary array object. diff --git a/packages/experimental/index.d.ts b/packages/experimental/index.d.ts index 1de6f8c92..161e9437d 100644 --- a/packages/experimental/index.d.ts +++ b/packages/experimental/index.d.ts @@ -79,7 +79,8 @@ export declare class ThreadSlots { * (@scriptc/types ProgramLifetime classes): `readonly f: InlineArray`, * where C is a ProgramLifetime class. The class's constructor writes the * field once, from one of its parameters, and nothing else writes it. - * Reading the field gives an array of its elements, frozen in scriptc - * builds; two reads need not give the same array. Under Node.js it is the - * array the constructor stored. */ + * The constructor is passed an array nothing else keeps, and the arrays + * fields hold are never written. Reading the field gives an array of its + * elements, frozen in scriptc builds; two reads need not give the same + * array. Under Node.js it is the array the constructor stored. */ export type InlineArray = readonly T[]; diff --git a/tests/corpus/contract-inline-array-fields.ts b/tests/corpus/contract-inline-array-fields.ts index ac86fa60b..0baa32d24 100644 --- a/tests/corpus/contract-inline-array-fields.ts +++ b/tests/corpus/contract-inline-array-fields.ts @@ -137,6 +137,23 @@ const copy = new List(0, 0, big.nodes); const part = new ModifierList(0, 0, big.nodes.slice(10, 13), 1, "part"); console.log(copy.nodes.length, copy.nodes[39] === big.nodes[39], sum(part), part.nodes[0]!.pos); +// Inside a value that becomes dynamic, the field shows as an array. +class Holder { + readonly list: List; + constructor(list: List) { + this.list = list; + } +} +const shown = new ModifierList(1, 2, leaves(7, 2), 3, "shown"); +console.log(JSON.stringify({ shown })); +console.log(JSON.stringify(new Holder(shown))); + +// The arguments of `new` run before the instance is made, so an argument +// that adds to the array the list is built from adds to the list. +const growing = leaves(50, 2); +const grown = new List(growing.push(new Leaf(52, 1)), 0, growing); +console.log(grown.pos, grown.nodes.length, grown.nodes[2]!.pos); + // Trees whose groups hold lists. function tree(level: number, pos: number): Node { if (level === 0) return new Leaf(pos, pos);