diff --git a/docs/content/docs/contracts.mdx b/docs/content/docs/contracts.mdx index ad5953fa6..7815ccc40 100644 --- a/docs/content/docs/contracts.mdx +++ b/docs/content/docs/contracts.mdx @@ -366,6 +366,10 @@ Static code has no `WeakRef`, a `FinalizationRegistry` never runs its cleanup ca A long-running program, such as a compiler in watch mode, can free program-lifetime instances one generation at a time with `LifetimeRegion` from [`@scriptc/experimental`](/docs/experimental). Instances created while `region.run(fn)` runs belong to the region. After `region.release()`, scriptc frees them together once nothing outside the region refers to any of them, and removes the `WeakMap` and `WeakSet` entries they are keys of. Instances created outside every region are never freed, as described above. +### Inline array fields + +An array field of a program-lifetime class whose elements are program-lifetime instances can keep its elements in the instance itself, after the other fields, with `InlineArray` from [`@scriptc/experimental`](/docs/experimental). The constructor writes the field once from one of its parameters; reads that index the field, read its `length` or iterate it load the elements from the instance, and other reads get an array of them, which is frozen in scriptc builds. scriptc reports a field that does not keep these rules (SC7009). + ## Diagnostics Each failed contract names the declaration and the reason: @@ -468,5 +472,9 @@ $ scriptc build reject.ts SC7008 A ProgramLifetime class is not the root of its hierarchy, or is part of a library build. + + SC7009 + An InlineArray field is not on a ProgramLifetime class, is written other than once from a constructor parameter, or is read in a way that could tell its reads apart. + diff --git a/docs/content/docs/experimental.mdx b/docs/content/docs/experimental.mdx index 97e048fc4..77ea52982 100644 --- a/docs/content/docs/experimental.mdx +++ b/docs/content/docs/experimental.mdx @@ -334,3 +334,71 @@ scriptc requires: Each instance of a key class, and of every class in its hierarchy, carries a 4-byte slot index, 0 until a value is set for it. The first `set` for a key assigns the index; indices are never reused. `get` and `has` read the index, a page of the thread's family and the entry, with no lock, hash or atomic update. A family stores values in pages of 1024 entries, allocated as the thread sets values in them. A family holds a reference to each of the thread's values. A value is released when it is replaced or deleted, when its key is freed, and when the thread exits. Keys that are freed with a [`LifetimeRegion`](#lifetimeregion) keep their values until the thread exits, a value that belongs to a region keeps the region allocated while the family holds it, and a value that refers to its own key keeps the key alive. Programs that never construct a `ThreadSlots` do not run any of its code, and their classes have no slot index. + +## InlineArray + +`InlineArray` marks an array field of a [`ProgramLifetime`](/docs/contracts#programlifetime) class whose elements scriptc keeps in the instance itself, after its other fields, instead of in an array object of their own. It suits a program with millions of short arrays that never change once built, such as the children of each node of a syntax tree: an ordinary array costs a separate allocation with a header, and a reference to it in the instance. + +```ts:lists.ts +import type { ProgramLifetime } from "@scriptc/types"; +import type { InlineArray } from "@scriptc/experimental"; + +class Token implements ProgramLifetime { + readonly text: string; + constructor(text: string) { + this.text = text; + } +} + +class TokenList implements ProgramLifetime { + readonly pos: number; + readonly tokens: InlineArray; + constructor(pos: number, tokens: readonly Token[]) { + this.pos = pos; + this.tokens = tokens; + } +} + +function width(list: TokenList): number { + let total = 0; + for (const token of list.tokens) total += token.text.length; + return total; +} + +const list = new TokenList(0, ["let", "x", "=", "1"].map((text) => new Token(text))); +console.log(list.tokens.length, width(list), list.tokens[0]!.text); +console.log(list.tokens.map((token) => token.text).join(" ")); +``` + +Both runtimes print the same: + +```console +$ node lists.ts +4 6 let +let x = 1 +$ scriptc build lists.ts -o lists >/dev/null +$ ./lists +4 6 let +let x = 1 +``` + +### Field rules + +scriptc requires: + +- `T` to be a `ProgramLifetime` class, and the field to be an instance field of a `ProgramLifetime` class; +- the constructor of the class that declares the field to write it once, from one of its parameters. A subclass constructor passes one of its own parameters there through `super(...)`. Nothing else writes the field; +- reads of the field not to be compared with `===` or `!==`: two reads need not give the same array. Compare a read kept in a local instead; +- instances of the class not to be constructed through a class value, and not to become dynamic values (for example through `console.log` or `JSON.stringify`). + +scriptc reports a field that does not keep these rules (SC7009). + +### Reads + +Indexing the field, reading its `length` and iterating it with `for...of` read the elements from the instance, directly or through a `const` local used only in those ways. Any other use, such as passing the field to a function or calling one of its methods, gets an array of the elements, which is frozen in scriptc builds. Under Node.js the field holds the array the constructor stored. + +### Inline array cost + +Each element takes 4 bytes in the instance, and the field's own slot holds the count. The constructor copies the elements from the array it is passed, so that array can be released. The elements start at the same offset in every class of the hierarchy, after the largest set of fixed fields. A read that gets an array allocates a small header that refers to the instance's elements, and frees it when the read's value is released. + +The elements stay in an ordinary array object, and the field behaves like any other array field, in builds with sanitizers, in programs that use [`LifetimeRegion`](#lifetimeregion), and on platforms other than 64-bit Linux. diff --git a/docs/test/fixtures/migration-baseline.json b/docs/test/fixtures/migration-baseline.json index bd9749038..ea0ce3dc4 100644 --- a/docs/test/fixtures/migration-baseline.json +++ b/docs/test/fixtures/migration-baseline.json @@ -316,6 +316,22 @@ { "text": "Slot cost", "id": "slot-cost" + }, + { + "text": "InlineArray", + "id": "inlinearray" + }, + { + "text": "Field rules", + "id": "field-rules" + }, + { + "text": "Reads", + "id": "reads" + }, + { + "text": "Inline array cost", + "id": "inline-array-cost" } ] }, diff --git a/packages/compiler/src/backend/llvm/classes.ts b/packages/compiler/src/backend/llvm/classes.ts index 695acc273..d25233dc8 100644 --- a/packages/compiler/src/backend/llvm/classes.ts +++ b/packages/compiler/src/backend/llvm/classes.ts @@ -43,6 +43,7 @@ import { llvmCommentText } from "./common.js"; import { INLINE_UNION_FIELD } from "./inline-unions.js"; import { NullableRefFields } from "./nullable-fields.js"; import type { Int32Slots } from "../../ir/int32-slots.js"; +import type { InlineArrayClass } from "../../ir/inline-arrays.js"; import { cycleRootLines, traceFieldVisitLines, @@ -115,12 +116,30 @@ export class LlClassMeta { * a compact class instance as a 32-bit index (COMPACT_REF) rather than a * pointer: compactRefLoadLines decodes it, compactRefStoreLines encodes it. */ compressed: boolean[] = []; + /** Compact classes with an inline array field (ir/inline-arrays.ts): the + * field's slot holds the element count as an i32, and the elements follow + * the fixed fields as 32-bit references from `tailOffset`, the same offset + * in every class of the declaring class's subtree. Absent otherwise. */ + inlineTail?: LlInlineTail; constructor(readonly def: IrClassDef) { this.root = this; } } +/** A class's inline array field (LlClassMeta.inlineTail). */ +export interface LlInlineTail { + /** The field's index in def.fields. */ + readonly field: number; + /** Byte offset of the first element. */ + readonly tailOffset: number; + /** The elements' class. */ + readonly elemClass: string; + /** The index in a `new` expression's args of the array the field is + * initialized from, whose length the allocation reads. */ + readonly lengthArg: number; +} + /** The class graph: link base/children, number the forest in preorder * (roots and children in module class order, matching the * runtime's stamped error vtables), and compute each hierarchy's virtual @@ -767,6 +786,7 @@ export function assignCompactLayouts( field: number, isCompact: (className: string) => boolean, ) => boolean, + inlineArrays?: ReadonlyMap, ): void { const isCompact = (className: string): boolean => { const target = metaMap.get(className); @@ -798,7 +818,18 @@ export function assignCompactLayouts( ? base!.compressed[i]! : compressedRef !== undefined && compressedRef(meta.def, i, isCompact), ); + // An inline array field (ir/inline-arrays.ts) whose elements are compact + // keeps its element count in its slot. + const inline = inlineArrays?.get(meta.def.name); + const inlineField = + inline !== undefined && isCompact(inline.elemClass) && isCompact(inline.declaringClass) + ? inline.fieldIndex + : -1; const types = fieldTypes(meta.def).map((t, i) => { + if (i === inlineField) { + compressed[i] = false; + return "i32"; + } if (!compressed[i]) return t; if (t !== "ptr") throw new InternalCompilerError( @@ -846,6 +877,44 @@ export function assignCompactLayouts( meta.compactSlots = slots; meta.compressed = compressed; } + if (inlineArrays !== undefined) assignInlineTails(metaMap, inlineArrays); +} + +/** Gives each compact class of an inline array field's subtree its tail + * (LlClassMeta.inlineTail): the elements start after the largest fixed part + * of any class in the subtree, so that a read through any static type of + * the subtree finds them at the same offset. */ +function assignInlineTails( + metaMap: ReadonlyMap, + inlineArrays: ReadonlyMap, +): void { + const subtrees = new Map(); + for (const [name, plan] of inlineArrays) { + const meta = metaMap.get(name); + if (meta === undefined || !meta.compact) continue; + // The layout gave the field its i32 count only when the elements are + // compact (assignCompactLayouts). + const slot = meta.compactSlots[plan.fieldIndex]; + if (slot === undefined || meta.compactElems[slot]?.type !== "i32") continue; + const members = subtrees.get(plan.declaringClass); + if (members === undefined) subtrees.set(plan.declaringClass, [meta]); + else members.push(meta); + } + for (const members of subtrees.values()) { + let end = 0; + for (const meta of members) + for (const e of meta.compactElems) end = Math.max(end, e.offset + e.size); + const tailOffset = alignTo(end, 4); + for (const meta of members) { + const plan = inlineArrays.get(meta.def.name)!; + meta.inlineTail = { + field: plan.fieldIndex, + tailOffset, + elemClass: plan.elemClass, + lengthArg: plan.lengthArg, + }; + } + } } function alignTo(n: number, align: number): number { @@ -1390,8 +1459,12 @@ export function emitClassShapes( // members, undefined-admitting union fields at the interned unit // instance. Traced shapes allocate with the collector header; alloc.ts // inlines the allocator fast paths and the alloc note. + // An inline array field's instance (LlClassMeta.inlineTail) takes the + // field's element count and has room for the elements after its fixed + // fields. + const tail = meta.compact && regionRefs ? (meta.inlineTail ?? null) : null; const nw: string[] = [ - `define internal ptr @${mangleClassNew(cls.name)}() ${FN_ATTRS} { ; new ${cls.name}`, + `define internal ptr @${mangleClassNew(cls.name)}(${tail !== null ? "i32 %inl.len" : ""}) ${FN_ATTRS} { ; new ${cls.name}`, `entry:`, ]; const cycType = `@${mangleClassTrace(cls.name)}.cyctype`; @@ -1419,7 +1492,14 @@ export function emitClassShapes( // Compressed fields refer to compact instances by their index in the // program-lifetime region, so every compact instance lives there: // zeroed, 8-byte aligned and never freed. - nw.push(...plrAllocLines(host, sizeOf)); + if (tail !== null) { + nw.push( + ` %inl.n = zext i32 %inl.len to ${host.sizeType}`, + ` %inl.bytes = shl nuw nsw ${host.sizeType} %inl.n, 2`, + ` %inl.size = add nuw nsw ${host.sizeType} %inl.bytes, ${tail.tailOffset}`, + ); + nw.push(...plrAllocLines(host, "%inl.size")); + } else nw.push(...plrAllocLines(host, sizeOf)); } else nw.push( ...emitObjectAlloc( @@ -1496,6 +1576,11 @@ export function emitClassShapes( ); } nw.push(...undefFieldInits(host, meta, nullable)); + if (tail !== null) + nw.push( + ` %inl.countp = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${meta.compactSlots[tail.field]!}`, + ` store i32 %inl.len, ptr %inl.countp ; ${llvmCommentText(cls.name)} inline array count`, + ); // Young instances: program-owned traced classes. Classes rooted at a // runtime base (errors, emitters, events, streams) carry runtime state // and stay with trial deletion, like every runtime family. @@ -1646,6 +1731,10 @@ export function emitClassObjDefs( throw new InternalCompilerError( `llvm emitter bug: class object for ${className} without a constructor`, ); + if (meta.inlineTail !== undefined) + throw new InternalCompilerError( + `llvm emitter bug: class object for ${className}, which has an inline array field`, + ); const params = ctor.params.slice(valueCtor.firstArg); const paramDecls = params.map((p, i) => `${llType(p.type)} %a${i}`).join(", "); const ctorArgs = params.map((p, i) => `${llType(p.type)} %a${i}`); diff --git a/packages/compiler/src/backend/llvm/emit-statements.ts b/packages/compiler/src/backend/llvm/emit-statements.ts index 1eeceff4a..805a49973 100644 --- a/packages/compiler/src/backend/llvm/emit-statements.ts +++ b/packages/compiler/src/backend/llvm/emit-statements.ts @@ -1,4 +1,5 @@ /* LlEmitter's statement emission: the statement switch, try/catch, switch, loop copies, speculative writes and string self-concatenation. */ +import { emitInlineArrayInit, inlineFieldTail } from "./inline-arrays.js"; import { emitLoopBytesHeaders } from "./bytes-headers.js"; import { noteBytesReceiver } from "./expr-bytes.js"; import { @@ -609,6 +610,17 @@ export function emitStmtBody(emitter: LlEmitter, s: IrStmt): void { // release — a release can trigger a cycle collection, which must // never see a heap edge whose count was already given up). // Classes and records share the struct layout, so one emission. + // The constructor's write of an inline array field copies the array's + // elements into the instance (backend/llvm/inline-arrays.ts). + const tail = + s.kind === "fieldSet" ? inlineFieldTail(emitter, s.className, s.field) : undefined; + if (tail !== undefined && s.kind === "fieldSet") { + const obj = emitter.emitStableReceiver(s.obj, [s.value]); + const v = emitter.emitExpr(s.value); + emitter.emitStoreGuard(obj.name, s); + emitInlineArrayInit(emitter, obj.name, s.className, tail, v.name); + break; + } const nullable = s.kind === "fieldSet" ? emitter.nullableFields.get(s.className, s.field) : null; if (nullable && s.kind === "fieldSet") { diff --git a/packages/compiler/src/backend/llvm/emitter.ts b/packages/compiler/src/backend/llvm/emitter.ts index 14e2e9d6c..3e9f663de 100644 --- a/packages/compiler/src/backend/llvm/emitter.ts +++ b/packages/compiler/src/backend/llvm/emitter.ts @@ -85,6 +85,13 @@ import { import { specializeNumericCalls } from "../../ir/numeric-call-specialization.js"; import { versionLoopBoundParameters } from "../../ir/parameter-versions.js"; import { inlineExpressionFunctions } from "../../ir/expression-inlining.js"; +import { + inlineArrayReads, + NO_INLINE_ARRAYS, + planInlineArrays, + type InlineArrayPlan, +} from "../../ir/inline-arrays.js"; +import { inlineArraysApply, programLifetimeClassOf } from "./inline-arrays.js"; import { pairRowSums } from "../../ir/row-pairs.js"; import { allocatedLiteral, @@ -341,6 +348,7 @@ import { classStructSym, programLifetimeClass, type LlClassMeta, + type LlInlineTail, } from "./classes.js"; import { LifetimeRegions, REGION_TYPES } from "./regions.js"; import { @@ -661,6 +669,13 @@ function resultEntryTargets(mod: IrModule): ReadonlySet { return targets; } +/** The reads and constructions of inline array fields prepared for their + * layout (ir/inline-arrays.ts), where the target keeps them inline. */ +function inlineArraysPass(options: LlvmTargetOptions, mod: IrModule): IrModule { + if (!inlineArraysApply(mod, options)) return mod; + return inlineArrayReads(mod, planInlineArrays(mod, programLifetimeClassOf(mod))); +} + /** The emitter for `input` after the module passes every emission runs. */ function moduleEmitter(input: IrModule, options: LlvmTargetOptions): LlEmitter { // Facts are keyed by node identity: emit the same module whether it was @@ -705,14 +720,17 @@ function moduleEmitter(input: IrModule, options: LlvmTargetOptions): LlEmitter { specializeCallbackCalls( specializeNumericCalls( pairRowSums( - inlineExpressionFunctions( - foldConstantGlobals( - copyStableCaptures( - resolveSelfCaptures( - hoistFinalFieldReads( - freezeInitOnlyGlobals( - holdCharLocalsAsUnits( - dropUnwrittenPropertyBags(runPairedWhileLoopsAsFor(mod)), + inlineArraysPass( + options, + inlineExpressionFunctions( + foldConstantGlobals( + copyStableCaptures( + resolveSelfCaptures( + hoistFinalFieldReads( + freezeInitOnlyGlobals( + holdCharLocalsAsUnits( + dropUnwrittenPropertyBags(runPairedWhileLoopsAsFor(mod)), + ), ), ), ), @@ -1724,6 +1742,9 @@ export class LlEmitter { /** ProgramLifetime hierarchy roots (contract types): their instances are * immortal from construction (classes.ts programLifetimeClass). */ readonly programLifetime: ReadonlySet; + /** Inline array fields this emission may keep inline (ir/inline-arrays.ts); + * the classes that do have LlClassMeta.inlineTail. */ + readonly inlineArrays: InlineArrayPlan; numericLocals = new Map(); captureIds = new Set(); /** Active induction bindings retain their width independently of size_t. */ @@ -2162,6 +2183,12 @@ export class LlEmitter { this.immutableFields, this.nodeIndex, ); + // Inline array fields (ir/inline-arrays.ts), where the module passes + // prepared their reads and constructions. + this.inlineArrays = + options.debugSources === undefined && inlineArraysApply(mod, options) + ? planInlineArrays(mod, programLifetimeClassOf(mod)) + : NO_INLINE_ARRAYS; const walkParameters = new Map>(); if (this.debug === null) { const host = { @@ -2470,6 +2497,7 @@ export class LlEmitter { return nullable !== null && isCompact(nullable.arm.className); } : undefined, + this.inlineArrays.classes, ); this.compactRegion = mod.lib === undefined && @@ -4669,6 +4697,13 @@ export class LlEmitter { refs: "pointer" | "compressed" = "pointer", ): { ptr: string; type: IrType; inline: boolean; compressed: boolean } { const meta = this.classMetaOf(className); + // An inline array field's slot holds its element count: its reads and + // writes go through backend/llvm/inline-arrays.ts. + const tail = this.inlineTailOf(className); + if (tail !== undefined && meta.def.fields[tail.field]!.name === field) + throw new InternalCompilerError( + `llvm emitter bug: field pointer to the inline array field ${className}.${field}`, + ); const { index, type, compressed } = classFieldIndex(meta, field, refs); const p = this.B.tmp(); this.B.line( @@ -7041,7 +7076,19 @@ export class LlEmitter { fieldBase: (shapeId) => this.nullableUnions.fieldBase(shapeId), classInfo: (className) => { const meta = this.classMetaOf(className); + const tail = this.inlineTailOf(className); + const tailField = tail === undefined ? undefined : meta.def.fields[tail.field]!; return { + ...(tail !== undefined && tailField !== undefined + ? { + tail: { + name: tailField.name, + countIndex: meta.compactSlots[tail.field]!, + offset: tail.tailOffset, + elem: { kind: "object" as const, className: tail.elemClass }, + }, + } + : {}), def: meta.def, hierarchy: meta.hierarchy, pre: meta.pre, @@ -7049,9 +7096,10 @@ export class LlEmitter { runtimeRooted: meta.root.def.runtime === true, idInRc: meta.idInRc, compact: meta.compact, - // Inline scalar-union fields are plain bits: nothing to push. - fields: meta.def.fields.flatMap((f) => - this.inlineUnions.classField(className, f.name) + // Inline scalar-union fields are plain bits: nothing to push. An + // inline array field's elements follow the fixed fields (`tail`). + fields: meta.def.fields.flatMap((f, i) => + this.inlineUnions.classField(className, f.name) || tail?.field === i ? [] : [ { @@ -7274,6 +7322,13 @@ export class LlEmitter { return emitReadReceiver(this, e); } + /** The inline array field of `className`'s instances, where they keep it + * inline (LlClassMeta.inlineTail; classes.ts gives their `new` the count). */ + inlineTailOf(className: string): LlInlineTail | undefined { + const meta = this.classMeta.get(className); + return meta !== undefined && meta.compact && this.compactRegion ? meta.inlineTail : undefined; + } + /** The frame slot a reassigned stack-object local keeps for its value. */ stackObjectSlot(localId: string, type: IrType): string { let slot = this.stackObjectSlots.get(localId); @@ -7463,6 +7518,15 @@ export class LlEmitter { return canBorrowReceiver(this, e); } + /** A read of an inline array field (ir/inline-arrays.ts). It makes a view + * of the elements that the read owns: no field holds an array that could + * keep a borrowed copy alive. Known before the class layouts are, from + * the module's plan; layouts that keep the field an ordinary array only + * lose the borrow. */ + inlineArrayRead(e: IrExpr): boolean { + return e.kind === "fieldGet" && this.inlineArrays.classes.get(e.className)?.field === e.field; + } + /** A reference held as one plain pointer that emitReadReceiver can read * without owning: class instances, records, arrays, strings, typed arrays * and nullable-pointer unions. Tagged union boxes, closures and dynamic @@ -7496,7 +7560,7 @@ export class LlEmitter { if (matchMapRead(e, this.boxedUnionsById)) return false; switch (e.kind) { case "fieldGet": - return this.nullableFieldGet(e) === null; + return this.nullableFieldGet(e) === null && !this.inlineArrayRead(e); case "recordGet": return true; // A keyed read whose hits are all the result's own pointer. diff --git a/packages/compiler/src/backend/llvm/expr-calls.ts b/packages/compiler/src/backend/llvm/expr-calls.ts index 18181de50..aedfdb907 100644 --- a/packages/compiler/src/backend/llvm/expr-calls.ts +++ b/packages/compiler/src/backend/llvm/expr-calls.ts @@ -49,6 +49,7 @@ import { ffiNativeTypeLl, ffiNativeParamLl, ffiNativeReturnLl, + SCR_ARR_FIELD, } from "./common.js"; import { canStackUnion, emitStackUnion } from "./stack-unions.js"; import { emitStackRecordKeyGet, scalarRecordKeyGet } from "./expr-records.js"; @@ -1103,8 +1104,28 @@ export function emitCallExpr( throw new InternalCompilerError( `llvm emitter bug: new ${e.className} without a constructor`, ); + // An instance with an inline array field is allocated with room for + // the elements of the array its constructor writes the field from + // (ir/inline-arrays.ts made that argument a local). + const tail = host.inlineTailOf(e.className); + let count = ""; + if (tail !== undefined) { + const source = e.args[tail.lengthArg]; + if (source?.kind !== "varRef") + throw new InternalCompilerError( + `llvm emitter bug: new ${e.className} reads its inline array from an expression`, + ); + const arr = host.emitReadReceiver(source); + const lenp = B.tmp(); + const len = B.tmp(); + B.line( + `${lenp} = getelementptr inbounds %ScrArr, ptr ${arr.name}, i32 0, i32 ${SCR_ARR_FIELD.len}`, + ); + B.line(`${len} = load i32, ptr ${lenp}`); + count = `i32 ${len}`; + } const o = B.tmp(); - B.line(`${o} = call ptr @${host.allocSym(mangleClassNew(e.className), e)}()`); + B.line(`${o} = call ptr @${host.allocSym(mangleClassNew(e.className), e)}(${count})`); const out = host.own({ name: o, type: e.type }); emitConstruction(host, e, o, ctor); return out; diff --git a/packages/compiler/src/backend/llvm/expr-containers.ts b/packages/compiler/src/backend/llvm/expr-containers.ts index 279afdcd0..51a81bd07 100644 --- a/packages/compiler/src/backend/llvm/expr-containers.ts +++ b/packages/compiler/src/backend/llvm/expr-containers.ts @@ -1,5 +1,6 @@ import { dynDesc } from "../../ir/analysis.js"; /* Focused LLVM expression emission extracted from emitter.ts. */ +import { emitInlineArrayLength, inlineFieldTail } from "./inline-arrays.js"; import { arraySearchHelperName, arraySearchKind } from "./array-search.js"; import { InternalCompilerError } from "../../errors.js"; import { isStableReceiverOperand, undefinedArmTag } from "../../ir/analysis.js"; @@ -650,6 +651,14 @@ export function emitArrIntrinsic( into?: { box: string; tag: string; payload: string }, ): LlValue { const B = host.B; + // The length of an inline array field, read from the instance. + if (e.method === "length" && e.receiver.kind === "fieldGet") { + const tail = inlineFieldTail(host, e.receiver.className, e.receiver.field); + if (tail !== undefined) { + const obj = host.emitStableReceiver(e.receiver.obj, e.args); + return emitInlineArrayLength(host, obj.name, e.receiver.className, tail, e.type); + } + } // Every array intrinsic borrows its receiver, including mutation and // retained-identity results. Scalar reads also admit the narrower proof // that a mutable binding survives the remaining operand evaluations. diff --git a/packages/compiler/src/backend/llvm/expr-primitives.ts b/packages/compiler/src/backend/llvm/expr-primitives.ts index 5d2a1cfeb..2ce8de895 100644 --- a/packages/compiler/src/backend/llvm/expr-primitives.ts +++ b/packages/compiler/src/backend/llvm/expr-primitives.ts @@ -1,4 +1,10 @@ /* Focused LLVM expression emission extracted from emitter.ts. */ +import { + emitInlineArrayGet, + emitInlineArrayState, + emitInlineArrayView, + inlineFieldTail, +} from "./inline-arrays.js"; import { InternalCompilerError } from "../../errors.js"; import { matchStringSelfConcat, undefinedArmTag } from "../../ir/analysis.js"; import { isRefCounted, type IrExpr, type IrStrCmpOp, type IrType } from "../../ir/ir.js"; @@ -1155,6 +1161,26 @@ export function emitContainerExpr( return out; } case "arrayGet": { + // An element of an inline array field, read from the instance. + if (e.arr.kind === "fieldGet" && e.arr.type.kind === "array") { + const tail = inlineFieldTail(host, e.arr.className, e.arr.field); + if (tail !== undefined) { + const obj = host.emitStableReceiver(e.arr.obj, [e.index]); + const idx = host.emitExpr(e.index); + const value = emitInlineArrayGet( + host, + obj.name, + e.arr.className, + tail, + idx, + e.arr.type.elem, + true, + ); + return isRefCounted(e.arr.type.elem) + ? host.own({ name: value, type: e.type }) + : { name: value, type: e.type }; + } + } // Element reads run no user code: a projection receiver (an array in // a parameter's field) stays rooted while the index preserves edges. const arr = host.emitStableReceiver(e.arr, [e.index]); @@ -1207,6 +1233,16 @@ export function emitContainerExpr( return host.own({ name: read.value, type: e.type }); } case "arrayHas": { + // An inline array field has no holes: presence is the bounds check. + if (e.arr.kind === "fieldGet") { + const tail = inlineFieldTail(host, e.arr.className, e.arr.field); + if (tail !== undefined) { + const obj = host.emitStableReceiver(e.arr.obj, [e.index]); + const idx = host.emitExpr(e.index); + const name = emitInlineArrayState(host, obj.name, e.arr.className, tail, idx, "has"); + return { name, type: e.type }; + } + } const arr = host.emitStableReceiver(e.arr, [e.index]); const idx = host.emitExpr(e.index); if (e.arr.type.kind !== "array") @@ -1226,6 +1262,16 @@ export function emitContainerExpr( return { name: t, type: e.type }; } case "arrayState": { + // An inline array field has no holes: presence is the bounds check. + if (e.arr.kind === "fieldGet") { + const tail = inlineFieldTail(host, e.arr.className, e.arr.field); + if (tail !== undefined) { + const obj = host.emitStableReceiver(e.arr.obj, [e.index]); + const idx = host.emitExpr(e.index); + const name = emitInlineArrayState(host, obj.name, e.arr.className, tail, idx, "state"); + return { name, type: e.type }; + } + } const arr = host.emitStableReceiver(e.arr, [e.index]); const idx = host.emitExpr(e.index); if (e.arr.type.kind !== "array") @@ -1330,6 +1376,14 @@ export function emitRecordExpr( switch (e.kind) { case "fieldGet": { const obj = host.emitReadReceiver(e.obj); + // An inline array field read as a value: a frozen array over the + // elements (backend/llvm/inline-arrays.ts). + const tail = inlineFieldTail(host, e.className, e.field); + if (tail !== undefined) + return host.own({ + name: emitInlineArrayView(host, obj.name, e.className, tail), + type: e.type, + }); const { ptr, type, inline } = host.classFieldPtr( obj.name, e.className, diff --git a/packages/compiler/src/backend/llvm/inline-arrays.ts b/packages/compiler/src/backend/llvm/inline-arrays.ts new file mode 100644 index 000000000..1ea36fd94 --- /dev/null +++ b/packages/compiler/src/backend/llvm/inline-arrays.ts @@ -0,0 +1,257 @@ +/* Inline array fields in the LLVM backend (ir/inline-arrays.ts plans them). + * + * A compact class with such a field (LlClassMeta.inlineTail) keeps the + * field's element count as an i32 in the field's slot and the elements as + * 32-bit region references after its fixed fields, at the subtree's tail + * offset. `new` reads the count from the constructor argument the field is + * written from and allocates the instance with room for them; the + * constructor's write copies the argument's elements there + * (scr_arr_copy_cref_tail). Reads that index the field, read its length or + * iterate it load the elements directly; any other read gives a frozen array + * over them (scr_arr_view_cref). */ + +import { isRefCounted, type IrModule, type IrType } from "../../ir/ir.js"; +import { mangleClassStruct } from "../mangle.js"; +import { compactRefLoadLines, type LlInlineTail } from "./classes.js"; +import { f64Lit } from "./common.js"; +import type { LlvmTargetOptions } from "./emitter.js"; +import type { LlValue, LlvmEmitterContext } from "./expr-context.js"; +import { arrOps } from "./shapes.js"; + +/** Whether `mod` emitted with `options` can keep inline array fields inline: + * an executable for a 64-bit target whose compact instances live in the low + * region (32-bit references), without lifetime regions (which keep their + * own class layouts), and with fields to keep. */ +export function inlineArraysApply(mod: IrModule, options: LlvmTargetOptions): boolean { + return ( + (mod.contracts?.inlineArrays?.length ?? 0) > 0 && + options.pointerBits !== 32 && + options.lowRegion === true && + mod.lib === undefined && + mod.lifetimeRegions !== true + ); +} + +/** Whether instances of `className` are ProgramLifetime instances: the + * root of its hierarchy is declared `implements ProgramLifetime`. */ +export function programLifetimeClassOf(mod: IrModule): (className: string) => boolean { + const roots = new Set(mod.contracts?.programLifetime ?? []); + const bases = new Map(); + for (const cls of mod.classes ?? []) bases.set(cls.name, cls.base); + return (className) => { + let name: string | undefined = className; + let root = className; + while (name !== undefined && bases.has(name)) { + root = name; + name = bases.get(name); + } + return bases.has(className) && roots.has(root); + }; +} + +/** The field of `className` whose elements its instances keep inline, when + * `field` is it. */ +export function inlineFieldTail( + host: LlvmEmitterContext, + className: string, + field: string, +): LlInlineTail | undefined { + const tail = host.inlineTailOf(className); + if (tail === undefined) return undefined; + return host.classMeta.get(className)!.def.fields[tail.field]!.name === field ? tail : undefined; +} + +/** Loads the element count of the inline array field of `obj` (an instance + * of `className`) as an i32; returns the value's name. */ +export function emitInlineCount( + host: LlvmEmitterContext, + obj: string, + className: string, + tail: LlInlineTail, +): string { + const B = host.B; + const meta = host.classMeta.get(className)!; + const countp = B.tmp(); + const count = B.tmp(); + B.line( + `${countp} = getelementptr inbounds %${mangleClassStruct(className)}, ptr ${obj}, i64 0, i32 ${meta.compactSlots[tail.field]!}`, + ); + B.line(`${count} = load i32, ptr ${countp}`); + return count; +} + +/** The address of the first element of the inline array field of `obj`. */ +export function emitInlineElements( + host: LlvmEmitterContext, + obj: string, + tail: LlInlineTail, +): string { + const B = host.B; + const at = B.tmp(); + B.line(`${at} = getelementptr inbounds i8, ptr ${obj}, i64 ${tail.tailOffset}`); + return at; +} + +/** The constructor's write of the inline array field of `obj`: copies the + * elements of `value` (an array) into the instance, which `new` allocated + * for that array's length. */ +export function emitInlineArrayInit( + host: LlvmEmitterContext, + obj: string, + className: string, + tail: LlInlineTail, + value: string, +): void { + const count = emitInlineCount(host, obj, className, tail); + const at = emitInlineElements(host, obj, tail); + host.declare(`declare void @scr_arr_copy_cref_tail(ptr, ptr, i32)`); + host.B.line(`call void @scr_arr_copy_cref_tail(ptr ${value}, ptr ${at}, i32 ${count})`); +} + +/** The inline array field of `obj` as an array value: a frozen array over + * the elements (scr_arr_view_cref). Returns the owned (+1) array's name. */ +export function emitInlineArrayView( + host: LlvmEmitterContext, + obj: string, + className: string, + tail: LlInlineTail, +): string { + const count = emitInlineCount(host, obj, className, tail); + const at = emitInlineElements(host, obj, tail); + const ops = arrOps(host.shapeHost, { kind: "object", className: tail.elemClass }); + host.declare(`declare ptr @scr_arr_view_cref(ptr, ptr, i32)`); + const view = host.B.tmp(); + host.B.line(`${view} = call ptr @scr_arr_view_cref(ptr ${ops}, ptr ${at}, i32 ${count})`); + return view; +} + +/** The index as an i64 slot, and whether the number is an integer below + * `count` (an i32). */ +function emitInlineIndex( + host: LlvmEmitterContext, + count: string, + index: LlValue, +): { slot: string; within: string } { + const B = host.B; + const bound = B.tmp(); + B.line(`${bound} = zext i32 ${count} to i64`); + const integer = index.integer; + if (integer !== undefined) { + let slot = integer.name; + if (integer.type === "i32") { + slot = B.tmp(); + B.line(`${slot} = ${integer.signed ? "sext" : "zext"} i32 ${integer.name} to i64`); + } + const within = B.tmp(); + B.line(`${within} = icmp ult i64 ${slot}, ${bound}`); + return { slot, within }; + } + host.declare(`declare i64 @llvm.fptosi.sat.i64.f64(double)`); + const truncated = B.tmp(); + const back = B.tmp(); + const exact = B.tmp(); + const below = B.tmp(); + const within = B.tmp(); + B.line(`${truncated} = call i64 @llvm.fptosi.sat.i64.f64(double ${index.name})`); + B.line(`${back} = sitofp i64 ${truncated} to double`); + B.line(`${exact} = fcmp oeq double ${back}, ${index.name}`); + B.line(`${below} = icmp ult i64 ${truncated}, ${bound}`); + B.line(`${within} = and i1 ${exact}, ${below}`); + return { slot: truncated, within }; +} + +/** `index in obj.field` (`has`, an i1) or the slot state (`state`, a double: + * HOLE=0, VALUE=1) for an inline array field. Its elements are instances, + * never holes or undefined, so an index is present exactly when it is an + * integer within the count. */ +export function emitInlineArrayState( + host: LlvmEmitterContext, + obj: string, + className: string, + tail: LlInlineTail, + index: LlValue, + kind: "has" | "state", +): string { + const count = emitInlineCount(host, obj, className, tail); + const { within } = emitInlineIndex(host, count, index); + if (kind === "has") return within; + const state = host.B.tmp(); + host.B.line(`${state} = select i1 ${within}, double ${f64Lit(1)}, double ${f64Lit(0)}`); + return state; +} + +/** `obj.field[index]` for an inline array field: the element at an index + * within the count, loaded from the instance. Any other index takes the + * runtime getter over a view, which reports it like a read of an ordinary + * array. With `retain`, the result is an owned reference when the element + * type is reference counted (like emitDenseArrayRead); without, borrowed + * from the instance. */ +export function emitInlineArrayGet( + host: LlvmEmitterContext, + obj: string, + className: string, + tail: LlInlineTail, + index: LlValue, + element: IrType, + retain: boolean, +): string { + const B = host.B; + const count = emitInlineCount(host, obj, className, tail); + const at = emitInlineElements(host, obj, tail); + const { slot, within } = emitInlineIndex(host, count, index); + const hit = B.newLabel("inl.get.hit"); + const miss = B.newLabel("inl.get.miss"); + const join = B.newLabel("inl.get.join"); + B.condBr(within, hit, miss); + B.startBlock(hit); + const p = B.tmp(); + const ref = B.tmp(); + const raw = B.tmp(); + B.line(`${p} = getelementptr inbounds i32, ptr ${at}, i64 ${slot}`); + B.line(`${ref} = load i32, ptr ${p}`); + for (const line of compactRefLoadLines(ref, raw, `${raw}.r`)) B.line(line.trimStart()); + const value = retain && isRefCounted(element) ? host.retainFrame(raw, element) : raw; + const hitEnd = B.currentLabel(); + B.br(join); + B.startBlock(miss); + // Outside the count: the getter of an ordinary array over the elements + // decides, as for any array (an out-of-bounds read traps). + const ops = arrOps(host.shapeHost, { kind: "object", className: tail.elemClass }); + const getter = retain ? "scr_arr_get_ref" : "scr_arr_borrow_ref"; + host.declare(`declare ptr @scr_arr_view_cref(ptr, ptr, i32)`); + host.declare(`declare ptr @${getter}(ptr, double)`); + host.declare(`declare void @scr_arr_release(ptr)`); + const view = B.tmp(); + const slow = B.tmp(); + B.line(`${view} = call ptr @scr_arr_view_cref(ptr ${ops}, ptr ${at}, i32 ${count})`); + B.line(`${slow} = call ptr @${getter}(ptr ${view}, double ${indexDouble(host, index)}) cold`); + B.line(`call void @scr_arr_release(ptr ${view})`); + const missEnd = B.currentLabel(); + B.br(join); + B.startBlock(join); + const result = B.tmp(); + B.line(`${result} = phi ptr [ ${value}, %${hitEnd} ], [ ${slow}, %${missEnd} ]`); + return result; +} + +/** The index as a double (an integer snapshot's own double is its name). */ +function indexDouble(host: LlvmEmitterContext, index: LlValue): string { + if (index.type.kind === "f64") return index.name; + const d = host.B.tmp(); + host.B.line(`${d} = sitofp i32 ${index.name} to double`); + return d; +} + +/** `obj.field.length` for an inline array field. */ +export function emitInlineArrayLength( + host: LlvmEmitterContext, + obj: string, + className: string, + tail: LlInlineTail, + type: IrType, +): LlValue { + const count = emitInlineCount(host, obj, className, tail); + const length = host.B.tmp(); + host.B.line(`${length} = uitofp i32 ${count} to double`); + return { name: length, type, uint32: count }; +} diff --git a/packages/compiler/src/backend/llvm/local-array-reads.ts b/packages/compiler/src/backend/llvm/local-array-reads.ts index 80ed2c440..8e9b1434d 100644 --- a/packages/compiler/src/backend/llvm/local-array-reads.ts +++ b/packages/compiler/src/backend/llvm/local-array-reads.ts @@ -8,6 +8,7 @@ import { type IrType, type IrUnionDef, } from "../../ir/ir.js"; +import { emitInlineArrayGet, inlineFieldTail } from "./inline-arrays.js"; import { everyStmtList } from "../../ir/traverse.js"; import { analyzeCallLifetimes, type CallLifetimes } from "./call-lifetimes.js"; import type { LlValue, LlvmEmitterContext } from "./expr-context.js"; @@ -607,6 +608,25 @@ export function emitBorrowedArrayRead( slot: string, ): void { const B = host.B; + // An element of an inline array field, borrowed from the instance. + if (read.arr.kind === "fieldGet" && read.arr.type.kind === "array") { + const tail = inlineFieldTail(host, read.arr.className, read.arr.field); + if (tail !== undefined) { + const obj = host.emitStableReceiver(read.arr.obj, [read.index]); + const index = host.emitExpr(read.index); + const value = emitInlineArrayGet( + host, + obj.name, + read.arr.className, + tail, + index, + read.arr.type.elem, + false, + ); + B.line(`store ptr ${value}, ptr ${slot}`); + return; + } + } const array = host.emitStableReceiver(read.arr, [read.index]); const loopIndex = host.emitIntegerLoopIndex(read.index); if (loopIndex !== null && host.provenDenseIndex(read.arr, read.index)) { diff --git a/packages/compiler/src/backend/llvm/publish.ts b/packages/compiler/src/backend/llvm/publish.ts index 62fdd75ec..2c389a527 100644 --- a/packages/compiler/src/backend/llvm/publish.ts +++ b/packages/compiler/src/backend/llvm/publish.ts @@ -80,6 +80,10 @@ export interface PublishClassInfo { * `compressed` when the field holds a 32-bit reference (classes.ts * LlClassMeta.compressed), which the walker decodes. */ fields: { name: string; index: number; type: IrType; compressed?: boolean }[]; + /** An inline array field (classes.ts LlClassMeta.inlineTail), which + * `fields` leaves out: the GEP index of its i32 element count, the byte + * offset of its 32-bit references and their type. */ + tail?: { name: string; countIndex: number; offset: number; elem: IrType }; } /** scr_pub_mark's object kinds (SCR_PUB_KIND_* in scr_runtime.h), for the RC audit. */ @@ -405,6 +409,7 @@ export class PublishWalkers { loc, info.idInRc, info.compact, + info.tail, ); } @@ -563,6 +568,7 @@ export class PublishWalkers { loc: SrcLoc, classId = false, compact = false, + tail?: PublishClassInfo["tail"], ): string[] { const lines = [ head, @@ -591,7 +597,33 @@ export class PublishWalkers { ); n++; } - lines.push(` br label %done`, `done:`, ` ret void`, `}`); + const tailFn = + tail === undefined ? null : this.walker(tail.elem, `${path}.${tail.name}[]`, loc); + if (tail !== undefined && tailFn !== null) { + // Each element of an inline array field, as a 32-bit reference. + lines.push( + ` br label %tail.pre`, + `tail.pre:`, + ` %tail.cp = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${tail.countIndex} ; ${llvmComment(tail.name)}`, + ` %tail.n = load i32, ptr %tail.cp`, + ` %tail.at = getelementptr inbounds i8, ptr %o, i64 ${tail.offset}`, + ` br label %tail.head`, + `tail.head:`, + ` %tail.i = phi i32 [ 0, %tail.pre ], [ %tail.next, %tail.latch ]`, + ` %tail.end = icmp uge i32 %tail.i, %tail.n`, + ` br i1 %tail.end, label %done, label %tail.body`, + `tail.body:`, + ` %tail.p = getelementptr inbounds i32, ptr %tail.at, i32 %tail.i`, + ` %tail.r = load i32, ptr %tail.p`, + ...compactRefLoadLines(`%tail.r`, `%tail.v`, `%tail.d`), + ...this.push(`%tail.v`, tailFn, `tail`, this.compactRef(tail.elem)), + ` br label %tail.latch`, + `tail.latch:`, + ` %tail.next = add nuw i32 %tail.i, 1`, + ` br label %tail.head`, + ); + } else lines.push(` br label %done`); + lines.push(`done:`, ` ret void`, `}`); return lines; } } diff --git a/packages/compiler/src/backend/llvm/receivers.ts b/packages/compiler/src/backend/llvm/receivers.ts index 8524a51fb..41c09b344 100644 --- a/packages/compiler/src/backend/llvm/receivers.ts +++ b/packages/compiler/src/backend/llvm/receivers.ts @@ -1,4 +1,5 @@ /* LlEmitter's receivers: receiver reads, and the analyses that decide when calls, receivers and returns borrow. */ +import { inlineFieldTail } from "./inline-arrays.js"; import type { LlEmitter } from "./emitter.js"; import { type IrExpr, @@ -128,6 +129,9 @@ export function emitReadReceiver(emitter: LlEmitter, e: IrExpr): LlValue { // A nullable-pointer field has no box to borrow. Projections use // emitUnionProjection; other borrowers get an owned heap box. if (emitter.nullableFieldGet(e)) return emitter.emitExpr(e); + // An inline array field has no array to borrow: its read makes a view. + if (e.kind === "fieldGet" && inlineFieldTail(emitter, e.className, e.field) !== undefined) + return emitter.emitExpr(e); if (e.kind === "recordGet" || e.kind === "fieldGet") { const receiver = emitter.emitReadReceiver(e.obj); if (e.kind === "recordGet") { @@ -201,8 +205,9 @@ export function canBorrowCallArgument(emitter: LlEmitter, value: IrExpr): boolea // An exact capsule owns its instance while the capsule's binding holds it. if (value.kind === "dynCheck" && value.exact) return value.value.kind === "varRef" && emitter.canBorrowCallArgument(value.value); - // An inline union field has no box to lend. - if (emitter.inlineFieldGet(value)) return false; + // An inline union field has no box to lend, an inline array field no + // array. + if (emitter.inlineFieldGet(value) || emitter.inlineArrayRead(value)) return false; // Class casts reinterpret the same pointer (prefix layouts). if ((value.kind === "upcast" || value.kind === "downcast") && value.value.type.kind === "object") return emitter.canBorrowCallArgument(value.value); @@ -285,10 +290,15 @@ export function canBorrowReceiver(emitter: LlEmitter, e: IrExpr): boolean { case "upcast": return emitter.canBorrowReceiver(e.value); // An inline union field has no box to lend: projections read it - // through emitReadReceiver's private stack box. + // through emitReadReceiver's private stack box. An inline array field + // has no array to lend. case "fieldGet": case "recordGet": - return !emitter.inlineFieldGet(e) && emitter.canBorrowReceiver(e.obj); + return ( + !emitter.inlineFieldGet(e) && + !emitter.inlineArrayRead(e) && + emitter.canBorrowReceiver(e.obj) + ); case "ternary": return emitter.canBorrowReceiver(e.then) && emitter.canBorrowReceiver(e.else_); case "seqExpr": { diff --git a/packages/compiler/src/contracts/check.ts b/packages/compiler/src/contracts/check.ts index 6d8a19956..812306eea 100644 --- a/packages/compiler/src/contracts/check.ts +++ b/packages/compiler/src/contracts/check.ts @@ -67,6 +67,7 @@ import { type ContractAnnotations, type ContractNote, } from "../frontend/contract-types.js"; +import { planInlineArrays } from "../ir/inline-arrays.js"; /** The largest array index: 2^32 - 2 (ECMA-262 array index). */ const MAX_ARRAY_INDEX = 2 ** 32 - 2; @@ -255,6 +256,7 @@ class ContractChecker { run(): ContractCheckResult { this.checkProgramLifetime(); + this.checkInlineArrays(); this.checkNoThrow(); this.checkAcyclic(); const elements = new Int32ElementContracts(this); @@ -323,6 +325,65 @@ class ContractChecker { this.record("programLifetime", roots); } + /* ── InlineArray ── */ + + /** `readonly f: InlineArray` (experimental, ir/inline-arrays.ts): the + * field's elements may live in the instance itself, which only a field + * written once from a constructor argument and never again can have. + * Checked: the class and C are ProgramLifetime classes, the constructor of + * the declaring class writes the field once from one of its parameters, + * every subclass constructor passes it one of its own through super(...), + * nothing else writes the field, no read of it is compared by identity + * (two reads need not give the same array), and no instance of the + * hierarchy is constructed through a class value or becomes a dynamic + * value. */ + private checkInlineArrays(): void { + if (this.notes.inlineArrays.size === 0) return; + const roots = new Set(this.contracts.programLifetime ?? []); + const lifetime = (className: string): boolean => { + let cls = this.classes.get(className); + while (cls?.base !== undefined) cls = this.classes.get(cls.base); + return cls !== undefined && roots.has(cls.name); + }; + const entries: string[] = []; + const notes = new Map(); + for (const [key, note] of this.notes.inlineArrays) { + const at = key.lastIndexOf("\0"); + const field = key.slice(at + 1); + for (const cls of this.classesAt(key.slice(0, at))) { + if (!lifetime(cls.name)) { + this.report( + "SC7009", + note.loc, + `'${note.name}' is an InlineArray field of a class that is not ProgramLifetime`, + "declare the root class of the hierarchy `implements ProgramLifetime`: inline elements live in the instance, which is never freed", + ); + continue; + } + const entry = `${cls.name}\0${field}`; + entries.push(entry); + notes.set(entry, note); + } + } + if (entries.length === 0) return; + const plan = planInlineArrays( + { ...this.mod, contracts: { ...this.mod.contracts, inlineArrays: entries } }, + lifetime, + ); + for (const r of plan.rejected) { + const note = notes.get(`${r.declaringClass}\0${r.field}`)!; + this.report( + "SC7009", + note.loc, + `'${note.name}' cannot keep its InlineArray contract: ${r.reason}`, + "write the field once in the constructor from one of its parameters (subclasses pass theirs through super(...)), never again, and keep the class's instances out of class values and dynamic values", + ); + } + const kept = new Set(); + for (const cls of plan.classes.values()) kept.add(`${cls.declaringClass}\0${cls.field}`); + this.record("inlineArrays", [...kept]); + } + /* ── Acyclic ── */ private checkAcyclic(): void { diff --git a/packages/compiler/src/diagnostics/codes.ts b/packages/compiler/src/diagnostics/codes.ts index b9d255df9..5e3fca2f0 100644 --- a/packages/compiler/src/diagnostics/codes.ts +++ b/packages/compiler/src/diagnostics/codes.ts @@ -107,6 +107,7 @@ export const DIAGNOSTIC_CODES = { SC7006: "Acyclic contract in an unchecked position", SC7007: "Acyclic type can lie on a reference cycle", SC7008: "ProgramLifetime class extends another class", + SC7009: "InlineArray field contract not kept", SC8002: "Node islands off for a package or module outside the native subset", SC8003: "package marked native is outside the native subset", SC8004: "unknown scriptc.packages or scriptc.nodeIslands configuration", diff --git a/packages/compiler/src/diagnostics/diagnostic.ts b/packages/compiler/src/diagnostics/diagnostic.ts index 256365b36..dd88e72b0 100644 --- a/packages/compiler/src/diagnostics/diagnostic.ts +++ b/packages/compiler/src/diagnostics/diagnostic.ts @@ -71,7 +71,7 @@ * build checks (SC6103, frontend/shape-inference) * SC7xxx contract types (@scriptc/types): a NoThrow, Int32, Immutable, * Acyclic or ProgramLifetime contract that does not hold or - * cannot be checked (SC7001–SC7008, contracts/check.ts) + * cannot be checked (SC7001–SC7009, contracts/check.ts) * SC8xxx Node island policy: packages and modules outside the native * subset that run (or, in report-only mode, would run) in a * Node island — unknown project configuration (SC8004), a diff --git a/packages/compiler/src/frontend/contract-types.ts b/packages/compiler/src/frontend/contract-types.ts index 71bbeb376..550a716ef 100644 --- a/packages/compiler/src/frontend/contract-types.ts +++ b/packages/compiler/src/frontend/contract-types.ts @@ -16,15 +16,33 @@ import { type HeritageElement, isHeritageElement } from "./heritage-elements.js" export const CONTRACT_TYPES_MODULE = "@scriptc/types"; -export type ContractMarker = "Int32" | "NoThrow" | "Immutable" | "Acyclic" | "ProgramLifetime"; +/** The module and name of the inline array field marker, which is + * experimental: `readonly f: InlineArray` on an instance field of a + * ProgramLifetime class (ir/inline-arrays.ts). */ +export const INLINE_ARRAY_MODULE = "@scriptc/experimental"; +export const INLINE_ARRAY_MARKER = "InlineArray"; + +export type ContractMarker = + | "Int32" + | "NoThrow" + | "Immutable" + | "Acyclic" + | "ProgramLifetime" + | typeof INLINE_ARRAY_MARKER; const MARKERS: ReadonlySet = new Set([ "Int32", "NoThrow", "Immutable", "Acyclic", "ProgramLifetime", + INLINE_ARRAY_MARKER, ]); +/** The module a marker is imported from. */ +function markerModule(marker: string): string { + return marker === INLINE_ARRAY_MARKER ? INLINE_ARRAY_MODULE : CONTRACT_TYPES_MODULE; +} + /** One written contract: what it promises and where the annotation is. */ export interface ContractNote { marker: ContractMarker; @@ -67,6 +85,9 @@ export interface ContractAnnotations { acyclic: AcyclicNote[]; /** ProgramLifetime classes (`implements ProgramLifetime`) by class key. */ classes: Map; + /** Inline array fields (`readonly f: InlineArray`) by + * `${class key}\0${field name}`. */ + inlineArrays: Map; diagnostics: ScrDiagnostic[]; /** Number of annotations found (for timing and tests). */ count: number; @@ -123,7 +144,7 @@ function markerNamed( importDecl === undefined || !ts.isImportDeclaration(importDecl) || !ts.isStringLiteral(importDecl.moduleSpecifier) || - importDecl.moduleSpecifier.text !== CONTRACT_TYPES_MODULE + importDecl.moduleSpecifier.text !== markerModule(member) ) return null; return MARKERS.has(member) ? (member as ContractMarker) : null; @@ -158,6 +179,7 @@ export function collectContractAnnotations( fields: new Map(), acyclic: [], classes: new Map(), + inlineArrays: new Map(), diagnostics: [], count: 0, }; @@ -166,7 +188,11 @@ export function collectContractAnnotations( ts.SyntaxKind.ExpressionWithTypeArguments, ]); for (const sf of files) { - if (sf.isDeclarationFile || !sf.text.includes(CONTRACT_TYPES_MODULE)) continue; + if ( + sf.isDeclarationFile || + (!sf.text.includes(CONTRACT_TYPES_MODULE) && !sf.text.includes(INLINE_ARRAY_MODULE)) + ) + continue; ts.walkPreorder( sf, (node) => { @@ -183,6 +209,7 @@ export function collectContractAnnotations( out.count++; if (marker === "Acyclic") placeAcyclicAlias(out, checker, node, resolveUnit); else if (marker === "ProgramLifetime") unsupported(out, node, marker); + else if (marker === INLINE_ARRAY_MARKER) placeInlineArray(out, node); else place(out, checker, node, marker); }, kinds, @@ -203,6 +230,17 @@ function unsupported(out: ContractAnnotations, node: ts.Node, marker: ContractMa ); return; } + if (marker === INLINE_ARRAY_MARKER) { + out.diagnostics.push( + contractDiag( + "SC7006", + locOf(node), + "the InlineArray contract is not checked in this position (InlineArray is checked on instance fields of ProgramLifetime classes)", + "write it directly as the type of an instance field: readonly f: InlineArray", + ), + ); + return; + } const where = marker === "NoThrow" ? "NoThrow wraps a function's written return type" @@ -282,6 +320,28 @@ function placeHeritage( unsupported(out, node, marker); } +/** `readonly f: InlineArray`: an instance field of a class. */ +function placeInlineArray(out: ContractAnnotations, ref: ts.TypeReferenceNode): void { + const parent = ref.parent; + if ( + parent === undefined || + !ts.isPropertyDeclaration(parent) || + parent.type !== ref || + ref.typeArguments?.nodes.length !== 1 || + ts.getModifiers(parent)?.some((m) => m.kind === ts.SyntaxKind.StaticKeyword) === true || + !(ts.isIdentifier(parent.name) || ts.isPrivateIdentifier(parent.name)) + ) + return unsupported(out, ref, INLINE_ARRAY_MARKER); + const cls = parent.parent; + if (cls === undefined || !(ts.isClassDeclaration(cls) || ts.isClassExpression(cls))) + return unsupported(out, ref, INLINE_ARRAY_MARKER); + out.inlineArrays.set(`${contractLocKey(locOf(cls))}\0${parent.name.text}`, { + marker: INLINE_ARRAY_MARKER, + loc: locOf(ref), + name: `${nameText(cls.name)}.${parent.name.text}`, + }); +} + /** `type T = Acyclic<{ ... }>`: the aliased object type. */ function placeAcyclicAlias( out: ContractAnnotations, diff --git a/packages/compiler/src/frontend/lowering/lowerer.ts b/packages/compiler/src/frontend/lowering/lowerer.ts index 3d10d1982..335bb7b40 100644 --- a/packages/compiler/src/frontend/lowering/lowerer.ts +++ b/packages/compiler/src/frontend/lowering/lowerer.ts @@ -4312,6 +4312,7 @@ export class Lowerer { classImplementedProtocol: (type) => this.classImplementedProtocol(type), isStdlibFile: (sf) => this.isStdlibFile(sf), isNpmFile: (sf) => this.isNpmFile(sf), + isExperimentalFile: (sf) => isExperimentalDeclarationFile(sf, this.experimentalFiles), // The declaration-typed boundary's generated types (decl-projection.ts) // describe data, like an explicitly mapped external type surface. isExternalTypeFile: (sf) => diff --git a/packages/compiler/src/frontend/type-mapper.ts b/packages/compiler/src/frontend/type-mapper.ts index 40ca10b38..a150b1411 100644 --- a/packages/compiler/src/frontend/type-mapper.ts +++ b/packages/compiler/src/frontend/type-mapper.ts @@ -1337,6 +1337,9 @@ export interface TypeMapperCtx { isStdlibFile: (sf: ts.SourceFile) => boolean; isNpmFile: (sf: ts.SourceFile) => boolean; + /** The declaration file of @scriptc/experimental, wherever it is installed + * (a workspace links it outside node_modules). */ + isExperimentalFile?: (sf: ts.SourceFile) => boolean; /** True for a declaration file explicitly mapped by coverage's * --external-types option. These declarations are trusted as structural * type descriptions for project-owned values, but their imported runtime @@ -7370,9 +7373,14 @@ function isTypeOnlyNamespace(d: ts.Node): boolean { ); } -function packageTypeFile(sf: ts.SourceFile, ctx: Pick): boolean { +function packageTypeFile( + sf: ts.SourceFile, + ctx: Pick, +): boolean { return ( - sf.isDeclarationFile && /[\\/]node_modules[\\/]/.test(sf.fileName) && !ctx.isStdlibFile(sf) + sf.isDeclarationFile && + (/[\\/]node_modules[\\/]/.test(sf.fileName) || ctx.isExperimentalFile?.(sf) === true) && + !ctx.isStdlibFile(sf) ); } diff --git a/packages/compiler/src/ir/function-references.test.ts b/packages/compiler/src/ir/function-references.test.ts index 450810368..9a59af607 100644 --- a/packages/compiler/src/ir/function-references.test.ts +++ b/packages/compiler/src/ir/function-references.test.ts @@ -219,6 +219,7 @@ const MODULE_FIELDS: Record>, Role> = { "contracts.acyclic[]": "other", "contracts.immortalGlobals[]": "other", "contracts.programLifetime[]": "other", + "contracts.inlineArrays[]": "other", "wasmModules[].sha256": "other", "wasmModules[].bytes": "other", "wasmUntranslated[].sha256": "other", diff --git a/packages/compiler/src/ir/inline-arrays.test.ts b/packages/compiler/src/ir/inline-arrays.test.ts new file mode 100644 index 000000000..accecf101 --- /dev/null +++ b/packages/compiler/src/ir/inline-arrays.test.ts @@ -0,0 +1,290 @@ +import { expect, test } from "vitest"; +import { inlineArrayReads, planInlineArrays } from "./inline-arrays.js"; +import { + F64, + VOID, + type IrClassDef, + type IrExpr, + type IrFunction, + type IrModule, + type IrStmt, + type IrType, +} from "./ir.js"; + +const loc = { file: "inline-arrays.ts", start: 0, end: 0 }; +const obj = (className: string): IrType => ({ kind: "object", className }); +const ITEMS: IrType = { kind: "array", elem: obj("Item") }; +const ref = (localId: string, type: IrType): IrExpr => ({ loc, kind: "varRef", localId, type }); +const set = (target: IrExpr, className: string, field: string, value: IrExpr): IrStmt => ({ + loc, + kind: "fieldSet", + obj: target, + className, + field, + value, +}); +const get = (target: IrExpr, className: string, field: string, type: IrType): IrExpr => ({ + loc, + kind: "fieldGet", + obj: target, + className, + field, + type, +}); +const fn = ( + name: string, + params: { id: string; type: IrType }[], + body: IrStmt[], + locals: { id: string; type: IrType; mutable?: boolean }[] = [], +): IrFunction => ({ + loc, + name, + params: params.map((p) => ({ localId: p.id, name: p.id, type: p.type })), + locals: [...params, ...locals].map((l) => ({ + id: l.id, + name: l.id, + type: l.type, + mutable: "mutable" in l ? l.mutable === true : false, + })), + returnType: VOID, + body, +}); +const cls = (name: string, fields: [string, IrType][], base?: string): IrClassDef => ({ + loc, + name, + ...(base ? { base } : {}), + fields: fields.map(([field, type]) => ({ name: field, type })), +}); +const listFields: [string, IrType][] = [ + ["pos", F64], + ["items", ITEMS], +]; +const listCtor = fn( + "%List.constructor", + [ + { id: "this", type: obj("List") }, + { id: "pos", type: F64 }, + { id: "items", type: ITEMS }, + ], + [ + set(ref("this", obj("List")), "List", "pos", ref("pos", F64)), + set(ref("this", obj("List")), "List", "items", ref("items", ITEMS)), + ], +); +// `class Flagged extends List { constructor(items, flags, pos) { super(pos, items) } }`. +const flaggedCtor = fn( + "%Flagged.constructor", + [ + { id: "this", type: obj("Flagged") }, + { id: "nodes", type: ITEMS }, + { id: "flags", type: F64 }, + { id: "pos", type: F64 }, + ], + [ + { + loc, + kind: "exprStmt", + expr: { + loc, + kind: "call", + callee: "%List.constructor", + args: [ + { loc, kind: "upcast", value: ref("this", obj("Flagged")), type: obj("List") }, + ref("pos", F64), + ref("nodes", ITEMS), + ], + type: VOID, + }, + }, + set(ref("this", obj("Flagged")), "Flagged", "flags", ref("flags", F64)), + ], +); +const itemCtor = fn("%Item.constructor", [{ id: "this", type: obj("Item") }], []); +const mod = (functions: IrFunction[], extra: IrClassDef[] = []): IrModule => ({ + irVersion: 17, + sourceFile: "inline-arrays.ts", + entry: "%main", + classes: [ + cls("Item", []), + cls("List", listFields), + cls("Flagged", [...listFields, ["flags", F64]], "List"), + ...extra, + ], + functions: [fn("%main", [], []), itemCtor, listCtor, flaggedCtor, ...functions], + records: [], + unions: [], + globals: [], + contracts: { programLifetime: ["Item", "List"], inlineArrays: ["List\0items"] }, +}); +const lifetime = (name: string): boolean => + name === "Item" || name === "List" || name === "Flagged"; + +test("a subtree is planned with the argument each class takes its length from", () => { + const plan = planInlineArrays(mod([]), lifetime); + expect(plan.rejected).toEqual([]); + expect(plan.classes.get("List")).toMatchObject({ field: "items", fieldIndex: 1, lengthArg: 1 }); + // Flagged passes its first parameter where List takes the array. + expect(plan.classes.get("Flagged")).toMatchObject({ fieldIndex: 1, lengthArg: 0 }); +}); + +test("other writes, class values and dynamic values keep an ordinary field", () => { + const conditional = mod([]); + conditional.functions = conditional.functions.map((f) => + f.name !== "%List.constructor" + ? f + : { + ...f, + body: [ + { + loc, + kind: "if", + cond: { loc, kind: "boolLit", value: true, type: { kind: "bool" } }, + then: [set(ref("this", obj("List")), "List", "items", ref("items", ITEMS))], + else_: null, + }, + ], + }, + ); + expect(planInlineArrays(conditional, lifetime).rejected[0]?.reason).toContain("conditionally"); + + const rewritten = mod([ + fn( + "%List.reset", + [ + { id: "this", type: obj("List") }, + { id: "other", type: ITEMS }, + ], + [set(ref("this", obj("List")), "List", "items", ref("other", ITEMS))], + ), + ]); + expect(planInlineArrays(rewritten, lifetime).rejected[0]?.reason).toContain( + "outside the constructor", + ); + + const value = mod([ + fn( + "%use", + [], + [ + { + loc, + kind: "exprStmt", + expr: { loc, kind: "classRef", className: "Flagged", type: { kind: "dyn" } }, + }, + ], + ), + ]); + expect(planInlineArrays(value, lifetime).rejected[0]?.reason).toContain("used as a value"); + + const dynamic = mod([ + fn( + "%show", + [{ id: "list", type: obj("List") }], + [ + { + loc, + kind: "exprStmt", + expr: { loc, kind: "dynFrom", value: ref("list", obj("List")), type: { kind: "dyn" } }, + }, + ], + ), + ]); + expect(planInlineArrays(dynamic, lifetime).rejected[0]?.reason).toContain("dynamic value"); +}); + +test("an alias that only indexes the field reads the field", () => { + const read: IrExpr = { + loc, + kind: "arrayGet", + arr: ref("nodes", ITEMS), + index: { loc, kind: "numLit", value: 0, type: F64 }, + type: obj("Item"), + }; + const sum = fn( + "%sum", + [{ id: "list", type: obj("List") }], + [ + { + loc, + kind: "varDecl", + localId: "nodes", + init: get(ref("list", obj("List")), "List", "items", ITEMS), + }, + { loc, kind: "exprStmt", expr: read }, + { + loc, + kind: "exprStmt", + expr: { + loc, + kind: "arrIntrinsic", + method: "length", + receiver: ref("nodes", ITEMS), + args: [], + type: F64, + }, + }, + ], + [{ id: "nodes", type: ITEMS }], + ); + const m = mod([sum]); + const out = inlineArrayReads(m, planInlineArrays(m, lifetime)).functions.find( + (f) => f.name === "%sum", + )!; + const text = JSON.stringify(out.body); + // The alias is gone: its reads read the field through the parameter. + expect(text).not.toContain('"localId":"nodes"'); + expect(text).toContain('"kind":"fieldGet"'); + expect(out.body[0]).toMatchObject({ kind: "block", body: [] }); +}); + +test("an alias that escapes stays an array local", () => { + const sum = fn( + "%keep", + [{ id: "list", type: obj("List") }], + [ + { + loc, + kind: "varDecl", + localId: "nodes", + init: get(ref("list", obj("List")), "List", "items", ITEMS), + }, + { loc, kind: "return", value: ref("nodes", ITEMS) }, + ], + [{ id: "nodes", type: ITEMS }], + ); + const m = mod([sum]); + const out = inlineArrayReads(m, planInlineArrays(m, lifetime)); + expect(out.functions.find((f) => f.name === "%keep")).toBe(sum); +}); + +test("new evaluates its arguments into locals before allocating", () => { + const literal: IrExpr = { loc, kind: "arrayLit", elems: [], type: ITEMS }; + const build = fn( + "%build", + [], + [ + { + loc, + kind: "exprStmt", + expr: { + loc, + kind: "new", + className: "List", + args: [{ loc, kind: "numLit", value: 1, type: F64 }, literal], + type: obj("List"), + }, + }, + ], + ); + const m = mod([build]); + const out = inlineArrayReads(m, planInlineArrays(m, lifetime)).functions.find( + (f) => f.name === "%build", + )!; + const stmt = out.body[0] as IrStmt & { kind: "exprStmt" }; + expect(stmt.expr.kind).toBe("seqExpr"); + const seq = stmt.expr as IrExpr & { kind: "seqExpr" }; + expect(seq.stmts.map((s) => s.kind)).toEqual(["varDecl", "varDecl"]); + const made = seq.result as IrExpr & { kind: "new" }; + expect(made.args.every((a) => a.kind === "varRef")).toBe(true); + expect(out.locals.filter((l) => l.id.startsWith("%inl.arg"))).toHaveLength(2); +}); diff --git a/packages/compiler/src/ir/inline-arrays.ts b/packages/compiler/src/ir/inline-arrays.ts new file mode 100644 index 000000000..4cb929e7a --- /dev/null +++ b/packages/compiler/src/ir/inline-arrays.ts @@ -0,0 +1,495 @@ +/* Inline array fields (IrContracts.inlineArrays): an array field of a + * ProgramLifetime class whose elements the backend may keep in the + * instance itself, after its fixed fields, instead of in an array object. + * + * The contract makes that possible: the declaring class's constructor writes + * the field once, from one of its parameters, and nothing else writes it, so + * the field's length is the length of a constructor argument and the + * elements never change. `planInlineArrays` finds the hierarchies where the + * IR shows exactly that, and the `new` argument each class's instance takes + * its length from: the declaring class's parameter, or the parameter a + * subclass constructor passes there through its `super(...)` call. A + * hierarchy whose IR shows anything else keeps an ordinary array field + * (the layout is the backend's choice; the field's value is the same). + * + * `inlineArrayReads` then prepares the IR for the layout: + * - a `new` of such a class evaluates its arguments into locals first, in + * order, when the length argument is not already a local, so that the + * backend can read the length before it allocates; + * - an immutable local initialized from the field (`const nodes = + * list.nodes`) whose every use indexes it, reads its length or iterates + * it with `for...of` is replaced by reads of the field, through a local + * holding the instance, so that those reads go straight to the elements. */ + +import type { + IrClassDef, + IrExpr, + IrFunction, + IrLocal, + IrModule, + IrStmt, + IrType, + SrcLoc, +} from "./ir.js"; +import { + everyExprChild, + everyStmtChild, + everyStmtList, + everyTypeChild, + transformStmtList, +} from "./traverse.js"; + +/** A class whose instances may hold an inline array field. */ +export interface InlineArrayClass { + /** The class that declares the field (the hierarchy's subtree root). */ + readonly declaringClass: string; + readonly field: string; + /** The field's index in the class's fields: the same in every class of the + * subtree, whose field lists extend their bases'. */ + readonly fieldIndex: number; + /** The class of the elements. */ + readonly elemClass: string; + /** The index in a `new` expression's `args` of the argument the field is + * initialized from. */ + readonly lengthArg: number; +} + +/** Why a contract entry keeps an ordinary array field. */ +export interface InlineArrayRejection { + readonly declaringClass: string; + readonly field: string; + readonly reason: string; +} + +export interface InlineArrayPlan { + /** By class name, for the declaring classes and their subclasses. */ + readonly classes: ReadonlyMap; + readonly rejected: readonly InlineArrayRejection[]; +} + +export const NO_INLINE_ARRAYS: InlineArrayPlan = { classes: new Map(), rejected: [] }; + +/** The entries of `IrContracts.inlineArrays` (`${class}\0${field}`) whose + * hierarchies the IR admits, given which element classes the backend can + * store as 32-bit region references (`compactElem`). */ +export function planInlineArrays( + mod: IrModule, + compactElem: (className: string) => boolean, +): InlineArrayPlan { + const entries = mod.contracts?.inlineArrays ?? []; + if (entries.length === 0) return NO_INLINE_ARRAYS; + const classes = new Map(); + const children = new Map(); + for (const cls of mod.classes ?? []) { + classes.set(cls.name, cls); + if (cls.base !== undefined) { + const list = children.get(cls.base); + if (list === undefined) children.set(cls.base, [cls.name]); + else list.push(cls.name); + } + } + const functions = new Map(); + for (const fn of mod.functions) functions.set(fn.name, fn); + const planned = new Map(); + const rejected: InlineArrayRejection[] = []; + for (const entry of entries) { + const at = entry.indexOf("\0"); + const declaringClass = entry.slice(0, at); + const field = entry.slice(at + 1); + const reject = (reason: string): void => { + rejected.push({ declaringClass, field, reason }); + }; + const decl = classes.get(declaringClass); + if (decl === undefined) { + reject("the class is not in the program"); + continue; + } + const fieldIndex = decl.fields.findIndex((f) => f.name === field); + if (fieldIndex < 0) { + reject("the class has no such field"); + continue; + } + const base = decl.base === undefined ? undefined : classes.get(decl.base); + if (base !== undefined && base.fields.some((f) => f.name === field)) { + reject("the field is declared by a base class"); + continue; + } + const type = decl.fields[fieldIndex]!.type; + if (type.kind !== "array" || type.elem.kind !== "object" || !compactElem(type.elem.className)) { + reject("the field's elements are not ProgramLifetime instances"); + continue; + } + const elemClass = type.elem.className; + // The subtree, preorder. + const subtree: string[] = []; + const collect = (name: string): void => { + subtree.push(name); + for (const child of children.get(name) ?? []) collect(child); + }; + collect(declaringClass); + const members = new Set(subtree); + const lengthParams = new Map(); + let reason: string | null = null; + for (const name of subtree) { + const ctor = functions.get(`%${name}.constructor`); + if (ctor === undefined) { + reason = `${name} has no constructor`; + break; + } + const param = + name === declaringClass + ? writtenParam(ctor, members, field) + : forwardedParam(ctor, classes.get(name)!.base!, lengthParams); + if (typeof param === "string") { + reason = `${name}: ${param}`; + break; + } + lengthParams.set(name, param); + } + reason ??= otherUses(mod, members, field, declaringClass); + if (reason !== null) { + reject(reason); + continue; + } + for (const name of subtree) + planned.set(name, { + declaringClass, + field, + fieldIndex, + elemClass, + lengthArg: lengthParams.get(name)! - 1, + }); + } + return { classes: planned, rejected }; +} + +/** The index in `ctor.params` (0 is `this`) of the parameter the declaring + * constructor writes the field from, or why there is none. */ +function writtenParam( + ctor: IrFunction, + members: ReadonlySet, + field: string, +): number | string { + const self = ctor.params[0]?.localId; + const writes: (IrStmt & { kind: "fieldSet" })[] = []; + everyStmtList(ctor.body, { + expr: () => true, + stmt: (s) => { + if (s.kind === "fieldSet" && s.field === field && members.has(s.className)) writes.push(s); + return true; + }, + }); + if (writes.length !== 1) return `the constructor writes the field ${writes.length} times`; + const write = writes[0]!; + // The write is one of the body's own statements: it runs exactly once. + if (!ctor.body.includes(write)) return "the constructor writes the field conditionally"; + if (write.obj.kind !== "varRef" || write.obj.localId !== self) + return "the constructor writes the field of another object"; + return parameterOf(ctor, write.value); +} + +/** The index in `ctor.params` of the parameter a subclass constructor + * passes to its base's at the base's length parameter, or why there is none. */ +function forwardedParam( + ctor: IrFunction, + base: string, + lengthParams: ReadonlyMap, +): number | string { + const baseParam = lengthParams.get(base); + if (baseParam === undefined) return "its base class is not planned"; + const calls: (IrExpr & { kind: "call" })[] = []; + everyStmtList(ctor.body, { + expr: (e) => { + if (e.kind === "call" && e.callee === `%${base}.constructor`) calls.push(e); + return true; + }, + stmt: () => true, + }); + if (calls.length !== 1) return `the constructor calls its base's ${calls.length} times`; + const arg = calls[0]!.args[baseParam]; + if (arg === undefined) return "the base constructor call has no argument for the field"; + return parameterOf(ctor, arg); +} + +/** The index in `fn.params` of the parameter `value` reads, when the + * parameter is never written; otherwise why it is not one. */ +function parameterOf(fn: IrFunction, value: IrExpr): number | string { + if (value.kind !== "varRef") return "the field's value is not a constructor parameter"; + const index = fn.params.findIndex((p, i) => i > 0 && p.localId === value.localId); + if (index < 0) return "the field's value is not a constructor parameter"; + const local = fn.locals.find((l) => l.id === value.localId); + if (local === undefined || local.boxed === true || assigns(fn, value.localId)) + return "the constructor parameter is written"; + return index; +} + +function assigns(fn: IrFunction, localId: string): boolean { + return !everyStmtList(fn.body, { + expr: () => true, + stmt: (s) => !(s.kind === "assign" && s.localId === localId), + }); +} + +/** Why the rest of the program rules out the layout: another write of the + * field, construction through a class value, an instance of the subtree + * crossing into dynamic values (whose views of an instance read its fields + * by name), or a read of the field compared by identity. Null when nothing + * does. */ +function otherUses( + mod: IrModule, + members: ReadonlySet, + field: string, + declaringClass: string, +): string | null { + const reaches = (type: IrType): boolean => { + let found = false; + const visit = (t: IrType): boolean => { + if (t.kind === "object" && members.has(t.className)) found = true; + return !found && everyTypeChild(t, visit); + }; + visit(type); + return found; + }; + let reason: string | null = null; + for (const fn of mod.functions) { + const isDeclaringCtor = fn.name === `%${declaringClass}.constructor`; + everyStmtList(fn.body, { + expr: (e) => { + if (reason !== null) return false; + if (e.kind === "classRef" && members.has(e.className)) + reason = `${e.className} is used as a value`; + else if ((e.kind === "dynFrom" || e.kind === "caughtToDyn") && reaches(e.value.type)) + reason = "an instance becomes a dynamic value"; + else if (e.kind === "fieldIncDec" && e.field === field && members.has(e.className)) + reason = "the field is written outside the constructor"; + else if ( + e.kind === "bin" && + (e.op === "===" || e.op === "!==") && + (readsField(e.left, members, field) || readsField(e.right, members, field)) + ) + reason = "the field is compared by identity (its reads need not be the same array)"; + return reason === null; + }, + stmt: (s) => { + if (reason !== null) return false; + if ( + s.kind === "fieldSet" && + s.field === field && + members.has(s.className) && + !isDeclaringCtor + ) + reason = "the field is written outside the constructor"; + return reason === null; + }, + }); + if (reason !== null) return reason; + } + return null; +} + +function readsField(e: IrExpr, members: ReadonlySet, field: string): boolean { + return e.kind === "fieldGet" && e.field === field && members.has(e.className); +} + +/** Prepares the reads and constructions of planned classes (see the module + * comment). Returns the module itself when nothing changes. */ +export function inlineArrayReads(mod: IrModule, plan: InlineArrayPlan): IrModule { + if (plan.classes.size === 0) return mod; + let changed = false; + const functions = mod.functions.map((fn) => { + const next = rewriteFunction(fn, plan); + if (next !== fn) changed = true; + return next; + }); + return changed ? { ...mod, functions } : mod; +} + +function isInlineFieldRead(e: IrExpr, plan: InlineArrayPlan): e is IrExpr & { kind: "fieldGet" } { + if (e.kind !== "fieldGet") return false; + const cls = plan.classes.get(e.className); + return cls !== undefined && cls.field === e.field; +} + +/** Whether `parent` uses its child `child` (an array value) only by indexing + * it, reading its length or iterating it. */ +function isDirectUse(parent: IrExpr | IrStmt, child: IrExpr): boolean { + // Element reads and presence tests (a for...of over an array that may + // have holes tests each index first). + if (parent.kind === "arrayGet" || parent.kind === "arrayHas" || parent.kind === "arrayState") + return parent.arr === child; + if (parent.kind === "arrIntrinsic") + return parent.method === "length" && parent.receiver === child; + if (parent.kind === "forOf") return parent.iterable === child; + return false; +} + +function rewriteFunction(fn: IrFunction, plan: InlineArrayPlan): IrFunction { + const locals = new Map(); + for (const l of fn.locals) locals.set(l.id, l); + // Aliases: immutable locals declared once from an inline field read. + const declared = new Map(); + const aliasDecl = new Map(); + everyStmtList(fn.body, { + expr: () => true, + stmt: (s) => { + if (s.kind === "varDecl") { + declared.set(s.localId, (declared.get(s.localId) ?? 0) + 1); + const local = locals.get(s.localId); + if ( + s.init !== null && + isInlineFieldRead(s.init, plan) && + local !== undefined && + !local.mutable && + local.boxed !== true + ) + aliasDecl.set(s.localId, s); + } + return true; + }, + }); + for (const id of [...aliasDecl.keys()]) if (declared.get(id) !== 1) aliasDecl.delete(id); + // Every read of an alias must be a direct use. + if (aliasDecl.size > 0) { + const direct = new Map(); + const visitChildren = (parent: IrExpr | IrStmt, children: IrExpr[]): void => { + for (const c of children) + if (c.kind === "varRef" && aliasDecl.has(c.localId) && !isDirectUse(parent, c)) + direct.set(c.localId, false); + }; + everyStmtList(fn.body, { + expr: (e) => { + if (e.kind === "varRef") { + if (aliasDecl.has(e.localId) && !direct.has(e.localId)) direct.set(e.localId, true); + return true; + } + visitChildren(e, directChildren(e)); + return true; + }, + stmt: (s) => { + visitChildren(s, directStmtChildren(s)); + return true; + }, + }); + for (const [id, ok] of direct) if (!ok) aliasDecl.delete(id); + for (const id of [...aliasDecl.keys()]) if (!direct.has(id)) aliasDecl.delete(id); + } + const needsNew = !everyStmtList(fn.body, { + expr: (e) => !(e.kind === "new" && newNeedsLocals(e, plan)), + stmt: () => true, + }); + if (aliasDecl.size === 0 && !needsNew) return fn; + const newLocals: IrLocal[] = []; + let counter = 0; + const fresh = (name: string, type: IrType): string => { + let id: string; + do id = `%inl.${name}.${counter++}`; + while (locals.has(id)); + const local: IrLocal = { id, name: `%inl.${name}`, type, mutable: false }; + locals.set(id, local); + newLocals.push(local); + return id; + }; + // Each alias reads the field through a local holding the instance: the + // instance expression's own local when it is an immutable one, else a new + // one the declaration initializes. + const owners = new Map(); + for (const [id, decl] of aliasDecl) { + const read = decl.init as IrExpr & { kind: "fieldGet" }; + const obj = read.obj; + const own = obj.kind === "varRef" ? locals.get(obj.localId) : undefined; + const ownerId = + own !== undefined && !own.mutable && own.boxed !== true && !assigns(fn, own.id) + ? own.id + : fresh("owner", obj.type); + owners.set(id, { localId: ownerId, read }); + } + const fieldRead = (aliasId: string, loc: SrcLoc): IrExpr => { + const owner = owners.get(aliasId)!; + return { + ...owner.read, + obj: { kind: "varRef", localId: owner.localId, type: owner.read.obj.type, loc }, + loc, + }; + }; + const body = transformStmtList(fn.body, { + expr: (e) => { + if (e.kind === "varRef" && owners.has(e.localId)) return fieldRead(e.localId, e.loc); + if (e.kind === "new" && newNeedsLocals(e, plan)) return newThroughLocals(e, fresh); + return e; + }, + stmt: (s) => { + if (s.kind === "varDecl" && owners.has(s.localId)) { + const owner = owners.get(s.localId)!; + const obj = owner.read.obj; + // The instance's own local needs no declaration; otherwise the + // alias's declaration becomes the owner's. + if (obj.kind === "varRef" && obj.localId === owner.localId) + return { kind: "block", body: [], loc: s.loc }; + return { kind: "varDecl", localId: owner.localId, init: obj, loc: s.loc }; + } + return s; + }, + }); + return { ...fn, body, locals: [...fn.locals, ...newLocals] }; +} + +/** A `new` of a planned class whose length argument is not a local read. */ +function newNeedsLocals(e: IrExpr & { kind: "new" }, plan: InlineArrayPlan): boolean { + const cls = plan.classes.get(e.className); + if (cls === undefined) return false; + const arg = e.args[cls.lengthArg]; + return arg !== undefined && arg.kind !== "varRef"; +} + +/** `new C(a, b, ...)` as `(t0 = a, t1 = b, ..., new C(t0, t1, ...))`: the + * arguments evaluate in order before the allocation. */ +function newThroughLocals( + e: IrExpr & { kind: "new" }, + fresh: (name: string, type: IrType) => string, +): IrExpr { + const stmts: IrStmt[] = []; + const args = e.args.map((a) => { + const id = fresh("arg", a.type); + stmts.push({ kind: "varDecl", localId: id, init: a, loc: a.loc }); + return { kind: "varRef", localId: id, type: a.type, loc: a.loc } as IrExpr; + }); + return { kind: "seqExpr", stmts, result: { ...e, args }, type: e.type, loc: e.loc } as IrExpr; +} + +/** The expression children of `e` that are array operands the alias test + * looks at (everything else that reads an alias makes it escape). */ +function directChildren(e: IrExpr): IrExpr[] { + const out: IrExpr[] = []; + everyExprChildList(e, out); + return out; +} + +function directStmtChildren(s: IrStmt): IrExpr[] { + const out: IrExpr[] = []; + everyStmtChildList(s, out); + return out; +} + +// Immediate expression children, through the generic traversal: a child +// visitor that collects instead of recursing. +function everyExprChildList(e: IrExpr, out: IrExpr[]): void { + everyExprChild( + e, + (c) => { + out.push(c); + return true; + }, + () => true, + ); +} + +function everyStmtChildList(s: IrStmt, out: IrExpr[]): void { + everyStmtChild( + s, + (c) => { + out.push(c); + return true; + }, + () => true, + ); +} diff --git a/packages/compiler/src/ir/ir.ts b/packages/compiler/src/ir/ir.ts index 9e28f6f7a..10c44dd4f 100644 --- a/packages/compiler/src/ir/ir.ts +++ b/packages/compiler/src/ir/ir.ts @@ -1190,6 +1190,12 @@ export interface IrContracts { * static reference counting is elided, and the cycle analysis treats them * as leaves. */ programLifetime?: string[]; + /** Inline array fields, `${declaring class}\0${field}`: an array field of + * a ProgramLifetime class that its constructor writes once, from one of + * its parameters, and nothing else writes. Its elements are fixed for the + * instance's lifetime, so the backend may keep them in the instance itself + * (ir/inline-arrays.ts). */ + inlineArrays?: string[]; } export type IrFfiValueParamClass = diff --git a/packages/compiler/test/inline-array-fields.test.ts b/packages/compiler/test/inline-array-fields.test.ts new file mode 100644 index 000000000..f948bb5fe --- /dev/null +++ b/packages/compiler/test/inline-array-fields.test.ts @@ -0,0 +1,236 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, test } from "vitest"; +import { compile } from "../src/index.js"; + +// InlineArray fields (@scriptc/experimental, ir/inline-arrays.ts): the +// elements of a ProgramLifetime class's array field written once from a +// constructor parameter live in the instance, after its fixed fields. + +const header = ` +import type { ProgramLifetime } from "@scriptc/types"; +import type { InlineArray } from "@scriptc/experimental"; +`; + +async function build( + source: string, +): Promise<{ ok: boolean; codes: string[]; llvm: string; messages: string[] }> { + // Inside the workspace, so that the program resolves both packages. + const dir = await mkdtemp(join(import.meta.dirname, ".inline-array-")); + try { + const entry = join(dir, "main.ts"); + const outPath = join(dir, "main.ll"); + await writeFile(entry, header + source); + const result = await compile(entry, { outDir: dir, outPath, outputKind: "llvm" }); + const llvm = result.ok ? await readFile(outPath, "utf8") : ""; + return { + ok: result.ok, + codes: (result.diagnostics ?? []).map((d) => d.code), + messages: (result.diagnostics ?? []).map((d) => d.message), + llvm, + }; + } finally { + await rm(dir, { recursive: true, force: true }); + } +} + +const classes = ` +class Item implements ProgramLifetime { + readonly kind: number; + constructor(kind: number) { + this.kind = kind; + } +} +class List implements ProgramLifetime { + readonly pos: number; + readonly items: InlineArray; + constructor(pos: number, items: readonly Item[]) { + this.pos = pos; + this.items = items; + } +} +class Flagged extends List { + readonly flags: number; + constructor(pos: number, items: readonly Item[], flags: number) { + super(pos, items); + this.flags = flags; + } +} +`; + +const body = (fn: string, llvm: string): string => + new RegExp(`^define internal [^@]+@sc_bf_${fn}\\([^]*?^}`, "m").exec(llvm)?.[0] ?? ""; + +// The inline layout applies to executables whose compact instances live in +// the low region: Linux, 64-bit. +const lowRegion = + process.platform === "linux" && (process.arch === "x64" || process.arch === "arm64"); + +test.runIf(lowRegion)("the elements follow the fixed fields, which keep their count", async () => { + const { ok, llvm } = await build( + classes + + ` +const a = new List(1, [new Item(1), new Item(2)]); +const b = new Flagged(2, [new Item(3)], 7); +console.log(a.items.length, b.items.length, b.flags); +`, + ); + expect(ok).toBe(true); + // The field's slot is an i32: the header, pos and the count. + expect(llvm).toMatch(/^%sc_o_List = type \{ i32, i32, i32 \}/m); + expect(llvm).toMatch(/^%sc_o_Flagged = type \{ i32, i32, i32, i32 \}/m); + // `new` takes the count and allocates room for the 32-bit references + // after the largest fixed part in the subtree (Flagged's 16 bytes). + expect(llvm).toContain("define internal ptr @sc_new_List(i32 %inl.len)"); + expect(llvm).toContain("%inl.size = add nuw nsw i64 %inl.bytes, 16"); + // The constructor copies the argument's elements into the instance. + expect(llvm).toContain("call void @scr_arr_copy_cref_tail("); +}); + +test.runIf(lowRegion)("indexing, length and local aliases read the instance", async () => { + const { ok, llvm } = await build( + classes + + ` +function sum(list: List): number { + let s = 0; + const items = list.items; + for (let i = 0; i < items.length; i++) s += items[i]!.kind; + for (const item of list.items) s += item.kind; + return s + list.items[0]!.kind; +} +const list = new List(0, [new Item(1), new Item(2)]); +console.log(sum(list)); +`, + ); + expect(ok).toBe(true); + const sum = body("sum", llvm); + expect(sum).not.toBe(""); + // Elements are 32-bit references loaded from the instance; only the + // out-of-bounds path makes an array of them, for the runtime getter. + expect(sum).toMatch(/getelementptr inbounds i32, ptr %\S+, i64 %\S+\n\s*%\S+ = load i32/); + const hits = sum.split(/\n(?=\S)/).filter((block) => block.startsWith("inl.get.hit")); + expect(hits.length).toBeGreaterThan(0); + for (const block of hits) expect(block).not.toContain("scr_arr_view_cref"); + for (const line of sum.split("\n").filter((l) => l.includes("scr_arr_view_cref("))) + expect(line).not.toContain("scr_arr_get_ref"); +}); + +test.runIf(lowRegion)("for...of tests presence on the instance", async () => { + const { ok, llvm } = await build( + classes + + ` +// An array that may have holes: each loop over a list tests each index. +function holed(items: readonly Item[]): Item[] { + const out: Item[] = new Array(items.length); + for (let i = 0; i < items.length; i++) out[i] = items[i]!; + return out; +} +function kinds(list: List): number { + let s = 0; + for (const item of list.items) s += item.kind; + return s; +} +console.log(kinds(new List(0, holed([new Item(1), new Item(2)])))); +`, + ); + expect(ok).toBe(true); + const kinds = body("kinds", llvm); + expect(kinds).not.toBe(""); + expect(kinds).not.toContain("scr_arr_state"); + // Only an index outside the count makes an array, for the runtime getter. + for (const block of kinds.split(/\n(?=\S)/).filter((b) => b.includes("scr_arr_view_cref("))) + expect(block.startsWith("inl.get.miss")).toBe(true); +}); + +test.runIf(lowRegion)("a read that escapes gives an array of the elements", async () => { + const { ok, llvm } = await build( + classes + + ` +function count(items: readonly Item[]): number { + return items.length; +} +function use(list: List): number { + return count(list.items) + list.items.indexOf(list.items[0]!); +} +console.log(use(new List(0, [new Item(1)]))); +`, + ); + expect(ok).toBe(true); + expect(body("use", llvm)).toContain("call ptr @scr_arr_view_cref("); +}); + +test("an InlineArray field must keep its contract", async () => { + const twice = await build(` +class Item implements ProgramLifetime {} +class List implements ProgramLifetime { + readonly items: InlineArray; + constructor(items: readonly Item[], more: readonly Item[]) { + this.items = items; + if (more.length > 0) (this as { items: readonly Item[] }).items = more; + } +} +console.log(new List([new Item()], []).items.length); +`); + expect(twice.ok).toBe(false); + expect(twice.codes).toContain("SC7009"); + + const mortal = await build(` +class Item implements ProgramLifetime {} +class List { + readonly items: InlineArray; + constructor(items: readonly Item[]) { + this.items = items; + } +} +console.log(new List([new Item()]).items.length); +`); + expect(mortal.ok).toBe(false); + expect(mortal.codes).toContain("SC7009"); + expect(mortal.messages.join("\n")).toContain("not ProgramLifetime"); + + const computed = await build(` +class Item implements ProgramLifetime {} +class List implements ProgramLifetime { + readonly items: InlineArray; + constructor(n: number) { + const items: Item[] = []; + for (let i = 0; i < n; i++) items.push(new Item()); + this.items = items; + } +} +console.log(new List(2).items.length); +`); + expect(computed.ok).toBe(false); + expect(computed.codes).toContain("SC7009"); + expect(computed.messages.join("\n")).toContain("constructor parameter"); +}); + +test("a read of an InlineArray field is not compared by identity", async () => { + const result = await build(` +class Item implements ProgramLifetime {} +class List implements ProgramLifetime { + readonly items: InlineArray; + constructor(items: readonly Item[]) { + this.items = items; + } +} +const items = [new Item()]; +const list = new List(items); +console.log(list.items === items); +`); + expect(result.ok).toBe(false); + expect(result.codes).toContain("SC7009"); + expect(result.messages.join("\n")).toContain("compared by identity"); +}); + +test("InlineArray is checked on instance fields only", async () => { + const result = await build(` +class Item implements ProgramLifetime {} +function count(items: InlineArray): number { + return items.length; +} +console.log(count([new Item()])); +`); + expect(result.ok).toBe(false); + expect(result.codes).toContain("SC7006"); +}); diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/contract-inline-array-fields.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/contract-inline-array-fields.ts.json new file mode 100644 index 000000000..7611149a9 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/contract-inline-array-fields.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/contract-inline-array-fields.ts" + ], + "diags": [] +} diff --git a/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/contract-inline-array-publish.ts.json b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/contract-inline-array-publish.ts.json new file mode 100644 index 000000000..9870e8cc3 --- /dev/null +++ b/packages/compiler/test/ts7/baselines/order-parity/tests/corpus/contract-inline-array-publish.ts.json @@ -0,0 +1,6 @@ +{ + "order": [ + "/tests/corpus/contract-inline-array-publish.ts" + ], + "diags": [] +} diff --git a/packages/experimental/README.md b/packages/experimental/README.md index 5ae105aaa..6c2e0649b 100644 --- a/packages/experimental/README.md +++ b/packages/experimental/README.md @@ -116,3 +116,34 @@ scriptc requires the name to be a string literal, every construction of one name In scriptc builds every instance of a key class's hierarchy carries a 4-byte slot index, assigned by the first `set` for it and never reused. `get` reads the index, a page of the thread's family and the entry, with no lock, hash or atomic update. A family holds a reference to each of the thread's values and releases it when it is replaced or deleted, when its key is freed, and when the thread exits. Keys freed with a `LifetimeRegion` keep their values until the thread exits, a value that belongs to a region keeps the region allocated while the family holds it, and a value that refers to its own key keeps the key alive. Under Node.js each family is a `WeakMap` per thread. + +## InlineArray + +`InlineArray` marks an array field of a `ProgramLifetime` class (`@scriptc/types`) whose elements scriptc keeps in the instance itself, after its other fields, instead of in an array object of their own. It suits a program with millions of short arrays that never change once built, such as the children of each node of a syntax tree. + +```ts +import type { ProgramLifetime } from "@scriptc/types"; +import type { InlineArray } from "@scriptc/experimental"; + +class NodeList implements ProgramLifetime { + readonly pos: number; + readonly nodes: InlineArray; + constructor(pos: number, nodes: readonly Node[]) { + this.pos = pos; + this.nodes = nodes; + } +} + +function visit(list: NodeList): void { + const nodes = list.nodes; + for (let i = 0; i < nodes.length; i++) visitNode(nodes[i]!); +} +``` + +- `T` is a `ProgramLifetime` class, and the field is an instance field of a `ProgramLifetime` class. +- The constructor of the class that declares the field writes it once, from one of its parameters. A subclass constructor passes one of its own parameters there through `super(...)`. Nothing else writes the field. +- Indexing the field, reading its `length` and iterating it with `for...of` read the elements from the instance, directly or through a `const` local used only in those ways. Any other use, such as passing the field to a function or calling one of its methods, gets an array of the elements, which is frozen in scriptc builds. +- Two reads of the field need not give the same array, so a read of the field is never compared with `===` or `!==`. Compare a read kept in a local instead. +- Instances of the class are not constructed through a class value, and do not become dynamic values (for example through `console.log` or `JSON.stringify`). + +scriptc reports a field that does not keep these rules (SC7009). Under Node.js the field holds the array the constructor stored. In builds with sanitizers, in programs that use `LifetimeRegion`, and on platforms other than 64-bit Linux, the elements stay in an ordinary array object. diff --git a/packages/experimental/index.d.ts b/packages/experimental/index.d.ts index f8515f95f..1de6f8c92 100644 --- a/packages/experimental/index.d.ts +++ b/packages/experimental/index.d.ts @@ -74,3 +74,12 @@ export declare class ThreadSlots { /** Removes this thread's value for `key`. Returns whether there was one. */ delete(key: K): boolean; } + +/** An array field whose elements scriptc keeps in the instance itself + * (@scriptc/types ProgramLifetime classes): `readonly f: InlineArray`, + * where C is a ProgramLifetime class. The class's constructor writes the + * field once, from one of its parameters, and nothing else writes it. + * Reading the field gives an array of its elements, frozen in scriptc + * builds; two reads need not give the same array. Under Node.js it is the + * array the constructor stored. */ +export type InlineArray = readonly T[]; diff --git a/packages/experimental/index.js b/packages/experimental/index.js index e5b824ed0..f5445955a 100644 --- a/packages/experimental/index.js +++ b/packages/experimental/index.js @@ -7,7 +7,9 @@ // thread: Node.js threads share no objects, so every thread has its own // tables, and stored values are published (deep-frozen) as in scriptc. // ThreadSlots is a WeakMap per name and thread: module state is per thread -// under Node.js, so each thread sees only the values it set. +// under Node.js, so each thread sees only the values it set. InlineArray is +// a type (`import type`), with no value here: under Node.js the field holds +// the array its constructor stored. import { publish } from "@scriptc/threads"; diff --git a/packages/experimental/package.json b/packages/experimental/package.json index 4636be674..74e213448 100644 --- a/packages/experimental/package.json +++ b/packages/experimental/package.json @@ -1,7 +1,7 @@ { "name": "@scriptc/experimental", "version": "0.4.5", - "description": "Experimental scriptc APIs, with Node.js implementations: lifetime regions for ProgramLifetime objects, intern tables shared between threads, and per-thread slots on shared objects", + "description": "Experimental scriptc APIs, with Node.js implementations: lifetime regions for ProgramLifetime objects, intern tables shared between threads, per-thread slots on shared objects, and array fields kept in their instances", "license": "Apache-2.0", "homepage": "https://scriptc.dev", "repository": { diff --git a/packages/runtime/src/scr_array.c b/packages/runtime/src/scr_array.c index 0e4513c6f..7731c98ea 100644 --- a/packages/runtime/src/scr_array.c +++ b/packages/runtime/src/scr_array.c @@ -1694,6 +1694,56 @@ ScrArr *scr_arr_new_ops_r(const ScrArrOps *ops, size_t initial_cap) { ScrArr *scr_arr_new_ops_small_r(const ScrArrOps *ops) { return scr_arr_make_ops(ops, 0, true, true); } +/* An inline array field's elements as an array value (llvm/inline-arrays.ts). + * The field keeps `len` 32-bit references to compact instances at `slots`, in + * a program-lifetime instance, and never changes them. The header borrows + * them: SCR_ARR_INLINE_SLOTS keeps scr_arr_free_parts from freeing them and + * SCR_ARR_FROZEN keeps every write, and so every move, away from them. + * Element types whose ops lie past the registry keep their ops pointer in + * the word before the slots (scr_arr_ext), which belongs to the instance: + * those get a frozen copy instead. */ +ScrArr *scr_arr_view_cref(const ScrArrOps *ops, uint32_t *slots, uint32_t len) { + ScrArr *a = scr_arr_make_ops_kind(ops, 0, false, SCR_ELEM_CREF, false); + if (scr_arr_ops_id(ops) == SCR_ARR_OPS_OVERFLOW) { + if (len > 0) { + scr_arr_grow_dense_to(a, len, SCR_ARR_GROW_EXACT); + memcpy(a->data, slots, (size_t)len * sizeof(uint32_t)); + } + } else { + a->data = (uint64_t *)slots; + a->cap = len; + a->spare |= SCR_ARR_INLINE_SLOTS; + } + a->len = len; + a->filled = len; + a->spare |= SCR_ARR_FROZEN; + return a; +} + +/* The constructor's write of an inline array field: the `len` elements of + * `src`, an array of the field's element class, as 32-bit references at + * `slots`. `len` is the length the instance was allocated for, read from the + * same constructor argument, so another length, a hole or an element that + * is not a compact instance means the program broke the field's contract. */ +void scr_arr_copy_cref_tail(const ScrArr *src, uint32_t *slots, uint32_t len) { + if (src->len != len) + scr_trap("scriptc: an inline array field's array changed length during construction\n"); + if (len == 0) return; + if (src->filled < len) scr_trap("scriptc: an inline array field's array has holes\n"); + if (src->elem == SCR_ELEM_CREF) { + memcpy(slots, src->data, (size_t)len * sizeof(uint32_t)); + return; + } + if (src->elem != SCR_ELEM_REF) + scr_trap("scriptc: an inline array field's array holds other values\n"); + for (uint32_t i = 0; i < len; i++) { + uint64_t slot = src->data[i]; + if (slot == 0) + scr_trap("scriptc: an inline array field's array holds a missing element\n"); + slots[i] = scr_cref_encode(slot); + } +} + /* Frees an acyclic array whose elements are already released. */ static inline __attribute__((always_inline)) void scr_arr_free(ScrArr *a) { scr_arr_free_parts(a); diff --git a/packages/runtime/src/scr_array.h b/packages/runtime/src/scr_array.h index 86ef8d999..80aa43ea3 100644 --- a/packages/runtime/src/scr_array.h +++ b/packages/runtime/src/scr_array.h @@ -481,6 +481,10 @@ ScrArr *scr_arr_new(ScrElemKind elem, size_t initial_cap); /* returns +1 */ * header (the array then allocates with the collector header — see the * struct comment). Returns +1. */ ScrArr *scr_arr_new_ops(const ScrArrOps *ops, size_t initial_cap); +/* Inline array fields (scr_array.c): a frozen array over a field's 32-bit + * references, and the constructor's copy of an array into them. */ +ScrArr *scr_arr_view_cref(const ScrArrOps *ops, uint32_t *slots, uint32_t len); +void scr_arr_copy_cref_tail(const ScrArr *src, uint32_t *slots, uint32_t len); /* An empty array expected to receive a few elements: its first slots live * in the array's own block (one allocation; the block of a plain empty * array plus five 8-byte slots). Otherwise exactly scr_arr_new(elem, 0) and diff --git a/packages/types/README.md b/packages/types/README.md index e5d409860..b17e5c5fb 100644 --- a/packages/types/README.md +++ b/packages/types/README.md @@ -25,4 +25,4 @@ function isLetter(ch: number): NoThrow { - `Acyclic` on a class (`class C implements Acyclic`), interface (`interface I extends Acyclic`) or object type alias (`type T = Acyclic<{ ... }>`): no chain of references can lead from such an object back to itself. Its objects carry no cycle-collector header. - `ProgramLifetime` in a class's `implements` clause: instances of the class and its subclasses are never freed, so scriptc skips reference counting and cycle collection for them. Memory only: discarded instances are not reclaimed. -scriptc proves each contract or rejects the program with a diagnostic (SC7001-SC7008) that explains why it does not hold. A program that compiles behaves exactly as under Node.js. See https://scriptc.dev/docs/contracts. +scriptc proves each contract or rejects the program with a diagnostic (SC7001-SC7009) that explains why it does not hold. A program that compiles behaves exactly as under Node.js. See https://scriptc.dev/docs/contracts. diff --git a/tests/corpus/contract-inline-array-fields.ts b/tests/corpus/contract-inline-array-fields.ts new file mode 100644 index 000000000..ac86fa60b --- /dev/null +++ b/tests/corpus/contract-inline-array-fields.ts @@ -0,0 +1,177 @@ +import type { ProgramLifetime } from "@scriptc/types"; +import type { InlineArray } from "@scriptc/experimental"; + +// InlineArray fields (@scriptc/experimental): a ProgramLifetime class whose +// constructor writes an array field once from one of its parameters may keep +// the elements in the instance itself. Reads that index the field, read its +// length or iterate it load the elements from the instance; other reads see +// an array of them. Both runtimes print the same. + +abstract class Node implements ProgramLifetime { + readonly kind: string; + readonly pos: number; + constructor(kind: string, pos: number) { + this.kind = kind; + this.pos = pos; + } + abstract text(): string; +} + +class Leaf extends Node { + readonly value: number; + constructor(pos: number, value: number) { + super("leaf", pos); + this.value = value; + } + text(): string { + return String(this.value); + } +} + +class List implements ProgramLifetime { + readonly pos: number; + readonly end: number; + readonly nodes: InlineArray; + constructor(pos: number, end: number, nodes: readonly Node[]) { + this.pos = pos | 0; + this.end = end | 0; + this.nodes = nodes; + } + hasTrailingComma(): boolean { + const nodes = this.nodes; + return nodes.length > 0 && nodes[nodes.length - 1]!.pos < this.end; + } +} + +// A subclass with the inherited constructor. +class MissingList extends List {} + +// A subclass with fields of its own, which passes its parameter through. +class ModifierList extends List { + readonly flags: number; + readonly label: string; + constructor(pos: number, end: number, nodes: readonly Node[], flags: number, label: string) { + super(pos, end, nodes); + this.flags = flags; + this.label = label; + } +} + +class Group extends Node { + readonly children: List; + constructor(pos: number, children: List) { + super("group", pos); + this.children = children; + } + text(): string { + const parts: string[] = []; + for (const child of this.children.nodes) parts.push(child.text()); + return `(${parts.join(" ")})`; + } +} + +function leaves(from: number, count: number): Leaf[] { + const out: Leaf[] = []; + for (let i = 0; i < count; i++) out.push(new Leaf(from + i, (from + i) * 3)); + return out; +} + +// Direct reads: index, length, a local alias, for...of. +function sum(list: List): number { + let s = 0; + for (let i = 0; i < list.nodes.length; i++) s += (list.nodes[i] as Leaf).value; + const nodes = list.nodes; + for (let i = 0; i < nodes.length; i++) s += nodes[i]!.pos; + for (const node of list.nodes) s += node.pos * 2; + return s; +} + +// The field read as an array value. +function describe(nodes: readonly Node[]): string { + return nodes.map((n) => n.text()).join(","); +} + +function depth(node: Node): number { + if (!(node instanceof Group)) return 1; + let max = 0; + for (const child of node.children.nodes) max = Math.max(max, depth(child)); + return max + 1; +} + +const lengths = [0, 1, 2, 3, 5, 16, 17, 40]; +const lists: List[] = []; +for (const n of lengths) lists.push(new List(n, n * 10, leaves(n, n))); +for (const list of lists) + console.log( + list.nodes.length, + sum(list), + list.hasTrailingComma(), + describe(list.nodes).slice(0, 40), + ); + +// Subclasses share the elements' layout. +const missing = new MissingList(5, 5, []); +const modifiers = new ModifierList(1, 9, leaves(1, 4), 7, "mods"); +const lists2: List[] = [missing, modifiers]; +for (const list of lists2) + console.log( + list instanceof ModifierList ? "ModifierList" : list instanceof MissingList ? "MissingList" : "List", + list.nodes.length, + sum(list), + list.pos, + list.end, + ); +console.log(modifiers.flags, modifiers.label, modifiers.nodes[3]!.text()); + +// Array methods over the field's elements, and element identity. +const big = lists[lists.length - 1]!; +const third = big.nodes[2]!; +console.log(big.nodes.indexOf(third), big.nodes.includes(third), big.nodes[2] === third); +console.log(big.nodes.slice(3, 6).map((n) => n.pos).join("|")); +console.log(big.nodes.filter((n) => n.pos % 7 === 0).length); +console.log([...big.nodes].reverse()[0]!.pos, Array.from(big.nodes).length); +console.log(big.nodes.some((n) => n.pos === 77), big.nodes.find((n) => n.pos > 70)?.pos); + +// A list built from another list's elements, and from a slice of them. +const copy = new List(0, 0, big.nodes); +const part = new ModifierList(0, 0, big.nodes.slice(10, 13), 1, "part"); +console.log(copy.nodes.length, copy.nodes[39] === big.nodes[39], sum(part), part.nodes[0]!.pos); + +// Trees whose groups hold lists. +function tree(level: number, pos: number): Node { + if (level === 0) return new Leaf(pos, pos); + const kids: Node[] = []; + for (let i = 0; i < level + 1; i++) kids.push(tree(level - 1, pos * 4 + i)); + return new Group(pos, new List(pos, pos + kids.length, kids)); +} +const root = tree(4, 1); +console.log(depth(root), root.text().length, root.text().slice(0, 60)); + +// Loops over lists built from arrays that may have holes, in a function that +// only reads: each loop keeps its own array of the elements for the whole +// loop, also while an inner loop makes another. +function holed(nodes: readonly Node[]): Node[] { + const out: Node[] = new Array(nodes.length); + for (let i = 0; i < nodes.length; i++) out[i] = nodes[i]!; + return out; +} +function leafSum(group: Group): number { + let n = 0; + for (const child of group.children.nodes) + if (child instanceof Group) for (const leaf of child.children.nodes) n += leaf.pos; + return n; +} +const nested = new Group( + 0, + new List(0, 2, holed([new Group(1, new List(1, 1, holed(leaves(10, 3)))), new Group(2, new List(2, 2, holed(leaves(20, 3))))])), +); +console.log(leafSum(nested), leafSum(nested)); + +// Many lists: every read stays on the instance. +let total = 0; +for (let round = 0; round < 2000; round++) { + const list = new List(round, round + 3, leaves(round, round % 9)); + total += sum(list) + list.nodes.length; + if (list.nodes.length > 0) total += list.nodes.indexOf(list.nodes[list.nodes.length - 1]!); +} +console.log(total); diff --git a/tests/corpus/contract-inline-array-publish.ts b/tests/corpus/contract-inline-array-publish.ts new file mode 100644 index 000000000..29ee995c1 --- /dev/null +++ b/tests/corpus/contract-inline-array-publish.ts @@ -0,0 +1,75 @@ +import { Worker, isMainThread, parentPort, workerData } from "node:worker_threads"; +import { publish } from "@scriptc/threads"; +import type { ProgramLifetime } from "@scriptc/types"; +import type { InlineArray } from "@scriptc/experimental"; + +// InlineArray fields of ProgramLifetime instances published with +// @scriptc/threads. scriptc shares the published graph by reference, with +// the elements in the instances; Node.js clones it for the worker. The +// worker reads fields only (a Node.js clone has no prototype), so both +// runtimes print the same. + +class Node implements ProgramLifetime { + readonly kind: string; + readonly pos: number; + readonly children: NodeList; + constructor(kind: string, pos: number, children: NodeList) { + this.kind = kind; + this.pos = pos; + this.children = children; + } +} + +class NodeList implements ProgramLifetime { + readonly pos: number; + readonly nodes: InlineArray; + constructor(pos: number, nodes: readonly Node[]) { + this.pos = pos; + this.nodes = nodes; + } +} + +class Modifiers extends NodeList { + readonly flags: number; + constructor(pos: number, nodes: readonly Node[], flags: number) { + super(pos, nodes); + this.flags = flags; + } +} + +const none = new NodeList(-1, []); + +function tree(level: number, pos: number): Node { + const kids: Node[] = []; + if (level > 0) for (let i = 0; i < level + 1; i++) kids.push(tree(level - 1, pos * 4 + i)); + const list = level % 2 === 0 ? new Modifiers(pos, kids, level) : new NodeList(pos, kids); + return new Node(level === 0 ? "leaf" : "group", pos, kids.length === 0 ? none : list); +} + +function summary(node: Node): string { + const nodes = node.children.nodes; + if (nodes.length === 0) return `${node.kind}@${node.pos}`; + const parts: string[] = []; + for (let i = 0; i < nodes.length; i++) parts.push(summary(nodes[i]!)); + return `${node.kind}@${node.pos}[${parts.join(" ")}]`; +} + +function count(node: Node): number { + let n = 1; + for (const child of node.children.nodes) n += count(child); + return n; +} + +if (isMainThread) { + const root = publish(tree(4, 1)); + console.log("main", count(root), summary(root).length, summary(root).slice(0, 80)); + const worker = new Worker(new URL(import.meta.url), { workerData: root }); + worker.on("message", (line: string) => console.log(line)); + worker.on("exit", (code) => console.log("worker exit", code)); +} else { + const root = workerData as Node; + const first = root.children.nodes[0]!; + parentPort!.postMessage( + `worker ${count(root)} ${summary(root).length} ${first.children.nodes.length} ${summary(first).slice(0, 60)}`, + ); +} diff --git a/tests/harness/corpus-costs/contract-inline-array-fields.ts.json b/tests/harness/corpus-costs/contract-inline-array-fields.ts.json new file mode 100644 index 000000000..4631d3c80 --- /dev/null +++ b/tests/harness/corpus-costs/contract-inline-array-fields.ts.json @@ -0,0 +1 @@ +{ "plain": 7100, "san": 5100 } diff --git a/tests/harness/corpus-costs/contract-inline-array-publish.ts.json b/tests/harness/corpus-costs/contract-inline-array-publish.ts.json new file mode 100644 index 000000000..d4c74e30f --- /dev/null +++ b/tests/harness/corpus-costs/contract-inline-array-publish.ts.json @@ -0,0 +1 @@ +{ "plain": 4500, "san": 3900 } diff --git a/tests/harness/wasi-differential.test.ts b/tests/harness/wasi-differential.test.ts index 68d4b1807..e553cd132 100644 --- a/tests/harness/wasi-differential.test.ts +++ b/tests/harness/wasi-differential.test.ts @@ -313,6 +313,8 @@ describe.skipIf(!zigOnPath())("wasm32-wasi differential", () => { // ThreadSlots blocks and pages hold 32-bit pointers. "5727-thread-slots-basics.ts", "5729-thread-slots-values.ts", + // InlineArray fields stay ordinary arrays with 32-bit pointers. + "contract-inline-array-fields.ts", // Weak-collection hashes mix 32-bit pointers in 64 bits. "4591-weak-keys-union-records.ts", "7742-weak-collections-many-keys.ts",