diff --git a/docs/content/docs/cli.mdx b/docs/content/docs/cli.mdx
index f08ec558a..eb5277533 100644
--- a/docs/content/docs/cli.mdx
+++ b/docs/content/docs/cli.mdx
@@ -54,7 +54,7 @@ fib.s
Different output kinds accumulate in `.scriptc/`; rebuilding a kind updates its file.
-The IR and LLVM source outputs require only Node. On supported macOS, Linux, and Windows hosts, assembly and object outputs use the matching native helper installed with scriptc and do not invoke an external compiler, archiver, linker, or SDK. The object is a relocatable program object with undefined scr_* runtime symbols and a required scr_runtime_abi_v9 marker, not a standalone library. External consumption is experimental and requires the exact runtime version reported by --print=native-link-info. That option still writes the object, performs no link, and prints a versioned JSON recipe with the target, main entry, installed precompiled runtime pack, FFI inputs, and system libraries. It never reports private scriptc cache paths. See Native Program Objects for complete C-driver and direct-linker examples. --emit=exe is the default. LLVM builds emit the program object through the helper and link precompiled runtime objects; runtime development with AddressSanitizer compiles an instrumented C runtime.
+The IR and LLVM source outputs require only Node. On supported macOS, Linux, and Windows hosts, assembly and object outputs use the matching native helper installed with scriptc and do not invoke an external compiler, archiver, linker, or SDK. The object is a relocatable program object with undefined scr_* runtime symbols and a required scr_runtime_abi_v12 marker, not a standalone library. External consumption is experimental and requires the exact runtime version reported by --print=native-link-info. That option still writes the object, performs no link, and prints a versioned JSON recipe with the target, main entry, installed precompiled runtime pack, FFI inputs, and system libraries. It never reports private scriptc cache paths. See Native Program Objects for complete C-driver and direct-linker examples. --emit=exe is the default. LLVM builds emit the program object through the helper and link precompiled runtime objects; runtime development with AddressSanitizer compiles an instrumented C runtime.
## scriptc run
diff --git a/docs/content/docs/how-it-works.mdx b/docs/content/docs/how-it-works.mdx
index a6b804097..f0d5efabb 100644
--- a/docs/content/docs/how-it-works.mdx
+++ b/docs/content/docs/how-it-works.mdx
@@ -68,7 +68,7 @@ Without `-o`, artifacts are written to `.scriptc/` next to the input. Different
### External linking
-Program objects define `main` and leave selected `scr_*` runtime functions undefined. A strong reference to `scr_runtime_abi_v9` checks runtime compatibility at link time.
+Program objects define `main` and leave selected `scr_*` runtime functions undefined. A strong reference to `scr_runtime_abi_v12` checks runtime compatibility at link time.
`--print=native-link-info` emits the program object and a JSON recipe containing the runtime pack and link order for external builds. This object ABI is experimental and requires an exact runtime version. See [Native Program Objects](/docs/native-objects) for examples.
diff --git a/docs/content/docs/native-objects.mdx b/docs/content/docs/native-objects.mdx
index 05c98bc3d..609065160 100644
--- a/docs/content/docs/native-objects.mdx
+++ b/docs/content/docs/native-objects.mdx
@@ -8,7 +8,7 @@ title: "Native Program Objects"
The external object ABI is **experimental**. Its `scr_*` function and data surface may change before 1.0, so consumers must use the exact runtime-pack version reported by the same compiler installation. This is stricter than semver compatibility.
-The object intentionally leaves its selected runtime symbols undefined. It also holds a strong reference to `scr_runtime_abi_v9`, which the matching runtime defines. Linking an object against a runtime with another ABI marker fails at link time with the missing versioned symbol; it cannot become a latent runtime incompatibility.
+The object intentionally leaves its selected runtime symbols undefined. It also holds a strong reference to `scr_runtime_abi_v12`, which the matching runtime defines. Linking an object against a runtime with another ABI marker fails at link time with the missing versioned symbol; it cannot become a latent runtime incompatibility.
## Machine-readable link information
diff --git a/examples/native-object/README.md b/examples/native-object/README.md
index cb90d8299..486612a0e 100644
--- a/examples/native-object/README.md
+++ b/examples/native-object/README.md
@@ -15,4 +15,4 @@ $ ./app-ld
`cc` uses the C compiler as a linker driver. `ld` links the same precompiled runtime objects directly with the selected Apple SDK. The object defines `main`; it is a complete program object, not a library to load into another process. Use `scriptc build --lib --profile ...` for a host-callable static library.
-The external object ABI is experimental. Always consume the runtime pack at the exact `runtime_pack.version` reported by the same scriptc installation. The object requires `scr_runtime_abi_v9`, so a mismatched runtime fails during the link instead of starting with an incompatible ABI.
+The external object ABI is experimental. Always consume the runtime pack at the exact `runtime_pack.version` reported by the same scriptc installation. The object requires `scr_runtime_abi_v12`, so a mismatched runtime fails during the link instead of starting with an incompatible ABI.
diff --git a/packages/cli/README.md b/packages/cli/README.md
index 4f83b694a..b4876f9b0 100644
--- a/packages/cli/README.md
+++ b/packages/cli/README.md
@@ -34,7 +34,7 @@ Builds use a bounded persistent cache. Unchanged source can reuse the validated
- `scriptc run ` — compile and run
- `scriptc coverage ` — what compiles statically, and why the rest doesn't
-`scriptc build app.ts --emit=ir|llvm|asm|obj` selects serialized typed IR, textual LLVM IR, target assembly, or a relocatable program object as the one primary artifact. `--emit=exe` is the default. Assembly/object emission uses the matching helper on supported macOS, Linux, and Windows hosts (and produces WASI artifacts when that target is selected). Objects retain undefined `scr_*` runtime references plus the `scr_runtime_abi_v9` compatibility marker; they are not library archives. External consumption is experimental and requires the exact matching runtime. `scriptc build app.ts --print=native-link-info -o app.o` prints the versioned JSON target/runtime/link recipe without performing a link. `--emit=asm|obj --sanitize` is rejected until ASan pipeline parity is available.
+`scriptc build app.ts --emit=ir|llvm|asm|obj` selects serialized typed IR, textual LLVM IR, target assembly, or a relocatable program object as the one primary artifact. `--emit=exe` is the default. Assembly/object emission uses the matching helper on supported macOS, Linux, and Windows hosts (and produces WASI artifacts when that target is selected). Objects retain undefined `scr_*` runtime references plus the `scr_runtime_abi_v12` compatibility marker; they are not library archives. External consumption is experimental and requires the exact matching runtime. `scriptc build app.ts --print=native-link-info -o app.o` prints the versioned JSON target/runtime/link recipe without performing a link. `--emit=asm|obj --sanitize` is rejected until ASan pipeline parity is available.
For embedder-hosted modules that are not installed npm packages, coverage can map an exact bare specifier to a local declaration with repeatable `--external-types ` options. This is analysis-only: the types unblock application measurement, while runtime module uses remain reported as blockers.
diff --git a/packages/cli/test/native-link-info.test.ts b/packages/cli/test/native-link-info.test.ts
index 9470f936f..ad7791c66 100644
--- a/packages/cli/test/native-link-info.test.ts
+++ b/packages/cli/test/native-link-info.test.ts
@@ -72,7 +72,7 @@ describe.runIf(supported)("macOS arm64 native link info", () => {
minimum_os: "14.0",
},
program: { object, entry_symbol: "main" },
- runtime_abi: { version: 9, marker: "scr_runtime_abi_v9" },
+ runtime_abi: { version: 12, marker: "scr_runtime_abi_v12" },
runtime_pack: {
kind: "precompiled",
package: "@scriptc/runtime-darwin-arm64",
@@ -165,6 +165,6 @@ describe.runIf(supported)("macOS arm64 native link info", () => {
() => null,
(failure: { stderr?: string }) => failure,
);
- expect(error?.stderr).toContain("scr_runtime_abi_v9");
+ expect(error?.stderr).toContain("scr_runtime_abi_v12");
});
});
diff --git a/packages/compiler/src/backend/llvm/alloc.test.ts b/packages/compiler/src/backend/llvm/alloc.test.ts
index 6b67b6825..8f2458e14 100644
--- a/packages/compiler/src/backend/llvm/alloc.test.ts
+++ b/packages/compiler/src/backend/llvm/alloc.test.ts
@@ -53,8 +53,8 @@ describe("emitted object allocation", () => {
"ok:",
" call void @scr_obj_alloc_note()",
]);
- expect(emitObjectAlloc(h, "24", { trace: "@t", free: "@f" })).toEqual([
- " %o = call ptr @scr_cyc_alloc(i64 24, ptr @t, ptr @f)",
+ expect(emitObjectAlloc(h, "24", { type: "@ty" })).toEqual([
+ " %o = call ptr @scr_cyc_alloc(i64 24, ptr @ty)",
" call void @scr_obj_alloc_note()",
]);
expect(emitObjectFree(h, true)).toEqual([
@@ -83,10 +83,10 @@ describe("emitted object allocation", () => {
expect(plain.filter((l) => l.includes("@scr_obj_alloc_note")).length).toBe(1);
expect(plain.at(-1)).toBe(" %o = phi ptr [ %sa.blk, %sa.fast ], [ %sa.s, %sa.noted ]");
- const cyc = emitObjectAlloc(h, "40", { trace: "@t", free: "@f" });
+ const cyc = emitObjectAlloc(h, "40", { type: "@ty" });
expectSingleDefinitions(cyc);
- expect(cyc).toContain(" %sa.n = add i64 40, 32");
- expect(cyc).toContain(" %sa.s = call ptr @scr_cyc_alloc(i64 40, ptr @t, ptr @f)");
+ expect(cyc).toContain(" %sa.n = add i64 40, 16");
+ expect(cyc).toContain(" %sa.s = call ptr @scr_cyc_alloc(i64 40, ptr @ty)");
expect(cyc).toContain(" store i64 %sa.live1, ptr @scr_cyc_live");
expect(h.decls).toContain(
"@scr_sa = external global { i64, i64, i32, [32 x ptr], [32 x ptr], [32 x ptr] }",
diff --git a/packages/compiler/src/backend/llvm/alloc.ts b/packages/compiler/src/backend/llvm/alloc.ts
index 7529c5669..6ea2a2b91 100644
--- a/packages/compiler/src/backend/llvm/alloc.ts
+++ b/packages/compiler/src/backend/llvm/alloc.ts
@@ -108,9 +108,11 @@ const SA_BUMP = 280;
const SA_LIM = 536;
const SA_STEP_SHIFT = 4; // 16-byte classes
const SA_MAX = 512;
-/** sizeof(ScrCycHdr) on 64-bit targets; trace and free_fn are its first
- * two words, and every other header word starts zero. */
-const CYC_HDR = 32;
+/** sizeof(ScrCycHdr); on 64-bit targets the type descriptor is its first
+ * word, and every other header word starts zero. */
+const CYC_HDR = 16;
+/** ScrCycHdr.gen, an i8 at this header offset (scr_runtime.h asserts it). */
+const CYC_GEN = 9;
function declareState(host: AllocHost): void {
if (host.threadLocalAlloc !== true) host.declare(`@scr_sa = external global ${SA_TYPE}`);
@@ -121,10 +123,24 @@ function declareGlobal(host: AllocHost, decl: string): void {
if (host.threadLocalAlloc !== true) host.declare(decl);
}
-/** A collector header to install: the object's trace and teardown. */
+/** A collector header to install: the object type's constant ScrCycType
+ * descriptor (its trace and teardown; see cycTypeDescriptor). */
export interface CycHeader {
- trace: string;
- free: string;
+ type: string;
+}
+
+/** The first collector statistics index of an emitted ScrCycType
+ * descriptor; the runtime's own families use the ones below
+ * (SCR_CYC_KIND_EMITTED in scr_runtime.h). */
+export const FIRST_EMITTED_CYC_KIND = 32;
+
+/** The definition of a shape's ScrCycType descriptor `{ trace, free_fn,
+ * kind }`, referenced by every header of that shape. `kind` indexes the
+ * collector's per-type statistics (scr_runtime.h); emitted shapes number
+ * themselves from FIRST_EMITTED_CYC_KIND (ShapeHost.nextCycKind) and 0
+ * keeps none. */
+export function cycTypeDescriptor(name: string, trace: string, free: string, kind = 0): string {
+ return `${name} = internal constant { ptr, ptr, i32 } { ptr ${trace}, ptr ${free}, i32 ${kind} }`;
}
/**
@@ -157,9 +173,9 @@ function allocLines(
const S = host.sizeType;
if (host.inlineAlloc !== true) {
if (cyc) {
- host.declare(`declare ptr @scr_cyc_alloc(${S}, ptr, ptr)`);
+ host.declare(`declare ptr @scr_cyc_alloc(${S}, ptr)`);
return [
- ` %o = call ptr @scr_cyc_alloc(${S} ${size}, ptr ${cyc.trace}, ptr ${cyc.free})`,
+ ` %o = call ptr @scr_cyc_alloc(${S} ${size}, ptr ${cyc.type})`,
` call void @scr_obj_alloc_note()`,
];
}
@@ -178,8 +194,8 @@ function allocLines(
}
declareState(host);
// Bytes of the block that the fast path itself stores after the clear:
- // the header's trace/free_fn words, or the object's leading words.
- const keep = cyc ? 16 : stored;
+ // the header's descriptor word, or the object's leading words.
+ const keep = cyc ? 8 : stored;
const lines = [
` %sa.n = add i64 ${size}, ${cyc ? CYC_HDR : 0}`,
` %sa.small = icmp ule i64 %sa.n, ${SA_MAX}`,
@@ -223,21 +239,19 @@ function allocLines(
];
host.declare(`declare void @llvm.memset.p0.i64(ptr, i8, i64, i1)`);
if (cyc) {
- host.declare(`declare ptr @scr_cyc_alloc(${S}, ptr, ptr)`);
+ host.declare(`declare ptr @scr_cyc_alloc(${S}, ptr)`);
declareGlobal(host, `@scr_cyc_live = external global i64`);
lines.push(
`sa.fast:`,
` %sa.blk = phi ptr [ %sa.b, %sa.pop ], [ %sa.p, %sa.carve ]`,
- ` store ptr ${cyc.trace}, ptr %sa.blk ; ScrCycHdr.trace`,
- ` %sa.ffp = getelementptr inbounds i8, ptr %sa.blk, i64 8`,
- ` store ptr ${cyc.free}, ptr %sa.ffp ; ScrCycHdr.free_fn`,
+ ` store ptr ${cyc.type}, ptr %sa.blk ; ScrCycHdr.type`,
` %sa.live = load i64, ptr @scr_cyc_live`,
` %sa.live1 = add i64 %sa.live, 1`,
` store i64 %sa.live1, ptr @scr_cyc_live`,
` %sa.obj = getelementptr inbounds i8, ptr %sa.blk, i64 ${CYC_HDR}`,
` br label %sa.done`,
`sa.slow:`,
- ` %sa.s = call ptr @scr_cyc_alloc(i64 ${size}, ptr ${cyc.trace}, ptr ${cyc.free})`,
+ ` %sa.s = call ptr @scr_cyc_alloc(i64 ${size}, ptr ${cyc.type})`,
` call void @scr_obj_alloc_note()`,
` br label %sa.done`,
`sa.done:`,
@@ -315,9 +329,9 @@ function freeLines(host: AllocHost, cyc: boolean): string[] {
` %sf.live1 = sub i64 %sf.live, 1`,
` store i64 %sf.live1, ptr @scr_cyc_live`,
` %sf.blk = getelementptr inbounds i8, ptr %o, i64 -${CYC_HDR}`,
- ` %sf.genp = getelementptr inbounds i8, ptr %sf.blk, i64 22`,
- ` %sf.gen = load i16, ptr %sf.genp`,
- ` %sf.old = icmp eq i16 %sf.gen, 2`,
+ ` %sf.genp = getelementptr inbounds i8, ptr %sf.blk, i64 ${CYC_GEN}`,
+ ` %sf.gen = load i8, ptr %sf.genp`,
+ ` %sf.old = icmp eq i8 %sf.gen, 2`,
` br i1 %sf.old, label %sf.oldcount, label %sf.accounted`,
`sf.oldcount:`,
` %sf.freed = load i64, ptr @scr_cyc_old_freed`,
diff --git a/packages/compiler/src/backend/llvm/classes.ts b/packages/compiler/src/backend/llvm/classes.ts
index 64baa829d..9f7794925 100644
--- a/packages/compiler/src/backend/llvm/classes.ts
+++ b/packages/compiler/src/backend/llvm/classes.ts
@@ -26,12 +26,13 @@ import {
mangleVtInstance,
mangleVtStruct,
} from "../mangle.js";
-import { emitObjectAlloc, emitObjectFree } from "./alloc.js";
+import { cycTypeDescriptor, emitObjectAlloc, emitObjectFree } from "./alloc.js";
import { llvmCommentText } from "./common.js";
import { NullableRefFields } from "./nullable-fields.js";
import type { Int32Slots } from "../../ir/int32-slots.js";
import {
cycleRootLines,
+ traceFieldVisitLines,
FN_ATTRS,
cycleDeadLines,
llFieldType,
@@ -591,15 +592,19 @@ export function emitClassShapes(
`define internal ptr @${mangleClassNew(cls.name)}() ${FN_ATTRS} { ; new ${cls.name}`,
`entry:`,
];
+ const cycType = `@${mangleClassTrace(cls.name)}.cyctype`;
+ if (traced)
+ defs.push(
+ cycTypeDescriptor(
+ cycType,
+ `@${mangleClassTrace(cls.name)}`,
+ `@${mangleClassGcFree(cls.name)}`,
+ host.nextCycKind?.() ?? 0,
+ ),
+ ``,
+ );
nw.push(
- ...emitObjectAlloc(
- host,
- sizeOf,
- traced
- ? { trace: `@${mangleClassTrace(cls.name)}`, free: `@${mangleClassGcFree(cls.name)}` }
- : null,
- meta.hierarchy ? 16 : 8,
- ),
+ ...emitObjectAlloc(host, sizeOf, traced ? { type: cycType } : null, meta.hierarchy ? 16 : 8),
);
nw.push(` store ${host.sizeType} 1, ptr %o`);
if (meta.hierarchy) {
@@ -642,11 +647,7 @@ export function emitClassShapes(
if (isStreamRooted)
tr.push(...stCall("trs", "scr_stream_st_trace", ", ptr %visit, ptr %ctx"));
tracedFields.forEach((f, i) => {
- tr.push(
- ` %f${i} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${f.index}`,
- ` %v${i} = load ptr, ptr %f${i}`,
- ` call void %visit(ptr %v${i}, ptr %ctx) ; ${llvmCommentText(f.name)}`,
- );
+ tr.push(...traceFieldVisitLines(struct, f.index, i, llvmCommentText(f.name)));
});
tr.push(` ret void`, `}`, ``);
defs.push(...tr);
diff --git a/packages/compiler/src/backend/llvm/dyn.ts b/packages/compiler/src/backend/llvm/dyn.ts
index 079ca898e..b474068cc 100644
--- a/packages/compiler/src/backend/llvm/dyn.ts
+++ b/packages/compiler/src/backend/llvm/dyn.ts
@@ -1201,12 +1201,6 @@ export class LlDyn {
{ index: 2, type: host.sizeType, name: "capacity" },
{ index: 7, type: "ptr" as const, name: "data" },
{ index: 8, type: "ptr" as const, name: "presence" },
- { index: 9, type: "ptr" as const, name: "sparse slots" },
- { index: 10, type: host.sizeType, name: "sparse length" },
- { index: 11, type: host.sizeType, name: "sparse capacity" },
- { index: 12, type: "ptr" as const, name: "numeric properties" },
- { index: 13, type: host.sizeType, name: "property length" },
- { index: 14, type: host.sizeType, name: "property capacity" },
];
members.forEach((member) => {
const cachedPtr = B.tmp();
@@ -1227,6 +1221,12 @@ export class LlDyn {
B.line(`store ${member.type} ${newValue}, ptr ${cachedPtr}`);
B.line(`store ${member.type} ${oldValue}, ptr ${checkedPtr}`);
});
+ if (t.kind !== "record") {
+ // Sparse slots and numeric properties live out of line; each
+ // array keeps its own match metadata.
+ host.declare("declare void @scr_arr_swap_indexed_ext(ptr, ptr)");
+ B.line(`call void @scr_arr_swap_indexed_ext(ptr ${cached}, ptr ${checked})`);
+ }
B.line(`call void ${releaseSym(host, t)}(ptr ${checked})`);
B.terminate(`ret ptr ${cached}`);
B.startBlock(lCache);
diff --git a/packages/compiler/src/backend/llvm/emitter.ts b/packages/compiler/src/backend/llvm/emitter.ts
index d9b8754c7..ff9f4cc6d 100644
--- a/packages/compiler/src/backend/llvm/emitter.ts
+++ b/packages/compiler/src/backend/llvm/emitter.ts
@@ -1,5 +1,5 @@
import { emitLlvmLayouts } from "./layouts.js";
-import { inlineAllocSupported } from "./alloc.js";
+import { FIRST_EMITTED_CYC_KIND, inlineAllocSupported } from "./alloc.js";
import { llvmBytes as llBytes } from "../literals.js";
import { InternalCompilerError } from "../../errors.js";
/** Lower typed IR to LLVM, using the runtime's C ABI.
@@ -522,7 +522,10 @@ export class LlEmitter {
private debugScope: string | null = null;
readonly sizeType: "i32" | "i64";
readonly ffiExtendNarrowIntegers: boolean;
- readonly cycleColorOffset: number;
+ readonly cycleBufferedOffset: number;
+ /** Emitted ScrCycType descriptors so far (their collector statistics
+ * indices; see ShapeHost.nextCycKind). */
+ private cycKinds = 0;
readonly wasi: boolean;
/** The target lacks a rounding instruction at its baseline CPU (x86-64
* before SSE4.1), so `llvm.trunc` would lower to a libm call: emit
@@ -859,10 +862,10 @@ export class LlEmitter {
this.runtimeAbiMarker = options.runtimeAbiMarker === true;
this.detachedExit = options.detachedExit === true;
this.rcHelpers = options.inlineRc === true ? new Set() : null;
- // ScrCycHdr.color stays 12 bytes behind a wasm32 object and 16 bytes
- // behind a 64-bit object. The wasm32 header pads before color so its
- // payload remains double-aligned without changing this ABI offset.
- this.cycleColorOffset = options.pointerBits === 32 ? 12 : 16;
+ // ScrCycHdr.buffered sits 6 bytes behind every object. The wasm32
+ // header pads before its descriptor so its payload remains
+ // double-aligned without changing this ABI offset.
+ this.cycleBufferedOffset = 6;
this.ffiCallbackAdapters = allocateFfiCallbackAdapters(mod.ffiImports ?? []);
this.ffiHasRetainedCallback = hasRetainedFfiCallback(mod.ffiImports ?? []);
this.ffiHasForeignCallback = hasForeignFfiCallback(mod.ffiImports ?? []);
@@ -894,7 +897,10 @@ export class LlEmitter {
this.nullableFields = new NullableRefFields(mod.classes ?? [], this.boxedUnionsById);
// Optional array reads also cover nullable unions: emitLocalArrayRead
// then yields the (borrowed or owned) element pointer, not a stack box.
- this.optionalArrayReads = new OptionalArrayReads(this.fnByName, this.unionsById);
+ // Arrays of such a union are read the same way.
+ this.optionalArrayReads = new OptionalArrayReads(this.fnByName, this.unionsById, (id) =>
+ this.nullableUnions.has(id),
+ );
this.checkedNarrows = new CheckedNarrows(this.fnByName);
this.referenceEffects = new ReferenceEffects(
this.fnByName,
@@ -956,7 +962,7 @@ export class LlEmitter {
this.tracedUnions = traced.unions;
this.shapeHost = {
sizeType: this.sizeType,
- cycleColorOffset: this.cycleColorOffset,
+ cycleBufferedOffset: this.cycleBufferedOffset,
tracedShapes: this.tracedShapes,
tracedUnions: this.tracedUnions,
nullableUnions: this.nullableUnions,
@@ -973,6 +979,7 @@ export class LlEmitter {
// paths address its thread-local state.
threadLocalAlloc: mod.workers === true,
rcHelpers: this.rcHelpers,
+ nextCycKind: () => FIRST_EMITTED_CYC_KIND + this.cycKinds++,
unionsById: this.unionsById,
declare: (decl) => this.declare(decl),
needOom: () => this.needOom(),
@@ -1873,10 +1880,13 @@ export class LlEmitter {
`%ScrClosure = type { ${this.sizeType}, ptr, ${this.sizeType}, ptr, i32, i32, ptr }`,
`%ScrFfiTable = type { ptr, ${this.sizeType}, ${this.sizeType}, ptr, i8, ptr, ptr, ${this.sizeType}, ${this.sizeType}, ${this.sizeType}, ptr, ptr }`,
`%ScrRegex = type { ${this.sizeType}, ptr, ptr, ptr, double, ptr, ptr }`,
- // ScrArr mirrors scr_runtime.h field-for-field. Live dynamic stream
- // commits swap its mutable dense, sparse, presence, and property
- // storage while preserving the target object's identity.
- `%ScrArr = type { ${this.sizeType}, ${this.sizeType}, ${this.sizeType}, i32, ptr, ptr, ptr, ptr, ptr, ptr, ${this.sizeType}, ${this.sizeType}, ptr, ${this.sizeType}, ${this.sizeType}, ptr }`,
+ // ScrArr mirrors scr_runtime.h field-for-field: { rc, len, cap, elem,
+ // elem_retain, elem_release, elem_trace, data, present, ext }, where
+ // ext points at the out-of-line sparse, property and metadata storage
+ // (NULL until first used). Live dynamic stream commits swap its dense,
+ // presence and out-of-line storage while preserving the target
+ // object's identity.
+ `%ScrArr = type { ${this.sizeType}, ${this.sizeType}, ${this.sizeType}, i32, ptr, ptr, ptr, ptr, ptr, ptr }`,
// The ScrMap prefix { rc, key_kind, val_kind, val_retain, val_release,
// val_trace, key_retain, key_release, key_trace } — the inline RC
// fast paths test the two trace slots (shapes.ts).
@@ -2057,7 +2067,7 @@ export class LlEmitter {
out.push(
`@${inst.sym}_data = internal constant [${n} x ptr] [ ${inst.slots.map((s) => `ptr ${s}`).join(", ")} ]`,
`@${inst.sym}_present = internal constant [${n} x i8] ${present}`,
- `@${inst.sym} = internal global %ScrArr { ${this.sizeType} -1, ${this.sizeType} ${n}, ${this.sizeType} ${n}, i32 2, ptr null, ptr null, ptr null, ptr @${inst.sym}_data, ptr @${inst.sym}_present, ptr null, ${this.sizeType} 0, ${this.sizeType} 0, ptr null, ${this.sizeType} 0, ${this.sizeType} 0, ptr null }`,
+ `@${inst.sym} = internal global %ScrArr { ${this.sizeType} -1, ${this.sizeType} ${n}, ${this.sizeType} ${n}, i32 2, ptr null, ptr null, ptr null, ptr @${inst.sym}_data, ptr @${inst.sym}_present, ptr null }`,
);
}
if (this.templateStringsInstances.size > 0) out.push(``);
@@ -5129,12 +5139,14 @@ export class LlEmitter {
this.callLifetimes,
this.optionalArrayReads,
this.loopArrayBorrows,
+ this.debug === null ? this.referenceEffects : undefined,
);
this.callArrayReads = findCallArrayReads(
fn,
this.optionalArrayReads,
this.referenceEffects.functions,
this.callLifetimes,
+ this.debug === null ? this.referenceEffects : undefined,
);
this.scalarStringSlices =
this.debug === null ? findScalarStringSlices(fn, this.callLifetimes) : new Map();
diff --git a/packages/compiler/src/backend/llvm/expr-control.ts b/packages/compiler/src/backend/llvm/expr-control.ts
index 634671057..b425862f7 100644
--- a/packages/compiler/src/backend/llvm/expr-control.ts
+++ b/packages/compiler/src/backend/llvm/expr-control.ts
@@ -290,9 +290,18 @@ export function emitControlExpr(
isRefCounted(narrowed) &&
host.canBorrowReceiver(e.receiver) &&
host.referenceEffects.preserves(e.body);
+ // An optional array read of a nullable-pointer union is the element
+ // pointer itself, read inline and owned by the statement frame until
+ // the body finishes; the bind slot borrows it like an emitExpr value.
+ const nullableRead =
+ !borrowReceiver &&
+ !multiple &&
+ host.nullableUnions.has(def.id) &&
+ host.optionalArrayReads.get(e.receiver) !== null;
// A single present arm is only tested and extracted (+1) before the
// body runs, so a fresh receiver box can stay on the stack.
- const stacked = !borrowReceiver && !multiple && host.canStackReceiver(e.receiver);
+ const stacked =
+ !borrowReceiver && !nullableRead && !multiple && host.canStackReceiver(e.receiver);
let r: LlValue;
const nullableField = borrowReceiver ? host.nullableFieldGet(e.receiver) : null;
if (nullableField !== null && e.receiver.kind === "fieldGet") {
@@ -307,7 +316,7 @@ export function emitControlExpr(
name: host.nullableUnions.get(def.id) ? p : host.stackNullableBox(p, nullableField),
type: e.receiver.type,
};
- } else if (borrowReceiver || stacked) r = host.emitReadReceiver(e.receiver);
+ } else if (borrowReceiver || stacked || nullableRead) r = host.emitReadReceiver(e.receiver);
else r = host.emitExpr(e.receiver);
// The bind slot borrows a ref payload whenever something else keeps it
// alive until the body finishes: the borrowed projection above, or the
diff --git a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts
index d415ee091..d6cd7ff4c 100644
--- a/packages/compiler/src/backend/llvm/expr-stream-bridges.ts
+++ b/packages/compiler/src/backend/llvm/expr-stream-bridges.ts
@@ -178,13 +178,7 @@ export function streamTypedRefCommitAdapter(
{ name: "cap", index: 2, type: host.sizeType },
{ name: "data", index: 7, type: "ptr" },
{ name: "present", index: 8, type: "ptr" },
- { name: "sparse", index: 9, type: "ptr" },
- { name: "sparse_len", index: 10, type: host.sizeType },
- { name: "sparse_cap", index: 11, type: host.sizeType },
- { name: "props", index: 12, type: "ptr" },
- { name: "prop_len", index: 13, type: host.sizeType },
- { name: "prop_cap", index: 14, type: host.sizeType },
- { name: "metadata", index: 15, type: "ptr" },
+ { name: "ext", index: 9, type: "ptr" },
];
for (const member of storageMembers) {
lines.push(
diff --git a/packages/compiler/src/backend/llvm/local-array-reads.ts b/packages/compiler/src/backend/llvm/local-array-reads.ts
index a438a2c95..abe88c9d1 100644
--- a/packages/compiler/src/backend/llvm/local-array-reads.ts
+++ b/packages/compiler/src/backend/llvm/local-array-reads.ts
@@ -22,8 +22,17 @@ export interface LocalArrayRead {
presentTag: number;
missingTag: number;
borrow?: boolean;
+ /** The array's elements are themselves the read's `T | undefined`
+ * nullable-pointer union, so a present slot may hold NULL (undefined):
+ * owned copies retain and release through the union's NULL-tolerant
+ * entry points. */
+ nullableElement?: boolean;
}
+/** Whether a union is held as its reference arm's plain pointer
+ * (nullable-unions.ts). Without it, only reference elements are matched. */
+export type NullablePointerUnion = (unionId: string) => boolean;
+
/** Kept as the array-analysis entry point; the same edge-preservation
* proof also governs projections passed to borrowing consumers. */
export function findArrayPreservingFunctions(
@@ -39,6 +48,7 @@ interface ArrayReadShape {
element: IrType;
presentTag: number;
missingTag: number;
+ nullableElement?: boolean;
}
/** f64 and bool elements copy their slot bits into the box payload. They
@@ -53,6 +63,7 @@ function isScalarElement(element: IrType): boolean {
function inlineArrayRead(
value: IrExpr,
unions: ReadonlyMap,
+ nullable?: NullablePointerUnion,
): LocalArrayRead | null {
if (value.kind !== "ternary" || value.type.kind !== "union") return null;
const { cond, then, else_: missing } = value;
@@ -62,15 +73,20 @@ function inlineArrayRead(
cond.left.kind !== "arrayState" ||
cond.right.kind !== "numLit" ||
cond.right.value !== 1 ||
- then.kind !== "unionWrap" ||
- then.unionId !== value.type.unionId ||
- then.value.kind !== "arrayGet" ||
missing.kind !== "unionWrap" ||
missing.unionId !== value.type.unionId ||
missing.value.kind !== "unitLit" ||
missing.value.unit !== "undefined"
)
return null;
+ // A `T | undefined` element is read as it is (arrayValueRead adds no wrap).
+ const elementRead =
+ then.kind === "unionWrap" && then.unionId === value.type.unionId
+ ? then.value
+ : then.kind === "arrayGet" && typeEquals(then.type, value.type)
+ ? then
+ : null;
+ if (elementRead?.kind !== "arrayGet") return null;
const sameOperand = (left: IrExpr, right: IrExpr): boolean =>
(left.kind === "varRef" && right.kind === "varRef" && left.localId === right.localId) ||
(left.kind === "numLit" && right.kind === "numLit" && Object.is(left.value, right.value));
@@ -80,23 +96,40 @@ function inlineArrayRead(
array.kind !== "varRef" ||
array.type.kind !== "array" ||
index.type.kind !== "f64" ||
- !sameOperand(array, then.value.arr) ||
- !sameOperand(index, then.value.index)
+ !sameOperand(array, elementRead.arr) ||
+ !sameOperand(index, elementRead.index)
)
return null;
- const element = array.type.elem;
const arms = unions.get(value.type.unionId)?.arms;
- if (
- (!isRefCounted(element) && !isScalarElement(element)) ||
- element.kind === "union" ||
- !arms ||
- arms.length !== 2
- )
+ if (!arms || arms.length !== 2) return null;
+ const missingTag = arms.findIndex((arm) => arm.kind === "undefinedT");
+ if (missingTag < 0 || missing.tag !== missingTag) return null;
+ if (then === elementRead) {
+ // The slots hold the union itself: only its plain-pointer form (NULL is
+ // undefined) shares the element encoding of the reference-arm reads.
+ const presentTag = 1 - missingTag;
+ const element = arms[presentTag]!;
+ if (
+ !typeEquals(array.type.elem, value.type) ||
+ !isRefCounted(element) ||
+ nullable?.(value.type.unionId) !== true
+ )
+ return null;
+ return {
+ type: value.type,
+ element,
+ presentTag,
+ missingTag,
+ array,
+ index,
+ nullableElement: true,
+ };
+ }
+ const element = array.type.elem;
+ if ((!isRefCounted(element) && !isScalarElement(element)) || element.kind === "union")
return null;
const presentTag = arms.findIndex((arm) => typeEquals(arm, element));
- const missingTag = arms.findIndex((arm) => arm.kind === "undefinedT");
- if (presentTag < 0 || missingTag < 0 || then.tag !== presentTag || missing.tag !== missingTag)
- return null;
+ if (presentTag < 0 || then.kind !== "unionWrap" || then.tag !== presentTag) return null;
return { type: value.type, element, presentTag, missingTag, array, index };
}
@@ -104,6 +137,7 @@ function inlineArrayRead(
function arrayReadShape(
fn: IrFunction,
unions: ReadonlyMap,
+ nullable?: NullablePointerUnion,
): ArrayReadShape | null {
if (
fn.async ||
@@ -118,7 +152,7 @@ function arrayReadShape(
return null;
const ret = fn.body[0]!;
if (ret.kind !== "return" || !ret.value) return null;
- const read = inlineArrayRead(ret.value, unions);
+ const read = inlineArrayRead(ret.value, unions, nullable);
if (
!read ||
!typeEquals(read.type, fn.returnType) ||
@@ -133,6 +167,7 @@ function arrayReadShape(
element: read.element,
presentTag: read.presentTag,
missingTag: read.missingTag,
+ ...(read.nullableElement ? { nullableElement: true } : {}),
};
}
@@ -145,15 +180,16 @@ export class OptionalArrayReads {
constructor(
functions: ReadonlyMap,
private readonly unions: ReadonlyMap,
+ private readonly nullable?: NullablePointerUnion,
) {
for (const fn of functions.values()) {
- const shape = arrayReadShape(fn, unions);
+ const shape = arrayReadShape(fn, unions, nullable);
if (shape) this.shapes.set(fn.name, shape);
}
}
get(call: IrExpr): LocalArrayRead | null {
- if (call.kind === "ternary") return inlineArrayRead(call, this.unions);
+ if (call.kind === "ternary") return inlineArrayRead(call, this.unions, this.nullable);
if (call.kind !== "call" || call.args.length !== 2) return null;
const shape = this.shapes.get(call.callee);
if (!shape || !typeEquals(shape.type, call.type)) return null;
@@ -206,6 +242,87 @@ function stableArrayOperand(
}
}
+interface StatementPosition {
+ list: readonly IrStmt[];
+ index: number;
+}
+
+/** The statement list and index of every statement of a body. Statements
+ * nested in expressions (sequence statements) and `for` initializers are
+ * absent: their declarations keep the frame-wide proof alone. */
+function statementPositions(body: readonly IrStmt[]): Map {
+ const positions = new Map();
+ const visit = (list: readonly IrStmt[]): void => {
+ list.forEach((stmt, index) => {
+ positions.set(stmt, { list, index });
+ switch (stmt.kind) {
+ case "if":
+ visit(stmt.then);
+ if (stmt.else_) visit(stmt.else_);
+ break;
+ case "while":
+ case "doWhile":
+ case "for":
+ case "forOf":
+ case "block":
+ visit(stmt.body);
+ break;
+ case "switch":
+ for (const c of stmt.cases) visit(c.body);
+ break;
+ case "tryCatch":
+ visit(stmt.tryBody);
+ if (stmt.catchBody) visit(stmt.catchBody);
+ if (stmt.finallyBody) visit(stmt.finallyBody);
+ break;
+ }
+ });
+ };
+ visit(body);
+ return positions;
+}
+
+function countReferences(body: readonly IrStmt[], localId: string): number {
+ let count = 0;
+ everyStmtList(body as IrStmt[], {
+ stmt: () => true,
+ expr: (e) => {
+ if (e.kind === "varRef" && e.localId === localId) count++;
+ return true;
+ },
+ });
+ return count;
+}
+
+/** A declaration whose whole live range preserves heap edges: every use
+ * of the local follows it in its own statement list, and neither its
+ * initializer nor any statement up to the last use can remove a reference
+ * (effects as in ReferenceEffects). The array operand, rooted at a stable
+ * parameter, then keeps the element alive until the local is dead, though
+ * other code of the function may mutate. */
+function preservingLiveRange(
+ decl: IrStmt & { kind: "varDecl" },
+ positions: ReadonlyMap,
+ body: readonly IrStmt[],
+ effects: ReferenceEffects,
+): boolean {
+ const position = positions.get(decl);
+ if (!position || !decl.init || !effects.preserves(decl.init)) return false;
+ const { list, index } = position;
+ let last = index,
+ seen = 0;
+ for (let i = index + 1; i < list.length; i++) {
+ const count = countReferences([list[i]!], decl.localId);
+ if (count === 0) continue;
+ last = i;
+ seen += count;
+ }
+ return (
+ seen === countReferences(body, decl.localId) &&
+ effects.preservesScope(list.slice(index + 1, last + 1) as IrStmt[])
+ );
+}
+
export function findLocalArrayReads(
fn: IrFunction,
functions: ReadonlyMap,
@@ -214,6 +331,7 @@ export function findLocalArrayReads(
lifetimes: CallLifetimes = analyzeCallLifetimes(functions),
reads = new OptionalArrayReads(functions, unions),
loopBorrows: ReadonlySet = new Set(),
+ effects?: ReferenceEffects,
): Map {
const result = new Map();
if (fn.async || fn.generator) return result;
@@ -224,6 +342,7 @@ export function findLocalArrayReads(
const params = new Set(fn.params.map((p) => p.localId));
const stableParams = stableArrayParameters(fn, lifetimes);
const borrow = arrayPreservingFunctions.has(fn.name);
+ let positions: Map | undefined;
everyStmtList(fn.body, {
expr: () => true,
stmt: (node) => {
@@ -233,11 +352,20 @@ export function findLocalArrayReads(
return true;
const read = reads.get(node.init);
if (read && lifetimes.locals.get(fn.name)?.has(local.id)) {
+ const stable = stableArrayOperand(read.array, stableParams, locals);
if (
- (borrow && stableArrayOperand(read.array, stableParams, locals)) ||
+ (borrow && stable) ||
(read.array.kind === "varRef" &&
loopBorrows.has(node) &&
- !locals.get(read.array.localId)?.boxed)
+ !locals.get(read.array.localId)?.boxed) ||
+ (stable &&
+ effects !== undefined &&
+ preservingLiveRange(
+ node,
+ (positions ??= statementPositions(fn.body)),
+ fn.body,
+ effects,
+ ))
)
read.borrow = true;
result.set(local.id, read);
@@ -250,14 +378,16 @@ export function findLocalArrayReads(
/** An immediate argument needs no heap union when its parameter only
* projects the box. The payload owns a snapshot through all later arguments
- * and the call, unless the enclosing function preserves its parameter's
- * array edges. The latter proof includes every later argument and callee,
- * so a mutation anywhere keeps the independent payload owner. */
+ * and the call, unless the enclosing function, or the call itself with all
+ * of its arguments, preserves its parameter's array edges. That proof
+ * includes every later argument and callee, so a mutation anywhere keeps
+ * the independent payload owner. */
export function findCallArrayReads(
fn: IrFunction,
reads: OptionalArrayReads,
arrayPreservingFunctions: ReadonlySet,
lifetimes: CallLifetimes,
+ effects?: ReferenceEffects,
): Map {
const result = new Map();
if (fn.async || fn.generator) return result;
@@ -274,7 +404,11 @@ export function findCallArrayReads(
if (!parameters.has(index)) return;
const read = reads.get(arg);
if (!read) return;
- if (borrow && stableArrayOperand(read.array, stableParams, locals)) read.borrow = true;
+ if (
+ (borrow || effects?.preserves(node) === true) &&
+ stableArrayOperand(read.array, stableParams, locals)
+ )
+ read.borrow = true;
result.set(arg, read);
});
return true;
@@ -457,10 +591,10 @@ export function emitLocalArrayRead(
const slow = B.newLabel("local.array.slow"),
no = B.newLabel("local.array.missing"),
join = B.newLabel("local.array.join");
+ // A slot of a nullable-union array may hold NULL even when present.
+ const owned = read.nullableElement ? read.type : read.element;
const store = (value: string): void => {
- B.line(
- `store ptr ${read.borrow ? value : host.retainValue(value, read.element)}, ptr ${localSlot}`,
- );
+ B.line(`store ptr ${read.borrow ? value : host.retainValue(value, owned)}, ptr ${localSlot}`);
};
if (inline)
emitDenseReferenceArrayRead(host, array, index, integerIndex, store, no, slow, join, "ptr");
@@ -482,7 +616,7 @@ export function emitLocalArrayRead(
);
B.br(join);
B.startBlock(join);
- return read.borrow ? null : { slot: localSlot, type: read.element };
+ return read.borrow ? null : { slot: localSlot, type: owned };
}
const box = B.slot(),
payload = B.slot(),
diff --git a/packages/compiler/src/backend/llvm/shapes.test.ts b/packages/compiler/src/backend/llvm/shapes.test.ts
index f98ba04a1..71a7b5472 100644
--- a/packages/compiler/src/backend/llvm/shapes.test.ts
+++ b/packages/compiler/src/backend/llvm/shapes.test.ts
@@ -23,7 +23,7 @@ function declarationHost(
},
needOom() {},
sizeType: pointerBits === 32 ? "i32" : "i64",
- cycleColorOffset: pointerBits === 32 ? 12 : 16,
+ cycleBufferedOffset: 6,
tracedShapes: new Set(),
tracedUnions: new Set(),
nullableUnions: new NullableUnions({
@@ -148,9 +148,9 @@ describe("LLVM runtime RC symbols", () => {
expect(fn("sc_rc_release_map")).toContain("%ScrMapRc, ptr %o, i32 0, i32 8");
expect(fn("sc_rc_release_map")).toContain("br i1 %headered, label %slow, label %dec");
expect(fn("sc_rc_release_map")).not.toContain("store i32 1");
- // Unions: no color store, only the i16 `buffered` read at obj-12.
+ // Unions: no color store, only the i16 `buffered` read at obj-6.
expect(fn("sc_rc_release_union")).not.toContain("store i32");
- expect(fn("sc_rc_release_union")).toContain("getelementptr i8, ptr %o, i64 -12");
+ expect(fn("sc_rc_release_union")).toContain("getelementptr i8, ptr %o, i64 -6");
expect(fn("sc_rc_release_union")).toContain("load i16, ptr %bufp");
expect(fn("sc_rc_release_union")).toContain("call void @scr_cyc_on_release(ptr %o)");
// Promises tolerate NULL and always carry a header.
@@ -163,7 +163,7 @@ describe("LLVM runtime RC symbols", () => {
releaseSym(host, { kind: "union", unionId: "u0" } as never);
const ir = emitInlineRcHelpers(host).join("\n");
expect(ir).toContain("load i32, ptr %o");
- expect(ir).toContain("getelementptr i8, ptr %o, i32 -8");
+ expect(ir).toContain("getelementptr i8, ptr %o, i32 -6");
});
});
@@ -182,7 +182,7 @@ describe("emitted object releases", () => {
const start = defs.indexOf("root:");
const root = defs.slice(start, defs.indexOf("done:", start));
expect(root).not.toContain("store i32");
- expect(root).toContain("getelementptr i8, ptr %o, i64 -12");
+ expect(root).toContain("getelementptr i8, ptr %o, i64 -6");
expect(root).toContain("br i1 %queued, label %done, label %enqueue");
expect(root.indexOf("enqueue:")).toBeLessThan(
root.indexOf("call void @scr_cyc_on_release(ptr %o)"),
diff --git a/packages/compiler/src/backend/llvm/shapes.ts b/packages/compiler/src/backend/llvm/shapes.ts
index f528b2f58..7f3e28d5d 100644
--- a/packages/compiler/src/backend/llvm/shapes.ts
+++ b/packages/compiler/src/backend/llvm/shapes.ts
@@ -26,7 +26,7 @@ import {
mangleRecordStruct,
mangleRecordTrace,
} from "../mangle.js";
-import { emitObjectAlloc, emitObjectFree } from "./alloc.js";
+import { cycTypeDescriptor, emitObjectAlloc, emitObjectFree } from "./alloc.js";
import { llvmCommentText } from "./common.js";
import type { NullableUnions } from "./nullable-unions.js";
import { LlvmUnsupportedError } from "./unsupported.js";
@@ -39,8 +39,8 @@ export interface ShapeHost {
needOom(): void;
/** The target C ABI's `size_t` / refcount integer type. */
readonly sizeType: "i32" | "i64";
- /** Byte offset from an object pointer back to ScrCycHdr.color. */
- readonly cycleColorOffset: number;
+ /** Byte offset from an object pointer back to ScrCycHdr.buffered. */
+ readonly cycleBufferedOffset: number;
readonly tracedShapes: Set;
readonly tracedUnions: Set;
/** Unions represented as a nullable arm pointer (nullable-unions.ts):
@@ -63,6 +63,11 @@ export interface ShapeHost {
* keeps every retain/release a runtime call: inline RC is a
* size-for-speed optimization of the `speed` posture only. */
readonly rcHelpers: Set | null;
+ /** The next collector statistics index for an emitted ScrCycType
+ * descriptor (cycTypeDescriptor), counting from FIRST_EMITTED_CYC_KIND
+ * in emission order.
+ * Absent: every descriptor gets 0, which keeps no statistics. */
+ nextCycKind?(): number;
}
/** Every emitted function/helper carries #0 = { sanitize_address } — see
@@ -390,7 +395,23 @@ function releaseHelper(host: ShapeHost, family: string, row: InlineRcFamily): st
` br i1 %untraced, label %done, label %root`,
`root:`,
]
- : []),
+ : family === "box"
+ ? [
+ // scr_box_release buffers a surviving box only when its trace
+ // visits the payload (scr_box_traced): a closure (kind 4), or a
+ // class instance whose obj_trace is set.
+ ` %kp = getelementptr i8, ptr %o, ${S} ${S === "i64" ? 8 : 4} ; ScrBox.kind`,
+ ` %kind = load i32, ptr %kp`,
+ ` %func = icmp eq i32 %kind, 4`,
+ ` br i1 %func, label %root, label %objbox`,
+ `objbox:`,
+ ` %otp = getelementptr i8, ptr %o, ${S} ${S === "i64" ? 32 : 16} ; ScrBox.obj_trace`,
+ ` %ot = load ptr, ptr %otp`,
+ ` %untraced = icmp eq ptr %ot, null`,
+ ` br i1 %untraced, label %done, label %root`,
+ `root:`,
+ ]
+ : []),
...(cycle === "always" ? cycleRootLines(host, "done") : [` br label %done`]),
`done:`,
` ret void`,
@@ -826,15 +847,38 @@ export function releaseFastPath(
/** The possible-cycle-root step of a release that left a headered object
* alive: scr_cyc_on_release with its early return inlined. Read `buffered`
- * (the i16 four bytes after color; nonzero for a queued candidate or a
+ * (the i16 at obj-6; nonzero for a queued candidate or a
* tenured object) and call the runtime only to enqueue a new candidate.
* Every emitted release of a headered object shares this sequence
* (scr_runtime.h asserts the offset); it ends by branching to `done`. */
+/** One traced field of an emitted trace function: load the slot and visit
+ * it unless it is null. Every collector visitor ignores null children, and
+ * most traced slots of a typical heap are empty (optional links, unset
+ * caches), so testing here saves an indirect call per empty slot on every
+ * collector walk. The sequence ends in the new block `tv.next`. */
+export function traceFieldVisitLines(
+ struct: string,
+ index: number,
+ i: number,
+ comment: string,
+): string[] {
+ return [
+ ` %f${i} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${index}`,
+ ` %v${i} = load ptr, ptr %f${i}`,
+ ` %z${i} = icmp eq ptr %v${i}, null`,
+ ` br i1 %z${i}, label %tv${i}.next, label %tv${i}.visit`,
+ `tv${i}.visit:`,
+ ` call void %visit(ptr %v${i}, ptr %ctx) ; ${comment}`,
+ ` br label %tv${i}.next`,
+ `tv${i}.next:`,
+ ];
+}
+
export function cycleRootLines(host: ShapeHost, done: string): string[] {
const S = host.sizeType;
host.declare(`declare void @scr_cyc_on_release(ptr)`);
return [
- ` %bufp = getelementptr i8, ptr %o, ${S} -${host.cycleColorOffset - 4}`,
+ ` %bufp = getelementptr i8, ptr %o, ${S} -${host.cycleBufferedOffset}`,
` %buf = load i16, ptr %bufp`,
` %queued = icmp ne i16 %buf, 0`,
` br i1 %queued, label %${done}, label %enqueue`,
@@ -852,7 +896,7 @@ export function cycleDeadLines(host: ShapeHost, next: string): string[] {
const S = host.sizeType;
host.declare(`declare void @scr_cyc_on_dead(ptr)`);
return [
- ` %dbufp = getelementptr i8, ptr %o, ${S} -${host.cycleColorOffset - 4}`,
+ ` %dbufp = getelementptr i8, ptr %o, ${S} -${host.cycleBufferedOffset}`,
` %dbuf = load i16, ptr %dbufp`,
` %dqueued = icmp ne i16 %dbuf, 0`,
` br i1 %dqueued, label %unbuffer, label %${next}`,
@@ -936,18 +980,18 @@ export function emitRecordShapes(
`define internal ptr @${mangleRecordNew(shape.id)}() ${FN_ATTRS} {`,
`entry:`,
];
- nw.push(
- ...emitObjectAlloc(
- host,
- sizeOf,
- traced
- ? {
- trace: `@${mangleRecordTrace(shape.id)}`,
- free: `@${mangleRecordGcFree(shape.id)}`,
- }
- : null,
- ),
- );
+ const cycType = `@${mangleRecordTrace(shape.id)}.cyctype`;
+ if (traced)
+ defs.push(
+ cycTypeDescriptor(
+ cycType,
+ `@${mangleRecordTrace(shape.id)}`,
+ `@${mangleRecordGcFree(shape.id)}`,
+ host.nextCycKind?.() ?? 0,
+ ),
+ ``,
+ );
+ nw.push(...emitObjectAlloc(host, sizeOf, traced ? { type: cycType } : null));
nw.push(` store ${host.sizeType} 1, ptr %o`);
if (shape.indexValue) {
// The overflow map (string-keyed): value handling is type-directed
@@ -1006,11 +1050,7 @@ export function emitRecordShapes(
`entry:`,
];
tracedMembers.forEach((m, i) => {
- tr.push(
- ` %f${i} = getelementptr inbounds %${struct}, ptr %o, i64 0, i32 ${m.index}`,
- ` %v${i} = load ptr, ptr %f${i}`,
- ` call void %visit(ptr %v${i}, ptr %ctx) ; ${llvmCommentText(m.name)}`,
- );
+ tr.push(...traceFieldVisitLines(struct, m.index, i, llvmCommentText(m.name)));
});
tr.push(` ret void`, `}`, ``);
defs.push(...tr);
diff --git a/packages/compiler/src/backend/runtime-abi.ts b/packages/compiler/src/backend/runtime-abi.ts
index 2d8fc3e62..cdbfbacc5 100644
--- a/packages/compiler/src/backend/runtime-abi.ts
+++ b/packages/compiler/src/backend/runtime-abi.ts
@@ -2,8 +2,8 @@
* of the link contract: a program object keeps this symbol undefined and the
* matching runtime defines it, so incompatible manual links fail before the
* program can start. */
-export const RUNTIME_ABI_VERSION = 9 as const;
-export const RUNTIME_ABI_MARKER = "scr_runtime_abi_v9" as const;
+export const RUNTIME_ABI_VERSION = 12 as const;
+export const RUNTIME_ABI_MARKER = "scr_runtime_abi_v12" as const;
/** Object consumption is useful today, but the complete scr_* surface may
* still change before 1.0. The versioned marker prevents accidental mixing;
diff --git a/packages/compiler/src/backend/runtime-pack-targets.test.ts b/packages/compiler/src/backend/runtime-pack-targets.test.ts
index ce0e894ac..8939fd4cc 100644
--- a/packages/compiler/src/backend/runtime-pack-targets.test.ts
+++ b/packages/compiler/src/backend/runtime-pack-targets.test.ts
@@ -52,7 +52,7 @@ async function fixture(target: NativeTargetSpec, workers = false) {
object_format: target.objectFormat,
minimum_os: target.minimumOs,
},
- runtime_abi: { version: 9, marker: "scr_runtime_abi_v9" },
+ runtime_abi: { version: 12, marker: "scr_runtime_abi_v12" },
compiler: { command: "fixture", identity: "fixture", target: target.llvmTriple },
macros: { executable: [], excluded: ["SCR_LIB"], sanitizer: "external-toolchain-required" },
flavors: {
diff --git a/packages/compiler/src/backend/runtime-pack.test.ts b/packages/compiler/src/backend/runtime-pack.test.ts
index 9cd6ed93c..e71cacd8a 100644
--- a/packages/compiler/src/backend/runtime-pack.test.ts
+++ b/packages/compiler/src/backend/runtime-pack.test.ts
@@ -113,7 +113,7 @@ async function fixture(target: NativeTargetSpec = MACOS_ARM64_TARGET) {
object_format: target.objectFormat,
minimum_os: target.minimumOs,
},
- runtime_abi: { version: 9, marker: "scr_runtime_abi_v9" },
+ runtime_abi: { version: 12, marker: "scr_runtime_abi_v12" },
compiler: {
command: "clang",
identity: "fixture clang",
diff --git a/packages/compiler/src/frontend/lowering/call-arguments.ts b/packages/compiler/src/frontend/lowering/call-arguments.ts
index 8343aeeca..09a259228 100644
--- a/packages/compiler/src/frontend/lowering/call-arguments.ts
+++ b/packages/compiler/src/frontend/lowering/call-arguments.ts
@@ -25,6 +25,9 @@ import { isSafeToDiscard } from "./expressions/evaluation-safety.js";
import { defaultAfterUndefined, lowerStaticallyUndefinedArgument } from "./optional-arguments.js";
import { checkedIterableSpread } from "./checked-iterable-spread.js";
import { type ParamShape } from "./call-signatures.js";
+import { arrayValueRead, arrayValueType } from "./array-values.js";
+import { buildArrayConversion } from "./coercions/builders.js";
+import { typeKey } from "../type-mapper.js";
function fixedTupleSpreadInfo(
lowerer: Lowerer,
@@ -63,11 +66,25 @@ export function completeArgs(
): IrExpr[] {
// Conversion eligibility can traverse the whole recursive parameter
// graph. Ordinary calls and fixed tuples never need that analysis.
+ const runtimeSpreads = argNodes.filter(
+ (arg): arg is ts.SpreadElement =>
+ ts.isSpreadElement(arg) && !fixedTupleSpreadInfo(lowerer, arg.expression),
+ );
+ // Spread operands lowered while choosing the completion, each exactly once.
+ const loweredSpreads = new Map();
+ const lowerSpread = (arg: ts.SpreadElement): IrExpr =>
+ loweredSpreads.get(arg) ?? lowerer.lowerExpr(arg.expression);
if (
- argNodes.some(
- (arg) => ts.isSpreadElement(arg) && !fixedTupleSpreadInfo(lowerer, arg.expression),
- ) &&
- canCompleteRuntimeSpread(lowerer, shapes)
+ runtimeSpreads.length > 0 &&
+ canCompleteRuntimeSpread(lowerer, shapes) &&
+ !spreadsFillTypedRest(
+ lowerer,
+ shapes,
+ argNodes,
+ leading?.length ?? 0,
+ runtimeSpreads,
+ loweredSpreads,
+ )
) {
// Runtime-length spreads determine the complete argument list before
// any parameter default runs. Build it once in source order, then
@@ -96,7 +113,7 @@ export function completeArgs(
push(lowerer.lowerExprExpecting(arg, DYN), "dyn.packPush");
return;
}
- let value = lowerer.lowerExpr(arg.expression);
+ let value = lowerSpread(arg);
if (value.type.kind === "set") {
value = {
kind: "setIntrinsic",
@@ -390,7 +407,11 @@ export function completeArgs(
const elems = sources.slice(restAt).map((a, i) => {
if (isIr(a)) return lowerer.coerceInto(blame, a.ir, restType.elem);
if (ts.isSpreadElement(a)) {
- let src = lowerer.lowerExpr(a.expression);
+ let src = lowerSpread(a);
+ // A `T[]` into a `(T | undefined)[]` rest: read every element as JS
+ // does, holes and present undefined as undefined.
+ const lift = holeReadingLift(lowerer, src.type, restType, loc);
+ if (lift) src = { kind: "call", callee: lift, args: [src], type: restType, loc: src.loc };
// A same-element Set spread drains first (setIntrinsic toArray).
if (src.type.kind === "set" && typeEquals(src.type.elem, restType.elem)) {
src = {
@@ -688,6 +709,102 @@ export function spreadNeedsRuntimeArity(
return false;
}
+/** True when every runtime-length spread only fills a typed rest with its
+ * own array type, or a `T[]` into an argument-widened `(T | undefined)[]`
+ * rest. The static completion then builds one fresh rest array and converts
+ * no argument to a checked-dynamic value: boxing an instance would give its
+ * class (and, through the generated property helpers, the classes it
+ * references) a hidden property bag. Lowers the spread operands it checks
+ * into `lowered`, so each is lowered exactly once. */
+function spreadsFillTypedRest(
+ lowerer: Lowerer,
+ shapes: readonly ParamShape[],
+ argNodes: readonly ts.Expression[],
+ leading: number,
+ runtimeSpreads: readonly ts.SpreadElement[],
+ lowered: Map,
+): boolean {
+ const restAt = typedRestAfterSpreads(lowerer, shapes, argNodes, leading);
+ const restType = restAt >= 0 ? shapes[restAt]!.type : null;
+ if (restType?.kind !== "array") return false;
+ for (const arg of runtimeSpreads) lowered.set(arg, lowerer.lowerExpr(arg.expression));
+ return runtimeSpreads.every((arg) => {
+ const type = lowered.get(arg)!.type;
+ return typeEquals(type, restType) || canReadHolesInto(lowerer, type, restType);
+ });
+}
+
+/** The index of the typed `rest` parameter when every runtime-length spread
+ * lands in it (after the fixed positions), else -1. */
+function typedRestAfterSpreads(
+ lowerer: Lowerer,
+ shapes: readonly ParamShape[],
+ argNodes: readonly ts.Expression[],
+ leading: number,
+): number {
+ const restAt = shapes.findIndex((s) => s.mode !== "required" && s.mode !== "omittable");
+ if (restAt < 0 || shapes[restAt]!.mode !== "rest") return -1;
+ let position = leading;
+ for (const arg of argNodes) {
+ if (ts.isSpreadElement(arg)) {
+ const tuple = fixedTupleSpreadInfo(lowerer, arg.expression);
+ if (tuple) {
+ position += tuple.fields.length;
+ continue;
+ }
+ if (position < restAt) return -1;
+ }
+ position++;
+ }
+ return restAt;
+}
+
+/** A `T[]` spread into a `(T | undefined)[]` rest, `T | undefined` being
+ * exactly what a JS read of a `T[]` element yields. */
+function canReadHolesInto(lowerer: Lowerer, from: IrType, to: IrType): boolean {
+ if (from.kind !== "array" || to.kind !== "array" || typeEquals(from, to)) return false;
+ const elem = from.elem;
+ if (elem.kind === "dyn" || elem.kind === "jsval") return false;
+ return typeEquals(arrayValueType(lowerer, elem), to.elem);
+}
+
+/** The interned `%arr.holes.` copy for canReadHolesInto pairs: spread
+ * iteration reads holes and present undefined as undefined. Null for any
+ * other pair. */
+function holeReadingLift(lowerer: Lowerer, from: IrType, to: IrType, loc: SrcLoc): string | null {
+ if (from.kind !== "array" || !canReadHolesInto(lowerer, from, to)) return null;
+ const elem = from.elem;
+ const key = `arr.holes:${typeKey(elem)}`;
+ const existing = lowerer.valueHelpers.get(key);
+ if (existing) return existing;
+ const name = `%arr.holes.${lowerer.valueHelpers.size}`;
+ lowerer.valueHelpers.set(key, name);
+ lowerer.liftedFns.push(
+ buildArrayConversion(
+ name,
+ from,
+ { kind: "arrayLit", elems: [], type: to, loc },
+ (element, index, result) => ({
+ kind: "arrIntrinsic",
+ method: "push",
+ receiver: result,
+ args: [arrayValueRead(lowerer, sourceArray(element), index, elem, loc)],
+ type: F64,
+ loc,
+ }),
+ loc,
+ ),
+ );
+ return name;
+}
+
+/** The array a buildArrayConversion element read indexes. */
+function sourceArray(element: IrExpr): IrExpr {
+ if (element.kind !== "arrayGet")
+ throw new InternalCompilerError("array conversion element is not an array read");
+ return element.arr;
+}
+
function canCompleteRuntimeSpread(lowerer: Lowerer, shapes: readonly ParamShape[]): boolean {
return (
shapes.length > 0 &&
diff --git a/packages/compiler/test/bytes-element-emission.test.ts b/packages/compiler/test/bytes-element-emission.test.ts
index 953687993..08da691ee 100644
--- a/packages/compiler/test/bytes-element-emission.test.ts
+++ b/packages/compiler/test/bytes-element-emission.test.ts
@@ -314,7 +314,7 @@ test("canonical byte loops keep their induction variable and indices integral",
expect(ll).toContain("alloca i64 ; integer induction index");
expect(ll).toContain("icmp ult i64");
- expect(ll).not.toContain("bytes.index.range");
+ expect(ll).not.toContain("bytes.index.valid");
});
test("a body mutation keeps the byte loop on the general f64 path", () => {
@@ -324,8 +324,8 @@ test("a body mutation keeps the byte loop on the general f64 path", () => {
const ll = emitLlvmModule(mod);
expect(ll).not.toContain("integer induction index");
- expect(ll).toContain("bytes.index.range");
- expect(ll).toContain("fptoui double");
+ expect(ll).toContain("bytes.index.valid");
+ expect(ll).toContain("fptosi double");
});
test("large record clones stay outlined while small clones remain inlineable", () => {
diff --git a/packages/compiler/test/constant-table-emission.test.ts b/packages/compiler/test/constant-table-emission.test.ts
index 60be74507..5fddb2663 100644
--- a/packages/compiler/test/constant-table-emission.test.ts
+++ b/packages/compiler/test/constant-table-emission.test.ts
@@ -33,14 +33,15 @@ console.log(lookup(1), mutated[0] * 1);
expect(llvm).toContain("@sc_const_numbers_0 = private constant [3 x double]");
expect(llvm).toContain("%initialized = icmp ne ptr %a, null");
- expect(llvm).toContain("br i1 %safe, label %convert, label %fallback");
- expect(llvm).toContain("br i1 %integer, label %read, label %fallback");
+ expect(llvm).toContain("%ix0 = fptosi double %i to i64");
+ expect(llvm).toContain("%safe = and i1 %initialized, %ix5");
+ expect(llvm).toContain("br i1 %safe, label %read, label %fallback");
expect(llvm).toMatch(/call double @sc_const_numbers_0_get\(/);
expect(llvm).toContain("call double @scr_arr_get_number(ptr %a, double %i)");
expect(llvm).toMatch(/call double @scr_arr_push_many\(ptr [^,]+, i64 3, ptr /);
const wasm = emitLlvmModule(mod, { pointerBits: 32, wasi: true });
- expect(wasm).toContain("%index = fptoui double %i to i32");
- expect(wasm).toContain("ptr @sc_const_numbers_0, i32 0, i32 %index");
+ expect(wasm).toContain("%ix0 = fptosi double %i to i64");
+ expect(wasm).toContain("ptr @sc_const_numbers_0, i64 0, i64 %ix1");
} finally {
await rm(dir, { recursive: true, force: true });
}
diff --git a/packages/compiler/test/native-codegen-integration.test.ts b/packages/compiler/test/native-codegen-integration.test.ts
index f7aaeca7a..d85579558 100644
--- a/packages/compiler/test/native-codegen-integration.test.ts
+++ b/packages/compiler/test/native-codegen-integration.test.ts
@@ -312,7 +312,7 @@ ${[0, 1, 2, 3, 4, 5].map((index) => ` call void @add${index}(i64 ${index + 2})`
expect(await symbols(helperObject, ["-u"])).toEqual(await symbols(clangObject, ["-u"]));
expect(await symbols(helperObject, ["-gU"])).toEqual(await symbols(clangObject, ["-gU"]));
expect(await symbols(helperObject, ["-gU"])).toEqual(["0000000000000000 T _main"]);
- expect(await symbols(helperObject, ["-u"])).toContain("_scr_runtime_abi_v9");
+ expect(await symbols(helperObject, ["-u"])).toContain("_scr_runtime_abi_v12");
const clangExe = join(dir, "clang-program");
const linkDriver = join(dir, "clang-driver.c");
@@ -646,7 +646,7 @@ ${[0, 1, 2, 3, 4, 5].map((index) => ` call void @add${index}(i64 ${index + 2})`
.split("\n");
expect(undefinedSymbols).toContain("_sf_scale");
expect(undefinedSymbols).toContain("_sf_callback_mix");
- expect(undefinedSymbols).toContain("_scr_runtime_abi_v9");
+ expect(undefinedSymbols).toContain("_scr_runtime_abi_v12");
});
test("helper and clang object paths retain the same outbound FFI C ABI", async () => {
diff --git a/packages/compiler/test/ts7/baselines/order-parity.json b/packages/compiler/test/ts7/baselines/order-parity.json
index e54edabc6..effe6ad03 100644
--- a/packages/compiler/test/ts7/baselines/order-parity.json
+++ b/packages/compiler/test/ts7/baselines/order-parity.json
@@ -15865,6 +15865,30 @@
"/tests/corpus/switch-integer-dispatch.ts"
],
"diags": []
+ },
+ "/tests/corpus/4660-typed-rest-spread-forwarding.ts": {
+ "order": [
+ "/tests/corpus/4660-typed-rest-spread-forwarding.ts"
+ ],
+ "diags": []
+ },
+ "/tests/corpus/4610-array-ext-dyn-refresh.ts": {
+ "order": [
+ "/tests/corpus/4610-array-ext-dyn-refresh.ts"
+ ],
+ "diags": []
+ },
+ "/tests/corpus/4690-cycle-bypass-long-lived.ts": {
+ "order": [
+ "/tests/corpus/4690-cycle-bypass-long-lived.ts"
+ ],
+ "diags": []
+ },
+ "/tests/corpus/4671-scoped-array-read-borrows.ts": {
+ "order": [
+ "/tests/corpus/4671-scoped-array-read-borrows.ts"
+ ],
+ "diags": []
}
}
}
diff --git a/packages/runtime-darwin-arm64/scripts/build.mjs b/packages/runtime-darwin-arm64/scripts/build.mjs
index e3c21484d..b062b5980 100644
--- a/packages/runtime-darwin-arm64/scripts/build.mjs
+++ b/packages/runtime-darwin-arm64/scripts/build.mjs
@@ -316,7 +316,7 @@ async function build() {
package: packageManifest.name,
version: packageManifest.version,
target: RUNTIME_PACK_MATRIX.target,
- runtime_abi: { version: 9, marker: "scr_runtime_abi_v9" },
+ runtime_abi: { version: 12, marker: "scr_runtime_abi_v12" },
compiler: {
command: compiler,
identity: compilerVersion,
diff --git a/packages/runtime-pack-common/scripts/build.mjs b/packages/runtime-pack-common/scripts/build.mjs
index b0f7cc72a..e0752938e 100644
--- a/packages/runtime-pack-common/scripts/build.mjs
+++ b/packages/runtime-pack-common/scripts/build.mjs
@@ -403,7 +403,7 @@ async function build() {
package: packageManifest.name,
version: packageManifest.version,
target: matrix.target,
- runtime_abi: { version: 9, marker: "scr_runtime_abi_v9" },
+ runtime_abi: { version: 12, marker: "scr_runtime_abi_v12" },
compiler: {
command: compiler,
identity: compilerVersion,
diff --git a/packages/runtime/src/scr_array.c b/packages/runtime/src/scr_array.c
index 923a83986..8a419e8ee 100644
--- a/packages/runtime/src/scr_array.c
+++ b/packages/runtime/src/scr_array.c
@@ -33,6 +33,19 @@ static void scr_arr_oom(void) {
scr_trap("scriptc: out of memory\n");
}
+/* Readers of the out-of-line storage see an all-empty one until the first
+ * sparse entry, noncanonical property or metadata allocates it. */
+static const ScrArrExt scr_arr_no_ext;
+#define SCR_AX(a) ((const ScrArrExt *)((a)->ext ? (a)->ext : &scr_arr_no_ext))
+
+ScrArrExt *scr_arr_ext(ScrArr *a) {
+ if (!a->ext) {
+ a->ext = scr_mem_calloc(sizeof(ScrArrExt));
+ if (!a->ext) scr_arr_oom();
+ }
+ return a->ext;
+}
+
/* Canonical array indices grow indexed storage and may leave holes. Negative,
* fractional, NaN, infinity, and 2^32-1 numeric keys use the ordinary
* property vector; only a missing property or a low-level read of a hole
@@ -63,9 +76,9 @@ static bool scr_arr_valid_index(double i, size_t *out) {
static size_t scr_arr_prop_index(const ScrArr *a, const char *key,
size_t key_len) {
- for (size_t i = 0; i < a->prop_len; i++) {
- if (a->props[i].key_len == key_len &&
- memcmp(a->props[i].key, key, key_len) == 0) {
+ for (size_t i = 0; i < SCR_AX(a)->prop_len; i++) {
+ if (SCR_AX(a)->props[i].key_len == key_len &&
+ memcmp(SCR_AX(a)->props[i].key, key, key_len) == 0) {
return i;
}
}
@@ -77,16 +90,17 @@ static size_t scr_arr_prop_key(double i, char *buf) {
}
static void scr_arr_grow_props(ScrArr *a, size_t need) {
- if (need <= a->prop_cap) return;
- size_t cap = a->prop_cap ? a->prop_cap : 4;
+ ScrArrExt *x = scr_arr_ext(a);
+ if (need <= x->prop_cap) return;
+ size_t cap = x->prop_cap ? x->prop_cap : 4;
while (cap < need) {
- if (cap > SIZE_MAX / 2 / sizeof(*a->props)) scr_arr_oom();
+ if (cap > SIZE_MAX / 2 / sizeof(*x->props)) scr_arr_oom();
cap *= 2;
}
- ScrArrProp *props = realloc(a->props, cap * sizeof(*props));
+ ScrArrProp *props = realloc(x->props, cap * sizeof(*props));
if (!props) scr_arr_oom();
- a->props = props;
- a->prop_cap = cap;
+ x->props = props;
+ x->prop_cap = cap;
}
/* ── slot packing: 8-byte slots hold doubles, bools, or pointers ───────── */
@@ -161,8 +175,8 @@ static size_t scr_arr_sparse_lower_bound(const ScrArrSparseSlot *slots,
/* The sorted side store is a deque. Its allocation holds `front` spare slots,
* one header slot whose index records `front`, then sparse_cap slots starting
- * at a->sparse (live entries first). The header sits directly before
- * a->sparse, so every owner of the pointer (including swapped or detached
+ * at SCR_AX(a)->sparse (live entries first). The header sits directly before
+ * SCR_AX(a)->sparse, so every owner of the pointer (including swapped or detached
* storage) can recover the allocation base. Inserts and removals move only
* the shorter side, so ascending and descending first-touch writes and prefix
* migration into dense storage are amortized O(1). */
@@ -186,75 +200,79 @@ static size_t scr_arr_sparse_grow_cap(size_t have, size_t need) {
/* Reallocate with `front` spare slots before the live entries and `back`
* slots from the first live entry onward (back >= sparse_len). */
static void scr_arr_sparse_realloc(ScrArr *a, size_t front, size_t back) {
+ ScrArrExt *x = scr_arr_ext(a);
const size_t unit = sizeof(ScrArrSparseSlot);
if (front > SIZE_MAX / unit / 2 || back > SIZE_MAX / unit / 2) scr_arr_oom();
size_t total = (front + 1 + back) * unit;
ScrArrSparseSlot *slots;
- if (a->sparse && front == scr_arr_sparse_front(a->sparse)) {
- ScrArrSparseSlot *base = realloc(a->sparse - 1 - front, total);
+ if (x->sparse && front == scr_arr_sparse_front(x->sparse)) {
+ ScrArrSparseSlot *base = realloc(x->sparse - 1 - front, total);
if (!base) scr_arr_oom();
slots = base + front + 1;
} else {
ScrArrSparseSlot *base = malloc(total);
if (!base) scr_arr_oom();
slots = base + front + 1;
- if (a->sparse_len) memcpy(slots, a->sparse, a->sparse_len * unit);
- scr_arr_sparse_free(a->sparse);
+ if (x->sparse_len) memcpy(slots, x->sparse, x->sparse_len * unit);
+ scr_arr_sparse_free(x->sparse);
}
slots[-1] = (ScrArrSparseSlot){front, 0, 0};
- a->sparse = slots;
- a->sparse_cap = back;
+ x->sparse = slots;
+ x->sparse_cap = back;
}
static void scr_arr_grow_sparse(ScrArr *a, size_t need) {
- if (need <= a->sparse_cap) return;
- scr_arr_sparse_realloc(a, scr_arr_sparse_front(a->sparse),
- scr_arr_sparse_grow_cap(a->sparse_cap, need));
+ if (need <= SCR_AX(a)->sparse_cap) return;
+ scr_arr_sparse_realloc(a, scr_arr_sparse_front(SCR_AX(a)->sparse),
+ scr_arr_sparse_grow_cap(SCR_AX(a)->sparse_cap, need));
}
static void scr_arr_sparse_insert(ScrArr *a, size_t pos, ScrArrSparseSlot entry) {
- size_t len = a->sparse_len;
+ ScrArrExt *x = scr_arr_ext(a);
+ size_t len = x->sparse_len;
if (pos < len - pos) {
- if (scr_arr_sparse_front(a->sparse) == 0) {
- scr_arr_sparse_realloc(a, len > 4 ? len : 4, a->sparse_cap);
+ if (scr_arr_sparse_front(x->sparse) == 0) {
+ scr_arr_sparse_realloc(a, len > 4 ? len : 4, x->sparse_cap);
}
- size_t front = scr_arr_sparse_front(a->sparse);
- ScrArrSparseSlot *slots = a->sparse - 1; /* the old header slot */
+ size_t front = scr_arr_sparse_front(x->sparse);
+ ScrArrSparseSlot *slots = x->sparse - 1; /* the old header slot */
slots[-1] = (ScrArrSparseSlot){front - 1, 0, 0};
- memmove(slots, a->sparse, pos * sizeof(*slots));
+ memmove(slots, x->sparse, pos * sizeof(*slots));
slots[pos] = entry;
- a->sparse = slots;
- a->sparse_cap++;
+ x->sparse = slots;
+ x->sparse_cap++;
} else {
scr_arr_grow_sparse(a, len + 1);
- memmove(a->sparse + pos + 1, a->sparse + pos, (len - pos) * sizeof(*a->sparse));
- a->sparse[pos] = entry;
+ memmove(x->sparse + pos + 1, x->sparse + pos, (len - pos) * sizeof(*x->sparse));
+ x->sparse[pos] = entry;
}
- a->sparse_len = len + 1;
+ x->sparse_len = len + 1;
}
static void scr_arr_sparse_remove(ScrArr *a, size_t pos) {
- size_t len = a->sparse_len;
+ ScrArrExt *x = scr_arr_ext(a);
+ size_t len = x->sparse_len;
if (pos < len - 1 - pos) {
- size_t front = scr_arr_sparse_front(a->sparse);
- memmove(a->sparse + 1, a->sparse, pos * sizeof(*a->sparse));
- a->sparse[0] = (ScrArrSparseSlot){front + 1, 0, 0};
- a->sparse++;
- a->sparse_cap--;
+ size_t front = scr_arr_sparse_front(x->sparse);
+ memmove(x->sparse + 1, x->sparse, pos * sizeof(*x->sparse));
+ x->sparse[0] = (ScrArrSparseSlot){front + 1, 0, 0};
+ x->sparse++;
+ x->sparse_cap--;
} else {
- memmove(a->sparse + pos, a->sparse + pos + 1, (len - pos - 1) * sizeof(*a->sparse));
+ memmove(x->sparse + pos, x->sparse + pos + 1, (len - pos - 1) * sizeof(*x->sparse));
}
- a->sparse_len = len - 1;
+ x->sparse_len = len - 1;
}
/* Drop the first `count` live entries after the caller moved them out. */
static void scr_arr_sparse_drop_prefix(ScrArr *a, size_t count) {
if (count == 0) return;
- size_t front = scr_arr_sparse_front(a->sparse);
- a->sparse[count - 1] = (ScrArrSparseSlot){front + count, 0, 0};
- a->sparse += count;
- a->sparse_cap -= count;
- a->sparse_len -= count;
+ ScrArrExt *x = scr_arr_ext(a);
+ size_t front = scr_arr_sparse_front(x->sparse);
+ x->sparse[count - 1] = (ScrArrSparseSlot){front + count, 0, 0};
+ x->sparse += count;
+ x->sparse_cap -= count;
+ x->sparse_len -= count;
}
/* Grow dense storage to cover at least [0, need). Callers decide whether the
@@ -287,10 +305,10 @@ static void scr_arr_grow_dense_to(ScrArr *a, size_t need, bool exact) {
a->data = data;
a->present = present;
a->cap = cap;
- size_t moved = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, cap);
+ size_t moved = scr_arr_sparse_lower_bound(SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, cap);
for (size_t i = 0; i < moved; i++) {
- a->data[a->sparse[i].index] = a->sparse[i].slot;
- a->present[a->sparse[i].index] = a->sparse[i].state;
+ a->data[SCR_AX(a)->sparse[i].index] = SCR_AX(a)->sparse[i].slot;
+ a->present[SCR_AX(a)->sparse[i].index] = SCR_AX(a)->sparse[i].state;
}
scr_arr_sparse_drop_prefix(a, moved);
}
@@ -321,7 +339,7 @@ static bool scr_arr_dense_ok(const ScrArr *a, size_t need, size_t added) {
if (need <= a->cap || need <= SCR_ARR_DENSE_LIMIT) return true;
size_t base = a->cap > SCR_ARR_DENSE_LIMIT ? a->cap : SCR_ARR_DENSE_LIMIT;
uint64_t span = (uint64_t)(need - base);
- uint64_t populated = (uint64_t)a->sparse_len + (uint64_t)added;
+ uint64_t populated = (uint64_t)SCR_AX(a)->sparse_len + (uint64_t)added;
return populated * 16 >= span * 3;
}
@@ -339,8 +357,8 @@ static void scr_arr_grow_for_index(ScrArr *a, size_t index) {
}
/* Densify the whole sparse tail once it is populated enough. */
size_t need = index + 1;
- if (a->sparse_len > 0 && a->sparse[a->sparse_len - 1].index >= need) {
- need = a->sparse[a->sparse_len - 1].index + 1;
+ if (SCR_AX(a)->sparse_len > 0 && SCR_AX(a)->sparse[SCR_AX(a)->sparse_len - 1].index >= need) {
+ need = SCR_AX(a)->sparse[SCR_AX(a)->sparse_len - 1].index + 1;
}
if (scr_arr_dense_ok(a, need, 1)) scr_arr_grow_dense_to(a, need, true);
}
@@ -365,13 +383,13 @@ static bool scr_arr_slot_at_storage(const ScrArrStorage *s, size_t index,
static bool scr_arr_slot_at(const ScrArr *a, size_t index, uint64_t *out) {
ScrArrStorage s = {
- a->data, a->present, a->cap, a->sparse, a->sparse_len, a->sparse_cap};
+ a->data, a->present, a->cap, SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, SCR_AX(a)->sparse_cap};
return scr_arr_slot_at_storage(&s, index, out);
}
static uint8_t scr_arr_state_at(const ScrArr *a, size_t index, uint64_t *out) {
ScrArrStorage s = {
- a->data, a->present, a->cap, a->sparse, a->sparse_len, a->sparse_cap};
+ a->data, a->present, a->cap, SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, SCR_AX(a)->sparse_cap};
return scr_arr_state_at_storage(&s, index, out);
}
@@ -385,10 +403,10 @@ static void scr_arr_store_state_owned(ScrArr *a, size_t index, uint64_t slot,
a->present[index] = state;
return;
}
- size_t pos = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, index);
- if (pos < a->sparse_len && a->sparse[pos].index == index) {
- a->sparse[pos].slot = slot;
- a->sparse[pos].state = state;
+ size_t pos = scr_arr_sparse_lower_bound(SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, index);
+ if (pos < SCR_AX(a)->sparse_len && SCR_AX(a)->sparse[pos].index == index) {
+ SCR_AX(a)->sparse[pos].slot = slot;
+ SCR_AX(a)->sparse[pos].state = state;
return;
}
scr_arr_sparse_insert(a, pos, (ScrArrSparseSlot){index, slot, state});
@@ -409,10 +427,10 @@ static bool scr_arr_take_state(ScrArr *a, size_t index, uint64_t *out,
if (state_out) *state_out = state;
return true;
}
- size_t pos = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, index);
- if (pos == a->sparse_len || a->sparse[pos].index != index) return false;
- *out = a->sparse[pos].slot;
- if (state_out) *state_out = a->sparse[pos].state;
+ size_t pos = scr_arr_sparse_lower_bound(SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, index);
+ if (pos == SCR_AX(a)->sparse_len || SCR_AX(a)->sparse[pos].index != index) return false;
+ *out = SCR_AX(a)->sparse[pos].slot;
+ if (state_out) *state_out = SCR_AX(a)->sparse[pos].state;
scr_arr_sparse_remove(a, pos);
return true;
}
@@ -432,15 +450,15 @@ static void scr_arr_replace_state_owned(ScrArr *a, size_t index, uint64_t slot,
a->data[index] = slot;
a->present[index] = state;
} else {
- size_t pos = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, index);
- if (pos == a->sparse_len || a->sparse[pos].index != index) {
+ size_t pos = scr_arr_sparse_lower_bound(SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, index);
+ if (pos == SCR_AX(a)->sparse_len || SCR_AX(a)->sparse[pos].index != index) {
scr_arr_store_state_owned(a, index, slot, state);
return;
}
- old_state = a->sparse[pos].state;
- old = a->sparse[pos].slot;
- a->sparse[pos].slot = slot;
- a->sparse[pos].state = state;
+ old_state = SCR_AX(a)->sparse[pos].state;
+ old = SCR_AX(a)->sparse[pos].slot;
+ SCR_AX(a)->sparse[pos].slot = slot;
+ SCR_AX(a)->sparse[pos].state = state;
}
if (old_state == SCR_ARR_VALUE && scr_elem_is_ref(a->elem)) scr_elem_release(a, old);
}
@@ -455,8 +473,8 @@ static bool scr_arr_prop_get_state(const ScrArr *a, double key, uint64_t *out,
size_t len = scr_arr_prop_key(key, text);
size_t pos = scr_arr_prop_index(a, text, len);
if (pos == SIZE_MAX) return false;
- if (out) *out = a->props[pos].slot;
- if (state_out) *state_out = a->props[pos].state;
+ if (out) *out = SCR_AX(a)->props[pos].slot;
+ if (state_out) *state_out = SCR_AX(a)->props[pos].state;
return true;
}
@@ -469,20 +487,20 @@ static void scr_arr_prop_set(ScrArr *a, double key, uint64_t slot) {
size_t len = scr_arr_prop_key(key, text);
size_t pos = scr_arr_prop_index(a, text, len);
if (pos != SIZE_MAX) {
- uint64_t old = a->props[pos].slot;
- a->props[pos].slot = slot;
- uint8_t old_state = a->props[pos].state;
- a->props[pos].state = SCR_ARR_VALUE;
+ uint64_t old = SCR_AX(a)->props[pos].slot;
+ SCR_AX(a)->props[pos].slot = slot;
+ uint8_t old_state = SCR_AX(a)->props[pos].state;
+ SCR_AX(a)->props[pos].state = SCR_ARR_VALUE;
if (old_state == SCR_ARR_VALUE && scr_elem_is_ref(a->elem)) {
scr_elem_release(a, old);
}
return;
}
- scr_arr_grow_props(a, a->prop_len + 1);
+ scr_arr_grow_props(a, SCR_AX(a)->prop_len + 1);
char *copy = malloc(len + 1);
if (!copy) scr_arr_oom();
memcpy(copy, text, len + 1);
- a->props[a->prop_len++] = (ScrArrProp){copy, len, slot, SCR_ARR_VALUE};
+ a->ext->props[a->ext->prop_len++] = (ScrArrProp){copy, len, slot, SCR_ARR_VALUE};
}
static void scr_arr_prop_set_undefined(ScrArr *a, double key) {
@@ -490,20 +508,20 @@ static void scr_arr_prop_set_undefined(ScrArr *a, double key) {
size_t len = scr_arr_prop_key(key, text);
size_t pos = scr_arr_prop_index(a, text, len);
if (pos != SIZE_MAX) {
- uint64_t old = a->props[pos].slot;
- uint8_t old_state = a->props[pos].state;
- a->props[pos].slot = 0;
- a->props[pos].state = SCR_ARR_UNDEFINED;
+ uint64_t old = SCR_AX(a)->props[pos].slot;
+ uint8_t old_state = SCR_AX(a)->props[pos].state;
+ SCR_AX(a)->props[pos].slot = 0;
+ SCR_AX(a)->props[pos].state = SCR_ARR_UNDEFINED;
if (old_state == SCR_ARR_VALUE && scr_elem_is_ref(a->elem)) {
scr_elem_release(a, old);
}
return;
}
- scr_arr_grow_props(a, a->prop_len + 1);
+ scr_arr_grow_props(a, SCR_AX(a)->prop_len + 1);
char *copy = malloc(len + 1);
if (!copy) scr_arr_oom();
memcpy(copy, text, len + 1);
- a->props[a->prop_len++] = (ScrArrProp){copy, len, 0, SCR_ARR_UNDEFINED};
+ a->ext->props[a->ext->prop_len++] = (ScrArrProp){copy, len, 0, SCR_ARR_UNDEFINED};
}
static bool scr_arr_prop_delete(ScrArr *a, double key) {
@@ -511,12 +529,12 @@ static bool scr_arr_prop_delete(ScrArr *a, double key) {
size_t len = scr_arr_prop_key(key, text);
size_t pos = scr_arr_prop_index(a, text, len);
if (pos == SIZE_MAX) return false;
- uint64_t old = a->props[pos].slot;
- uint8_t old_state = a->props[pos].state;
- free(a->props[pos].key);
- memmove(a->props + pos, a->props + pos + 1,
- (a->prop_len - pos - 1) * sizeof(*a->props));
- a->prop_len--;
+ uint64_t old = SCR_AX(a)->props[pos].slot;
+ uint8_t old_state = SCR_AX(a)->props[pos].state;
+ free(SCR_AX(a)->props[pos].key);
+ memmove(SCR_AX(a)->props + pos, SCR_AX(a)->props + pos + 1,
+ (SCR_AX(a)->prop_len - pos - 1) * sizeof(*SCR_AX(a)->props));
+ a->ext->prop_len--;
if (old_state == SCR_ARR_VALUE && scr_elem_is_ref(a->elem)) {
scr_elem_release(a, old);
}
@@ -525,14 +543,16 @@ static bool scr_arr_prop_delete(ScrArr *a, double key) {
static ScrArrStorage scr_arr_take_storage(ScrArr *a) {
ScrArrStorage old = {
- a->data, a->present, a->cap, a->sparse, a->sparse_len,
- a->sparse_cap};
+ a->data, a->present, a->cap, SCR_AX(a)->sparse, SCR_AX(a)->sparse_len,
+ SCR_AX(a)->sparse_cap};
a->data = NULL;
a->present = NULL;
a->cap = 0;
- a->sparse = NULL;
- a->sparse_len = 0;
- a->sparse_cap = 0;
+ if (a->ext) {
+ a->ext->sparse = NULL;
+ a->ext->sparse_len = 0;
+ a->ext->sparse_cap = 0;
+ }
return old;
}
@@ -546,7 +566,7 @@ static void scr_arr_free_storage(ScrArrStorage *s) {
* operations move ownership with the slots and move the states alongside
* them; they never inspect an uninitialized hole payload. */
static bool scr_arr_is_dense(const ScrArr *a) {
- return a->len <= a->cap && a->sparse_len == 0;
+ return a->len <= a->cap && SCR_AX(a)->sparse_len == 0;
}
static void scr_arr_move_dense(ScrArr *a, size_t to, size_t from, size_t count) {
@@ -602,13 +622,7 @@ ScrArr *scr_arr_new(ScrElemKind elem, size_t initial_cap) {
a->elem_trace = NULL;
a->data = NULL;
a->present = NULL;
- a->sparse = NULL;
- a->sparse_len = 0;
- a->sparse_cap = 0;
- a->props = NULL;
- a->prop_len = 0;
- a->prop_cap = 0;
- a->metadata = NULL;
+ a->ext = NULL;
if (initial_cap > 0) scr_arr_grow_dense(a, initial_cap);
#ifdef SCR_RC_AUDIT
scr_live_arrays++;
@@ -616,36 +630,63 @@ ScrArr *scr_arr_new(ScrElemKind elem, size_t initial_cap) {
return a;
}
+void scr_arr_swap_indexed_ext(ScrArr *a, ScrArr *b) {
+ const ScrArrExt *xa = SCR_AX(a), *xb = SCR_AX(b);
+ if (!xa->sparse && !xa->props && !xb->sparse && !xb->props) return;
+ ScrArrExt *wa = scr_arr_ext(a), *wb = scr_arr_ext(b), t = *wa;
+ wa->sparse = wb->sparse;
+ wa->sparse_len = wb->sparse_len;
+ wa->sparse_cap = wb->sparse_cap;
+ wa->props = wb->props;
+ wa->prop_len = wb->prop_len;
+ wa->prop_cap = wb->prop_cap;
+ wb->sparse = t.sparse;
+ wb->sparse_len = t.sparse_len;
+ wb->sparse_cap = t.sparse_cap;
+ wb->props = t.props;
+ wb->prop_len = t.prop_len;
+ wb->prop_cap = t.prop_cap;
+}
+
/* Collector trace of a cycle-capable array: every element is a headered
* child (elem_trace non-NULL means the element TYPE carries a header, and
* arrays are monomorphic), so trace visits all of them and the teardown
* below releases none — the complement contract in scr_runtime.h. */
void scr_arr_trace_v(void *a0, ScrTraceVisit visit, void *ctx) {
ScrArr *a = (ScrArr *)a0;
- if (a->metadata) visit(a->metadata, ctx);
+ if (SCR_AX(a)->metadata) visit(SCR_AX(a)->metadata, ctx);
/* Growth and truncation leave spare capacity empty. Sparse slots and
* named properties are separate edges, independent of dense length. */
for (size_t i = 0, end = a->len < a->cap ? a->len : a->cap; i < end; i++) {
if (a->present[i] == SCR_ARR_VALUE) visit(scr_slot_to_ptr(a->data[i]), ctx);
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- if (a->sparse[i].state == SCR_ARR_VALUE) {
- visit(scr_slot_to_ptr(a->sparse[i].slot), ctx);
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ if (SCR_AX(a)->sparse[i].state == SCR_ARR_VALUE) {
+ visit(scr_slot_to_ptr(SCR_AX(a)->sparse[i].slot), ctx);
}
}
- for (size_t i = 0; i < a->prop_len; i++) {
- if (a->props[i].state == SCR_ARR_VALUE) {
- visit(scr_slot_to_ptr(a->props[i].slot), ctx);
+ for (size_t i = 0; i < SCR_AX(a)->prop_len; i++) {
+ if (SCR_AX(a)->props[i].state == SCR_ARR_VALUE) {
+ visit(scr_slot_to_ptr(SCR_AX(a)->props[i].slot), ctx);
}
}
}
+/* Frees the out-of-line storage; callers check for it first, since most
+ * arrays have none. */
+static void scr_arr_ext_free(ScrArr *a) {
+ ScrArrExt *x = a->ext;
+ scr_arr_sparse_free(x->sparse);
+ for (size_t i = 0; i < x->prop_len; i++) free(x->props[i].key);
+ free(x->props);
+ scr_mem_free(x);
+ a->ext = NULL;
+}
+
static void scr_arr_gc_free(void *a0) {
ScrArr *a = (ScrArr *)a0;
scr_mem_free(a->data);
- scr_arr_sparse_free(a->sparse);
- for (size_t i = 0; i < a->prop_len; i++) free(a->props[i].key);
- free(a->props);
+ if (a->ext) scr_arr_ext_free(a);
#ifdef SCR_RC_AUDIT
scr_live_arrays--;
#endif
@@ -657,7 +698,8 @@ ScrArr *scr_arr_new_ref(void *(*elem_retain)(void *),
ScrTraceFn elem_trace, size_t initial_cap) {
ScrArr *a;
if (elem_trace) {
- a = scr_cyc_alloc_inline(sizeof(ScrArr), &scr_arr_trace_v, &scr_arr_gc_free);
+ static const ScrCycType cyc_type = { &scr_arr_trace_v, &scr_arr_gc_free, SCR_CYC_KIND_ARRAY };
+ a = scr_cyc_alloc_inline(sizeof(ScrArr), &cyc_type);
} else {
a = scr_mem_alloc(sizeof(ScrArr));
if (!a) scr_arr_oom();
@@ -671,13 +713,7 @@ ScrArr *scr_arr_new_ref(void *(*elem_retain)(void *),
a->elem_trace = elem_trace;
a->data = NULL;
a->present = NULL;
- a->sparse = NULL;
- a->sparse_len = 0;
- a->sparse_cap = 0;
- a->props = NULL;
- a->prop_len = 0;
- a->prop_cap = 0;
- a->metadata = NULL;
+ a->ext = NULL;
if (initial_cap > 0) scr_arr_grow_dense(a, initial_cap);
#ifdef SCR_RC_AUDIT
scr_live_arrays++;
@@ -687,8 +723,10 @@ ScrArr *scr_arr_new_ref(void *(*elem_retain)(void *),
static void scr_arr_destroy(void *object) {
ScrArr *a = object;
- scr_dyn_release(a->metadata);
- a->metadata = NULL;
+ if (a->ext) {
+ scr_dyn_release(a->ext->metadata);
+ a->ext->metadata = NULL;
+ }
if (scr_elem_is_ref(a->elem)) {
for (size_t i = 0, end = a->len < a->cap ? a->len : a->cap; i < end; i++) {
if (a->present[i] != SCR_ARR_HOLE) {
@@ -699,29 +737,29 @@ static void scr_arr_destroy(void *object) {
if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
}
}
- while (a->sparse_len > 0) {
- size_t i = a->sparse_len - 1;
- uint64_t old = a->sparse[i].slot;
- uint8_t state = a->sparse[i].state;
- a->sparse_len = i;
- if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
- }
- while (a->prop_len > 0) {
- size_t i = a->prop_len - 1;
- uint64_t old = a->props[i].slot;
- uint8_t state = a->props[i].state;
- free(a->props[i].key);
- a->prop_len = i;
- if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
+ if (a->ext) {
+ while (a->ext->sparse_len > 0) {
+ size_t i = a->ext->sparse_len - 1;
+ uint64_t old = a->ext->sparse[i].slot;
+ uint8_t state = a->ext->sparse[i].state;
+ a->ext->sparse_len = i;
+ if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
+ }
+ while (a->ext->prop_len > 0) {
+ size_t i = a->ext->prop_len - 1;
+ uint64_t old = a->ext->props[i].slot;
+ uint8_t state = a->ext->props[i].state;
+ free(a->ext->props[i].key);
+ a->ext->prop_len = i;
+ if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
+ }
}
}
if (a->elem_trace) {
scr_arr_gc_free(a);
} else {
scr_mem_free(a->data);
- scr_arr_sparse_free(a->sparse);
- for (size_t i = 0; i < a->prop_len; i++) free(a->props[i].key);
- free(a->props);
+ if (a->ext) scr_arr_ext_free(a);
#ifdef SCR_RC_AUDIT
scr_live_arrays--;
#endif
@@ -768,9 +806,9 @@ double scr_arr_next_present(const ScrArr *a, double start) {
for (; i < end; i++) {
if (a->present[i] != SCR_ARR_HOLE) return (double)i;
}
- size_t pos = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, i);
- if (pos < a->sparse_len && a->sparse[pos].index < a->len) {
- return (double)a->sparse[pos].index;
+ size_t pos = scr_arr_sparse_lower_bound(SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, i);
+ if (pos < SCR_AX(a)->sparse_len && SCR_AX(a)->sparse[pos].index < a->len) {
+ return (double)SCR_AX(a)->sparse[pos].index;
}
return (double)a->len;
}
@@ -803,20 +841,22 @@ void scr_arr_set_len(ScrArr *a, double length) {
if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
}
}
- size_t keep = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, next);
- while (a->sparse_len > keep) {
- size_t i = a->sparse_len - 1;
- uint64_t old = a->sparse[i].slot;
- uint8_t state = a->sparse[i].state;
- a->sparse_len = i;
- if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
+ if (a->ext) {
+ size_t keep = scr_arr_sparse_lower_bound(a->ext->sparse, a->ext->sparse_len, next);
+ while (a->ext->sparse_len > keep) {
+ size_t i = a->ext->sparse_len - 1;
+ uint64_t old = a->ext->sparse[i].slot;
+ uint8_t state = a->ext->sparse[i].state;
+ a->ext->sparse_len = i;
+ if (state == SCR_ARR_VALUE) scr_elem_release(a, old);
+ }
}
} else {
for (size_t i = dense_stop; i < dense_end; i++) {
a->present[i] = 0;
a->data[i] = 0;
}
- a->sparse_len = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, next);
+ if (a->ext) a->ext->sparse_len = scr_arr_sparse_lower_bound(a->ext->sparse, a->ext->sparse_len, next);
}
}
a->len = next;
@@ -862,9 +902,9 @@ void scr_arr_copy_range_ex(ScrArr *dst, size_t dst_start, const ScrArr *src,
scr_arr_copy_index(dst, dst_start + off, src, i);
}
}
- size_t pos = scr_arr_sparse_lower_bound(src->sparse, src->sparse_len, src_start);
- for (; pos < src->sparse_len && src->sparse[pos].index < end; pos++) {
- size_t i = src->sparse[pos].index;
+ size_t pos = scr_arr_sparse_lower_bound(SCR_AX(src)->sparse, SCR_AX(src)->sparse_len, src_start);
+ for (; pos < SCR_AX(src)->sparse_len && SCR_AX(src)->sparse[pos].index < end; pos++) {
+ size_t i = SCR_AX(src)->sparse[pos].index;
size_t off = reverse ? count - 1 - (i - src_start) : i - src_start;
scr_arr_copy_index(dst, dst_start + off, src, i);
}
@@ -1299,9 +1339,9 @@ double scr_arr_concat_copy(ScrArr *a, const ScrArr *src) {
scr_arr_copy_index(a, old_len + i, src, i);
}
}
- for (size_t i = 0; i < src->sparse_len; i++) {
- scr_arr_copy_index(a, old_len + src->sparse[i].index, src,
- src->sparse[i].index);
+ for (size_t i = 0; i < SCR_AX(src)->sparse_len; i++) {
+ scr_arr_copy_index(a, old_len + SCR_AX(src)->sparse[i].index, src,
+ SCR_AX(src)->sparse[i].index);
}
return (double)a->len;
}
@@ -1740,8 +1780,8 @@ double scr_arr_index_of_f64(ScrArr *a, double v) {
for (size_t i = 0; i < a->cap && i < a->len; i++) {
if (a->present[i] == SCR_ARR_VALUE && scr_slot_to_f64(a->data[i]) == v) return (double)i;
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- if (a->sparse[i].state == SCR_ARR_VALUE && scr_slot_to_f64(a->sparse[i].slot) == v) return (double)a->sparse[i].index;
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ if (SCR_AX(a)->sparse[i].state == SCR_ARR_VALUE && scr_slot_to_f64(SCR_AX(a)->sparse[i].slot) == v) return (double)SCR_AX(a)->sparse[i].index;
}
return -1;
}
@@ -1750,8 +1790,8 @@ double scr_arr_index_of_bool(ScrArr *a, bool v) {
for (size_t i = 0; i < a->cap && i < a->len; i++) {
if (a->present[i] == SCR_ARR_VALUE && ((a->data[i] != 0) == v)) return (double)i;
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- if (a->sparse[i].state == SCR_ARR_VALUE && (a->sparse[i].slot != 0) == v) return (double)a->sparse[i].index;
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ if (SCR_AX(a)->sparse[i].state == SCR_ARR_VALUE && (SCR_AX(a)->sparse[i].slot != 0) == v) return (double)SCR_AX(a)->sparse[i].index;
}
return -1;
}
@@ -1760,8 +1800,8 @@ double scr_arr_index_of_ref(ScrArr *a, void *v) {
for (size_t i = 0; i < a->cap && i < a->len; i++) {
if (a->present[i] == SCR_ARR_VALUE && scr_arr_ref_eq(a, a->data[i], v)) return (double)i;
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- if (a->sparse[i].state == SCR_ARR_VALUE && scr_arr_ref_eq(a, a->sparse[i].slot, v)) return (double)a->sparse[i].index;
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ if (SCR_AX(a)->sparse[i].state == SCR_ARR_VALUE && scr_arr_ref_eq(a, SCR_AX(a)->sparse[i].slot, v)) return (double)SCR_AX(a)->sparse[i].index;
}
return -1;
}
@@ -1772,9 +1812,9 @@ bool scr_arr_includes_f64(ScrArr *a, double v) {
double x = scr_slot_to_f64(a->data[i]);
if (x == v || (x != x && v != v)) return true; /* SameValueZero: NaN hits */
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- if (a->sparse[i].state != SCR_ARR_VALUE) continue;
- double x = scr_slot_to_f64(a->sparse[i].slot);
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ if (SCR_AX(a)->sparse[i].state != SCR_ARR_VALUE) continue;
+ double x = scr_slot_to_f64(SCR_AX(a)->sparse[i].slot);
if (x == v || (x != x && v != v)) return true;
}
return false;
@@ -1793,9 +1833,9 @@ bool scr_arr_includes_ref(ScrArr *a, void *v) {
const ScrDyn *d = (const ScrDyn *)scr_slot_to_ptr(a->data[i]);
if (d->kind == SCR_DYN_NUM && d->v.num != d->v.num) return true;
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- if (a->sparse[i].state != SCR_ARR_VALUE) continue;
- const ScrDyn *d = (const ScrDyn *)scr_slot_to_ptr(a->sparse[i].slot);
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ if (SCR_AX(a)->sparse[i].state != SCR_ARR_VALUE) continue;
+ const ScrDyn *d = (const ScrDyn *)scr_slot_to_ptr(SCR_AX(a)->sparse[i].slot);
if (d->kind == SCR_DYN_NUM && d->v.num != d->v.num) return true;
}
return false;
@@ -1862,13 +1902,13 @@ ScrArr *scr_arr_slice(ScrArr *a, double start, double end) {
scr_arr_retain_state_slot(a, a->data[i], state),
state);
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- size_t index = a->sparse[i].index;
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ size_t index = SCR_AX(a)->sparse[i].index;
if (index >= from && index < to) {
scr_arr_store_state_owned(out, index - from,
- scr_arr_retain_state_slot(a, a->sparse[i].slot,
- a->sparse[i].state),
- a->sparse[i].state);
+ scr_arr_retain_state_slot(a, SCR_AX(a)->sparse[i].slot,
+ SCR_AX(a)->sparse[i].state),
+ SCR_AX(a)->sparse[i].state);
}
}
return out;
@@ -1986,9 +2026,9 @@ ScrArr *scr_arr_sort_primitive(ScrArr *a, bool copy) {
count += a->present[i] == SCR_ARR_VALUE;
undefined_count += a->present[i] == SCR_ARR_UNDEFINED;
}
- for (size_t i = 0; i < a->sparse_len; i++) {
- count += a->sparse[i].state == SCR_ARR_VALUE;
- undefined_count += a->sparse[i].state == SCR_ARR_UNDEFINED;
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++) {
+ count += SCR_AX(a)->sparse[i].state == SCR_ARR_VALUE;
+ undefined_count += SCR_AX(a)->sparse[i].state == SCR_ARR_UNDEFINED;
}
if (count > SIZE_MAX / sizeof(uint64_t)) scr_arr_oom();
uint64_t *values = count ? malloc(count * sizeof(*values)) : NULL;
@@ -1996,8 +2036,8 @@ ScrArr *scr_arr_sort_primitive(ScrArr *a, bool copy) {
size_t used = 0;
for (size_t i = 0; i < dense; i++)
if (a->present[i] == SCR_ARR_VALUE) values[used++] = a->data[i];
- for (size_t i = 0; i < a->sparse_len; i++)
- if (a->sparse[i].state == SCR_ARR_VALUE) values[used++] = a->sparse[i].slot;
+ for (size_t i = 0; i < SCR_AX(a)->sparse_len; i++)
+ if (SCR_AX(a)->sparse[i].state == SCR_ARR_VALUE) values[used++] = SCR_AX(a)->sparse[i].slot;
ScrNumberSortKey *numbers = NULL;
if (a->elem == SCR_ELEM_F64 && count > 1) {
if (count > SIZE_MAX / sizeof(*numbers)) scr_arr_oom();
@@ -2225,8 +2265,8 @@ void scr_arr_sort_values(ScrArr *a, double count, ScrClosure *f, uint32_t arity)
if (i < a->cap) {
a->data[i] = src[i];
} else {
- size_t pos = scr_arr_sparse_lower_bound(a->sparse, a->sparse_len, i);
- a->sparse[pos].slot = src[i];
+ size_t pos = scr_arr_sparse_lower_bound(SCR_AX(a)->sparse, SCR_AX(a)->sparse_len, i);
+ SCR_AX(a)->sparse[pos].slot = src[i];
}
}
done:
@@ -2311,9 +2351,9 @@ ScrStr *scr_str_raw(ScrArr *raw, ScrArr *subs) {
}
void scr_arr_copy_metadata(ScrArr *a, ScrDyn *target) {
- if (!a->metadata) return;
- for (size_t i = 0; i < a->metadata->v.obj.len; i++) {
- ScrDynEntry *entry = &a->metadata->v.obj.entries[i];
+ if (!SCR_AX(a)->metadata) return;
+ for (size_t i = 0; i < SCR_AX(a)->metadata->v.obj.len; i++) {
+ ScrDynEntry *entry = &SCR_AX(a)->metadata->v.obj.entries[i];
ScrStr *key = scr_str_new(entry->key, entry->key_len);
scr_dyn_key_set(target, key, entry->value);
scr_str_release(key);
diff --git a/packages/runtime/src/scr_async.c b/packages/runtime/src/scr_async.c
index e78a6043e..37644f705 100644
--- a/packages/runtime/src/scr_async.c
+++ b/packages/runtime/src/scr_async.c
@@ -285,7 +285,8 @@ static void scr_promise_gcfree(void *o) {
}
ScrPromise *scr_promise_new(void) {
- ScrPromise *p = scr_cyc_alloc(sizeof *p, &scr_promise_trace, &scr_promise_gcfree);
+ static const ScrCycType cyc_type = { &scr_promise_trace, &scr_promise_gcfree, SCR_CYC_KIND_PROMISE };
+ ScrPromise *p = scr_cyc_alloc(sizeof *p, &cyc_type);
p->rc = 1;
p->identity = malloc(sizeof *p->identity);
if (!p->identity) scr_oom();
diff --git a/packages/runtime/src/scr_closure.c b/packages/runtime/src/scr_closure.c
index ea74ff6e5..cfccd31a9 100644
--- a/packages/runtime/src/scr_closure.c
+++ b/packages/runtime/src/scr_closure.c
@@ -70,7 +70,8 @@ static void scr_box_gcfree(void *o) {
}
ScrBox *scr_box_new(ScrBoxKind kind) {
- ScrBox *b = scr_cyc_alloc_inline(sizeof(ScrBox), &scr_box_trace, &scr_box_gcfree);
+ static const ScrCycType cyc_type = { &scr_box_trace, &scr_box_gcfree, SCR_CYC_KIND_BOX };
+ ScrBox *b = scr_cyc_alloc_inline(sizeof(ScrBox), &cyc_type);
b->rc = 1;
b->kind = kind;
#ifdef SCR_RC_AUDIT
@@ -126,7 +127,7 @@ void scr_box_release(ScrBox *b) {
if (--b->rc == 0) {
scr_cyc_dead(b);
scr_rc_destroy(b, scr_box_destroy);
- } else {
+ } else if (scr_box_traced(b)) {
scr_cyc_released(b); /* possible cycle root; may collect — b is done */
}
}
@@ -185,8 +186,8 @@ static void scr_closure_gcfree(void *o) {
}
ScrClosure *scr_closure_new(void *fn, size_t ncaps) {
- ScrClosure *c = scr_cyc_alloc_inline(sizeof(ScrClosure) + ncaps * sizeof(ScrBox *),
- &scr_closure_trace, &scr_closure_gcfree);
+ static const ScrCycType cyc_type = { &scr_closure_trace, &scr_closure_gcfree, SCR_CYC_KIND_CLOSURE };
+ ScrClosure *c = scr_cyc_alloc_inline(sizeof(ScrClosure) + ncaps * sizeof(ScrBox *), &cyc_type);
c->rc = 1;
c->fn = fn;
c->ncaps = ncaps;
diff --git a/packages/runtime/src/scr_console.c b/packages/runtime/src/scr_console.c
index 6d3dc0163..bc558318d 100644
--- a/packages/runtime/src/scr_console.c
+++ b/packages/runtime/src/scr_console.c
@@ -125,7 +125,7 @@ static void scr_enable_utf8_console_output(void) {
}
#endif
-void scr_runtime_abi_v9(void) {}
+void scr_runtime_abi_v12(void) {}
void scr_init(void) {
#ifdef SCR_WORKERS
diff --git a/packages/runtime/src/scr_cycle.c b/packages/runtime/src/scr_cycle.c
index 0f2663121..46e74659c 100644
--- a/packages/runtime/src/scr_cycle.c
+++ b/packages/runtime/src/scr_cycle.c
@@ -133,6 +133,7 @@ void scr_weak_dispose(void *object) {
#include
#include
+#include
/* Every heap object's first member is `size_t rc`. */
#define SCR_RC(obj) (*(size_t *)(obj))
@@ -188,6 +189,9 @@ typedef struct {
size_t trigger;
size_t scheduled_mature_age;
size_t xg_freed;
+ struct ScrCycKind *kinds;
+ size_t kinds_cap;
+ unsigned bypass_tick;
} ScrCycThread;
static _Thread_local ScrCycThread scr_cyc_thread = {.gen_limit = SCR_CYC_OLD};
#define scr_destroy_depth (scr_cyc_thread.destroy_depth)
@@ -211,6 +215,9 @@ static _Thread_local ScrCycThread scr_cyc_thread = {.gen_limit = SCR_CYC_OLD};
#define scr_cyc_trigger (scr_cyc_thread.trigger)
#define scr_cyc_scheduled_mature_age (scr_cyc_thread.scheduled_mature_age)
#define scr_xg_freed (scr_cyc_thread.xg_freed)
+#define scr_cyc_kinds (scr_cyc_thread.kinds)
+#define scr_cyc_kinds_cap (scr_cyc_thread.kinds_cap)
+#define scr_cyc_bypass_tick (scr_cyc_thread.bypass_tick)
#endif
#ifndef SCR_WORKERS
static SCR_TL unsigned scr_destroy_depth;
@@ -259,13 +266,15 @@ SCR_TL size_t scr_cyc_live = 0;
SCR_TL size_t scr_cyc_old_freed = 0;
#endif
-void *scr_cyc_alloc(size_t size, ScrTraceFn trace, ScrCycFreeFn free_fn) {
- return scr_cyc_alloc_inline(size, trace, free_fn);
+void *scr_cyc_alloc(size_t size, const ScrCycType *type) {
+ return scr_cyc_alloc_inline(size, type);
}
void scr_cyc_free(void *obj) { scr_cyc_free_inline(obj); }
static void scr_cyc_grow(ScrVec *vec) {
+ /* Candidate positions are 32-bit (ScrCycHdr.buf_index). */
+ if (vec->cap > (size_t)UINT32_MAX / 2) scr_cyc_oom();
vec->cap = vec->cap ? vec->cap * 2 : 64;
void **grown = realloc(vec->v, vec->cap * sizeof *grown);
if (!grown) scr_cyc_oom();
@@ -584,6 +593,108 @@ void scr_cyc_collect_scheduled(void) {
scr_cyc_collect_due();
}
+/* ── level bypass ─────────────────────────────────────────────────────── */
+
+/* A nursery pass costs about the same for every candidate it walks, and
+ * nearly every long-lived object is a candidate once, soon after it is
+ * built: the release of the constructing frame's reference leaves it held
+ * only by the structure it was stored into. So the nursery walks almost the
+ * whole long-lived heap once, then re-buffers it for the mature level, whose
+ * next pass walks it again. For most types neither walk finds anything: a
+ * compiler's symbols, symbol links and syntax nodes survive essentially
+ * always, and in tsc-ts they are half of what nursery passes walk. Garbage
+ * cycles in the younger levels come from a few types (inference contexts
+ * and their mappers, closures, some arrays).
+ *
+ * So each thread keeps, per collector type (ScrCycType.kind) and for the
+ * nursery and mature levels, how many of the type's candidates passes at
+ * that level walked and how many of its objects they freed. After every
+ * BYPASS_WINDOW walked candidates, the type bypasses that level for the
+ * next window if its objects were freed in fewer than one in BYPASS_RATIO
+ * of them. A release that buffers an object of a bypassed level promotes
+ * it past the level first, and so does a nursery pass that re-buffers a
+ * survivor into a level the type bypasses: the candidate waits in an older
+ * buffer, and passes of the bypassed level stop at it as they stop at any
+ * older object. One in BYPASS_SAMPLE such candidates stays put, so a
+ * bypassed type keeps being measured, and a type whose objects start dying
+ * at that level again is walked there again after one window.
+ *
+ * This changes only which generation an object is judged in. Any
+ * generation assignment is sound (a restricted pass treats edges from older
+ * objects as external), and a bypassing candidate is buffered like any
+ * other, so a dead cycle through it keeps a buffered member exactly as a
+ * survivor re-buffered into the next generation does. Full passes walk
+ * everything they did before, so the old level's bounds above are
+ * unchanged; garbage cycles made only of objects of a bypassing type wait
+ * for the next pass at an older level, by construction a type whose
+ * candidates almost never die at the bypassed one. Small programs never
+ * complete a window and are collected exactly as before. */
+#define SCR_CYC_BYPASS_WINDOW 8192 /* walked candidates per decision */
+#define SCR_CYC_BYPASS_RATIO 128 /* bypass when freed < walked / RATIO */
+#define SCR_CYC_BYPASS_SAMPLE 16 /* a bypassed type still walks 1 in SAMPLE */
+
+/* One level's statistics for a type. */
+typedef struct {
+ uint32_t walked; /* candidates walked by passes at this level, this window */
+ uint32_t freed; /* objects freed by passes at this level, this window */
+ uint32_t bypass; /* the last window's decision */
+} ScrCycLevelStat;
+typedef struct ScrCycKind {
+ ScrCycLevelStat level[2]; /* SCR_CYC_NURSERY, SCR_CYC_MATURE */
+} ScrCycKind;
+
+#ifndef SCR_WORKERS
+static SCR_TL ScrCycKind *scr_cyc_kinds;
+static SCR_TL size_t scr_cyc_kinds_cap;
+static SCR_TL unsigned scr_cyc_bypass_tick;
+#endif
+
+/* The statistics of `kind` (nonzero), growing the table to cover it. */
+static SCR_CYC_COMPACT ScrCycKind *scr_cyc_kind_grow(uint32_t kind) {
+ size_t cap = scr_cyc_kinds_cap ? scr_cyc_kinds_cap : 256;
+ while (cap <= kind) cap *= 2;
+ ScrCycKind *kinds = realloc(scr_cyc_kinds, cap * sizeof *kinds);
+ if (!kinds) scr_cyc_oom();
+ memset(kinds + scr_cyc_kinds_cap, 0, (cap - scr_cyc_kinds_cap) * sizeof *kinds);
+ scr_cyc_kinds = kinds;
+ scr_cyc_kinds_cap = cap;
+ return &kinds[kind];
+}
+static inline ScrCycKind *scr_cyc_kind(uint32_t kind) {
+ return kind < scr_cyc_kinds_cap ? &scr_cyc_kinds[kind] : scr_cyc_kind_grow(kind);
+}
+
+/* Whether a candidate of this type at level `gen` (nursery or mature)
+ * skips that level's passes (see above). */
+static inline bool scr_cyc_bypassed(const ScrCycType *type, unsigned gen) {
+ uint32_t kind = type->kind;
+ return kind < scr_cyc_kinds_cap && scr_cyc_kinds[kind].level[gen].bypass
+ && ++scr_cyc_bypass_tick % SCR_CYC_BYPASS_SAMPLE != 0;
+}
+
+/* The generation a release buffers an object into: its own, or past the
+ * levels its type bypasses. */
+static inline unsigned scr_cyc_buffer_gen(const ScrCycType *type, unsigned gen) {
+ if (gen == SCR_CYC_NURSERY && scr_cyc_bypassed(type, SCR_CYC_NURSERY)) gen = SCR_CYC_MATURE;
+ if (gen == SCR_CYC_MATURE && scr_cyc_bypassed(type, SCR_CYC_MATURE)) gen = SCR_CYC_OLD;
+ return gen;
+}
+
+/* A pass at `gen` (nursery or mature) walks a candidate of `type`: count
+ * it, and decide the next window when this one is full. */
+static inline void scr_cyc_walked(const ScrCycType *type, unsigned gen) {
+ if (!type->kind) return;
+ ScrCycLevelStat *k = &scr_cyc_kind(type->kind)->level[gen];
+ if (++k->walked == SCR_CYC_BYPASS_WINDOW) {
+ k->bypass = k->freed * SCR_CYC_BYPASS_RATIO < SCR_CYC_BYPASS_WINDOW;
+ k->walked = k->freed = 0;
+ }
+}
+/* A pass at `gen` (nursery or mature) frees an object of `type`. */
+static inline void scr_cyc_freed(const ScrCycType *type, unsigned gen) {
+ if (type->kind) scr_cyc_kind(type->kind)->level[gen].freed++;
+}
+
void scr_cyc_on_dead(void *obj) {
ScrCycHdr *h = scr_cyc_hdr(obj);
if (!h->buffered) return;
@@ -594,7 +705,7 @@ void scr_cyc_on_dead(void *obj) {
size_t i = h->buf_index;
void *last = b->v[--b->n];
b->v[i] = last;
- if (last != obj) scr_cyc_hdr(last)->buf_index = i;
+ if (last != obj) scr_cyc_hdr(last)->buf_index = (uint32_t)i;
h->buffered = 0;
scr_cyc_nbuffered--;
/* A death must not consume room reserved for a future candidate. */
@@ -611,10 +722,11 @@ void scr_cyc_on_release(void *obj) {
/* Releasing an existing candidate cannot advance the trigger. A pass
* always re-arms after its teardowns, including any candidates they add. */
if (h->buffered) return;
+ if (h->gen < SCR_CYC_OLD) h->gen = (uint8_t)scr_cyc_buffer_gen(h->type, h->gen);
ScrVec *b = &scr_roots[h->gen];
if (b->n == b->cap) scr_cyc_grow(b);
h->buffered = 1;
- h->buf_index = b->n;
+ h->buf_index = (uint32_t)b->n;
b->v[b->n++] = obj;
scr_cyc_nbuffered++;
/* Never re-entered: a teardown's releases of untraced children can buffer
@@ -650,7 +762,7 @@ static void scr_mark_gray(void *obj, uintptr_t depth) {
scr_cyc_push(&scr_pending, obj);
return;
}
- h->trace(obj, scr_mg_visit, (void *)(depth + 1));
+ h->type->trace(obj, scr_mg_visit, (void *)(depth + 1));
}
static void scr_scan_black(void *obj, uintptr_t depth);
@@ -668,7 +780,7 @@ static void scr_scan_black(void *obj, uintptr_t depth) {
scr_cyc_push(&scr_restore_pending, obj);
return;
}
- h->trace(obj, scr_sb_visit, (void *)(depth + 1));
+ h->type->trace(obj, scr_sb_visit, (void *)(depth + 1));
}
static void scr_scan(void *obj, uintptr_t depth);
@@ -683,7 +795,7 @@ static void scr_scan(void *obj, uintptr_t depth) {
scr_scan_black(obj, 0);
while (scr_restore_pending.n) {
void *next = scr_restore_pending.v[--scr_restore_pending.n];
- scr_cyc_hdr(next)->trace(next, scr_sb_visit, NULL);
+ scr_cyc_hdr(next)->type->trace(next, scr_sb_visit, NULL);
}
return;
}
@@ -695,7 +807,7 @@ static void scr_scan(void *obj, uintptr_t depth) {
scr_cyc_push(&scr_pending, obj);
return;
}
- h->trace(obj, scr_scan_visit, (void *)(depth + 1));
+ h->type->trace(obj, scr_scan_visit, (void *)(depth + 1));
}
/* ── cross-generation edges ───────────────────────────────────────────── */
@@ -752,8 +864,8 @@ static void scr_xg_child_visit(void *child, void *ctx) {
/* One genuine release, generically: the header carries everything needed. */
static void scr_xg_destroy(void *obj) {
ScrCycHdr *h = scr_cyc_hdr(obj);
- h->trace(obj, scr_xg_child_visit, NULL);
- h->free_fn(obj);
+ h->type->trace(obj, scr_xg_child_visit, NULL);
+ h->type->free_fn(obj);
scr_xg_freed++;
}
@@ -782,6 +894,7 @@ static SCR_CYC_COMPACT size_t scr_cyc_settle(unsigned gen_limit) {
ScrCycHdr *h = scr_cyc_hdr(obj);
if (h->color != SCR_CYC_WHITE) continue;
h->color = SCR_CYC_DOOMED;
+ if (gen_limit < SCR_CYC_OLD) scr_cyc_freed(h->type, gen_limit);
scr_white.v[dead++] = obj;
}
scr_white.n = dead;
@@ -793,7 +906,7 @@ static SCR_CYC_COMPACT size_t scr_cyc_settle(unsigned gen_limit) {
if (gen_limit < SCR_CYC_OLD) {
for (size_t i = 0; i < scr_white.n; i++) {
void *obj = scr_white.v[i];
- scr_cyc_hdr(obj)->trace(obj, scr_xg_pin_visit, NULL);
+ scr_cyc_hdr(obj)->type->trace(obj, scr_xg_pin_visit, NULL);
}
}
@@ -827,8 +940,12 @@ static SCR_CYC_COMPACT size_t scr_cyc_settle(unsigned gen_limit) {
ScrCycHdr *h = scr_cyc_hdr(obj);
if (h->color == SCR_CYC_DOOMED) continue; /* being freed below */
if (!h->buffered) {
+ /* A survivor promoted into a level its type bypasses moves past
+ * it, as a release would buffer it. */
+ if (h->gen == SCR_CYC_MATURE && scr_cyc_bypassed(h->type, SCR_CYC_MATURE))
+ h->gen = SCR_CYC_OLD;
h->buffered = 1;
- h->buf_index = scr_roots[h->gen].n;
+ h->buf_index = (uint32_t)scr_roots[h->gen].n;
scr_cyc_push(&scr_roots[h->gen], obj);
}
}
@@ -840,7 +957,7 @@ static SCR_CYC_COMPACT size_t scr_cyc_settle(unsigned gen_limit) {
size_t freed = scr_white.n;
for (size_t i = 0; i < scr_white.n; i++) {
void *obj = scr_white.v[i];
- scr_cyc_hdr(obj)->free_fn(obj);
+ scr_cyc_hdr(obj)->type->free_fn(obj);
}
scr_white.n = 0;
@@ -889,6 +1006,7 @@ static size_t scr_cyc_pass(unsigned gen_limit) {
void *obj = scr_roots[g].v[i];
ScrCycHdr *h = scr_cyc_hdr(obj);
h->buffered = 0;
+ if (gen_limit < SCR_CYC_OLD) scr_cyc_walked(h->type, gen_limit);
scr_cyc_push(&scr_cands, obj);
}
scr_roots[g].n = 0;
@@ -896,7 +1014,7 @@ static size_t scr_cyc_pass(unsigned gen_limit) {
for (size_t i = 0; i < scr_cands.n; i++) scr_mark_gray(scr_cands.v[i], 0);
while (scr_pending.n) {
void *obj = scr_pending.v[--scr_pending.n];
- scr_cyc_hdr(obj)->trace(obj, scr_mg_visit, NULL);
+ scr_cyc_hdr(obj)->type->trace(obj, scr_mg_visit, NULL);
}
scr_white.n = 0;
@@ -906,12 +1024,17 @@ static size_t scr_cyc_pass(unsigned gen_limit) {
/* A later outside root may have restored this deferred white node and
* its descendants already. Never scan a black node's edges again. */
if (scr_cyc_hdr(obj)->color == SCR_CYC_WHITE)
- scr_cyc_hdr(obj)->trace(obj, scr_scan_visit, NULL);
+ scr_cyc_hdr(obj)->type->trace(obj, scr_scan_visit, NULL);
}
return scr_cyc_settle(gen_limit);
}
+#ifdef SCR_CYC_TESTING
+/* Tests only: one pass at exactly `gen`, outside the schedule. */
+size_t scr_cyc_pass_at(unsigned gen) { return scr_cyc_pass(gen); }
+#endif
+
void scr_collect_cycles(void) {
/* The full pass, run to a fixpoint. A teardown's releases can drop the
* last reference to a further cycle, so one pass does not always drain
@@ -935,4 +1058,7 @@ void scr_cyc_context_cleanup(void) {
free(vectors[i]->v);
*vectors[i] = (ScrVec){0};
}
+ free(scr_cyc_kinds);
+ scr_cyc_kinds = NULL;
+ scr_cyc_kinds_cap = 0;
}
diff --git a/packages/runtime/src/scr_error.c b/packages/runtime/src/scr_error.c
index 34b4958f4..f2dee0383 100644
--- a/packages/runtime/src/scr_error.c
+++ b/packages/runtime/src/scr_error.c
@@ -170,10 +170,11 @@ ScrTraceFn scr_error_trace_arg(void) {
* slots' defaults (code 0, no cause). */
static ScrError *scr_error_alloc(int kind) {
size_t size = kind == SCR_ERR_DOMEX ? sizeof(ScrDomException) : sizeof(ScrError);
- void (*gcfree)(void *) = kind == SCR_ERR_DOMEX ? &scr_domex_gcfree : &scr_error_gcfree;
+ static const ScrCycType error_type = { &scr_error_trace, &scr_error_gcfree, SCR_CYC_KIND_ERROR };
+ static const ScrCycType domex_type = { &scr_error_trace, &scr_domex_gcfree, SCR_CYC_KIND_DOMEXCEPTION };
ScrError *e;
if (scr_error_traced) {
- e = scr_cyc_alloc(size, &scr_error_trace, gcfree);
+ e = scr_cyc_alloc(size, kind == SCR_ERR_DOMEX ? &domex_type : &error_type);
} else {
e = calloc(1, size);
if (!e) scr_error_oom();
diff --git a/packages/runtime/src/scr_events_emitter.c b/packages/runtime/src/scr_events_emitter.c
index ce47f7368..c95c1722d 100644
--- a/packages/runtime/src/scr_events_emitter.c
+++ b/packages/runtime/src/scr_events_emitter.c
@@ -284,7 +284,8 @@ ScrEmitter *scr_emitter_new(void) {
/* Always collector-headered: the registry holds closures, so the
* emitter hierarchy is unconditionally cycle-capable (the canonical
* capture cycle is `e.on('x', () => e.emit(...))`). */
- ScrEmitter *em = scr_cyc_alloc(sizeof *em, &scr_emitter_trace, &scr_emitter_gcfree);
+ static const ScrCycType cyc_type = { &scr_emitter_trace, &scr_emitter_gcfree, SCR_CYC_KIND_EMITTER };
+ ScrEmitter *em = scr_cyc_alloc(sizeof *em, &cyc_type);
em->rc = 1;
em->vt = &scr_emitter_vt;
em->reg = NULL;
diff --git a/packages/runtime/src/scr_exception.c b/packages/runtime/src/scr_exception.c
index e95af8270..ffc88b53e 100644
--- a/packages/runtime/src/scr_exception.c
+++ b/packages/runtime/src/scr_exception.c
@@ -282,7 +282,8 @@ static void scr_caught_gcfree(void *object) {
}
ScrCaught *scr_exc_take(void) {
- ScrCaught *c = scr_cyc_alloc(sizeof *c, scr_caught_trace_v, scr_caught_gcfree);
+ static const ScrCycType cyc_type = { &scr_caught_trace_v, &scr_caught_gcfree, SCR_CYC_KIND_CAUGHT };
+ ScrCaught *c = scr_cyc_alloc(sizeof *c, &cyc_type);
c->rc = 1;
c->kind = scr_exc_kind;
c->f64 = scr_exc_f64;
diff --git a/packages/runtime/src/scr_json.c b/packages/runtime/src/scr_json.c
index fe3c9daf3..f5b96be25 100644
--- a/packages/runtime/src/scr_json.c
+++ b/packages/runtime/src/scr_json.c
@@ -787,7 +787,8 @@ static ScrDyn *scr_dyn_alloc(ScrDynKind kind) {
return d;
}
#endif
- ScrDyn *fresh = scr_cyc_alloc_inline(sizeof *fresh, &scr_dyn_trace_v, &scr_dyn_gcfree);
+ static const ScrCycType cyc_type = { &scr_dyn_trace_v, &scr_dyn_gcfree, SCR_CYC_KIND_DYN };
+ ScrDyn *fresh = scr_cyc_alloc_inline(sizeof *fresh, &cyc_type);
if (!fresh) scr_json_oom();
fresh->rc = 1;
fresh->kind = kind;
@@ -9800,7 +9801,8 @@ static ScrDyn *scr_weak_collection_new(ScrDyn *seed, bool is_set) {
ScrDynHandleTag tag = is_set ? SCR_DYNH_WEAK_SET : SCR_DYNH_WEAK_MAP;
scr_weak_dispose_hook = &scr_weak_disposed;
scr_dyn_handle_install(tag, is_set ? &set_ops : &map_ops);
- ScrWeakMap *map = scr_cyc_alloc(sizeof *map, scr_weak_map_trace, scr_weak_map_gcfree);
+ static const ScrCycType cyc_type = { &scr_weak_map_trace, &scr_weak_map_gcfree, SCR_CYC_KIND_WEAK_MAP };
+ ScrWeakMap *map = scr_cyc_alloc(sizeof *map, &cyc_type);
if (!map) scr_json_oom();
map->rc = 1;
map->is_set = is_set;
diff --git a/packages/runtime/src/scr_map.c b/packages/runtime/src/scr_map.c
index 804396c3f..7bfa805ca 100644
--- a/packages/runtime/src/scr_map.c
+++ b/packages/runtime/src/scr_map.c
@@ -674,7 +674,8 @@ ScrMap *scr_map_new_typed(ScrMapKeyKind key_kind, ScrMapValKind val_kind,
ScrMap *m;
if (key_trace || val_trace) {
/* Either side can point back: collector header + trace. */
- m = scr_cyc_alloc(sizeof *m, &scr_map_trace, &scr_map_gcfree);
+ static const ScrCycType cyc_type = { &scr_map_trace, &scr_map_gcfree, SCR_CYC_KIND_MAP };
+ m = scr_cyc_alloc(sizeof *m, &cyc_type);
} else {
m = calloc(1, sizeof *m);
if (!m) scr_map_oom();
diff --git a/packages/runtime/src/scr_object.c b/packages/runtime/src/scr_object.c
index d49c1f335..eb1bac8fb 100644
--- a/packages/runtime/src/scr_object.c
+++ b/packages/runtime/src/scr_object.c
@@ -17,8 +17,8 @@ static void scr_classobj_gcfree(void *object) {
ScrClassObj *scr_classobj_new(const ScrClassObj *template, size_t ncaps) {
if (ncaps > (SIZE_MAX - sizeof(ScrClassObj)) / sizeof(ScrBox *))
scr_trap("scriptc: class capture allocation overflow\n");
- ScrClassObj *c = scr_cyc_alloc(sizeof(ScrClassObj) + ncaps * sizeof(ScrBox *),
- &scr_classobj_trace_v, &scr_classobj_gcfree);
+ static const ScrCycType cyc_type = { &scr_classobj_trace_v, &scr_classobj_gcfree, SCR_CYC_KIND_CLASS_OBJECT };
+ ScrClassObj *c = scr_cyc_alloc(sizeof(ScrClassObj) + ncaps * sizeof(ScrBox *), &cyc_type);
c->rc = 1;
c->pre = template->pre;
c->post = template->post;
diff --git a/packages/runtime/src/scr_publish.c b/packages/runtime/src/scr_publish.c
index 7ef5e50e9..d48a348f5 100644
--- a/packages/runtime/src/scr_publish.c
+++ b/packages/runtime/src/scr_publish.c
@@ -114,7 +114,7 @@ static void scr_pub_slot(ScrPub *ctx, uint64_t slot, ScrPubFn fn) {
void scr_pub_arr(void *v, ScrPub *ctx, ScrPubFn elem) {
ScrArr *a = v;
- if (a->props || a->metadata) {
+ if (a->ext && (a->ext->props || a->ext->metadata)) {
scr_pub_refuse(ctx, "Cannot publish an array with non-index properties");
return;
}
@@ -124,8 +124,9 @@ void scr_pub_arr(void *v, ScrPub *ctx, ScrPubFn elem) {
size_t dense = a->len < a->cap ? a->len : a->cap;
for (size_t i = 0; i < dense; i++)
if (!a->present || a->present[i]) scr_pub_slot(ctx, a->data[i], fn);
- for (size_t i = 0; i < a->sparse_len; i++)
- if (a->sparse[i].state) scr_pub_slot(ctx, a->sparse[i].slot, fn);
+ if (a->ext)
+ for (size_t i = 0; i < a->ext->sparse_len; i++)
+ if (a->ext->sparse[i].state) scr_pub_slot(ctx, a->ext->sparse[i].slot, fn);
}
/* publish() of a scalar Node.js refuses (a Date): nothing to walk. */
diff --git a/packages/runtime/src/scr_regex.c b/packages/runtime/src/scr_regex.c
index e86ed3046..dd6f40f44 100644
--- a/packages/runtime/src/scr_regex.c
+++ b/packages/runtime/src/scr_regex.c
@@ -609,9 +609,10 @@ ScrArr *scr_regex_exec(ScrStr *s, ScrRegex *re) {
if (rc == 1) {
int count = lre_get_capture_count(bc);
out = scr_arr_new(SCR_ELEM_STR, (size_t)count);
- out->metadata = scr_dyn_new_obj();
- scr_dyn_obj_set(out->metadata, "index", 5, scr_dyn_new_num((double)scr_capture_index(&input, capture[0])));
- scr_dyn_obj_set(out->metadata, "input", 5, scr_dyn_new_str(s));
+ ScrDyn *metadata = scr_dyn_new_obj();
+ scr_arr_ext(out)->metadata = metadata;
+ scr_dyn_obj_set(metadata, "index", 5, scr_dyn_new_num((double)scr_capture_index(&input, capture[0])));
+ scr_dyn_obj_set(metadata, "input", 5, scr_dyn_new_str(s));
for (int k = 0; k < count; k++) {
const uint8_t *cs = capture[2 * k], *ce = capture[2 * k + 1];
if (cs == NULL || ce == NULL) {
diff --git a/packages/runtime/src/scr_runtime.h b/packages/runtime/src/scr_runtime.h
index 80be65b29..7b8544211 100644
--- a/packages/runtime/src/scr_runtime.h
+++ b/packages/runtime/src/scr_runtime.h
@@ -62,7 +62,7 @@ void scr_init(void);
/* Program objects emitted by the bundled LLVM helper reference this symbol.
* Its versioned spelling makes a mismatched manual runtime link fail before
* the program can start. */
-void scr_runtime_abi_v9(void);
+void scr_runtime_abi_v12(void);
void scr_runtime_workers_v8(void);
/* ── the trap funnel (scr_console.c; scr_library.c under -DSCR_LIB) ──────
@@ -435,49 +435,83 @@ enum { SCR_CYC_BLACK = 0, SCR_CYC_GRAY = 1, SCR_CYC_WHITE = 2, SCR_CYC_DOOMED =
* SCR_CYC_OLD is a full pass. */
enum { SCR_CYC_NURSERY = 0, SCR_CYC_MATURE = 1, SCR_CYC_OLD = 2, SCR_CYC_NGENS = 3 };
-typedef struct ScrCycHdr {
+/* A headered type's collector entry points, shared by all its objects: the
+ * header names one constant descriptor instead of carrying both function
+ * pointers, which keeps the header at 16 bytes. Emitted shapes define one
+ * per traced type (llvm/shapes.ts); runtime families use SCR_CYC_TYPE. */
+typedef struct ScrCycType {
ScrTraceFn trace;
ScrCycFreeFn free_fn;
+ /* Index of the type's per-thread collector statistics (scr_cycle.c's
+ * nursery bypass): a runtime family's SCR_CYC_KIND_*, or for emitted
+ * descriptors their number counting from SCR_CYC_KIND_EMITTED. 0 keeps
+ * no statistics. Statistics only steer
+ * when a candidate is walked, never what a pass frees, so two types that
+ * share an index (separately emitted code in one process) are merely
+ * judged together. */
+ uint32_t kind;
+} ScrCycType;
+/* Runtime families' kinds. Emitted descriptors count from
+ * SCR_CYC_KIND_EMITTED (llvm/alloc.ts FIRST_EMITTED_CYC_KIND). */
+enum {
+ SCR_CYC_KIND_NONE = 0,
+ SCR_CYC_KIND_ARRAY,
+ SCR_CYC_KIND_PROMISE,
+ SCR_CYC_KIND_BOX,
+ SCR_CYC_KIND_CLOSURE,
+ SCR_CYC_KIND_ERROR,
+ SCR_CYC_KIND_DOMEXCEPTION,
+ SCR_CYC_KIND_EMITTER,
+ SCR_CYC_KIND_CAUGHT,
+ SCR_CYC_KIND_DYN,
+ SCR_CYC_KIND_WEAK_MAP,
+ SCR_CYC_KIND_MAP,
+ SCR_CYC_KIND_CLASS_OBJECT,
+ SCR_CYC_KIND_STREAM,
+ SCR_CYC_KIND_UNION,
+ SCR_CYC_KIND_WORKER_VIEW,
+ SCR_CYC_KIND_EMITTED = 32
+};
+#define SCR_CYC_TYPE(name, trace_fn, free) static const ScrCycType name = { (trace_fn), (free), 0 }
+
+typedef struct ScrCycHdr {
#if UINTPTR_MAX == UINT32_MAX
/* calloc aligns the block for doubles; keep the payload aligned too.
- * Padding BEFORE color preserves the emitted obj-8 `buffered` offset. */
+ * Padding BEFORE the descriptor keeps every object-relative offset of the
+ * fields below identical to 64-bit targets. */
uint32_t payload_alignment;
#endif
- uint32_t color; /* SCR_CYC_* */
- uint16_t buffered; /* 1 = sitting in its generation's candidate buffer */
- uint16_t gen; /* SCR_CYC_NURSERY..SCR_CYC_OLD (the walk filter) */
- size_t buf_index; /* position there (O(1) removal when rc hits 0) */
+ const ScrCycType *type; /* trace and teardown */
+ uint8_t color; /* SCR_CYC_* */
+ uint8_t gen; /* SCR_CYC_NURSERY..SCR_CYC_OLD (the walk filter) */
+ uint16_t buffered; /* 1 = sitting in its generation's candidate buffer */
+ uint32_t buf_index; /* position there (O(1) removal when rc hits 0) */
} ScrCycHdr;
/* This layout is an ABI, not an implementation detail. Every emitted release
* of a headered object (llvm/shapes.ts cycleRootLines, used by the record,
* class and inline RC helpers) mirrors scr_cyc_on_release's early return:
- * it reads `buffered` as an i16 at obj-12 on 64-bit targets and obj-8 on
- * wasm32 (the backend's cycleColorOffset - 4) and calls scr_cyc_on_release
- * only when it is 0. Hence the target-width assertions. */
-#if UINTPTR_MAX == UINT64_MAX
-_Static_assert(sizeof(ScrCycHdr) == 32, "LLVM backend expects a 32-byte cycle header");
-_Static_assert(offsetof(ScrCycHdr, color) == 16,
- "LLVM backend's cycleColorOffset is obj-16 on 64-bit targets");
-_Static_assert(offsetof(ScrCycHdr, gen) == 22,
- "LLVM backend's inline free reads the generation at header+22");
-#elif UINTPTR_MAX == UINT32_MAX
-_Static_assert(sizeof(ScrCycHdr) == 24, "wasm32 cycle payloads require an aligned 24-byte header");
-_Static_assert(offsetof(ScrCycHdr, color) == 12,
- "LLVM backend's cycleColorOffset is obj-12 on wasm32");
-#endif
+ * it reads `buffered` as an i16 at obj-6 (the backend's cycleBufferedOffset)
+ * and calls scr_cyc_on_release only when it is 0; the emitted inline free
+ * reads the generation as an i8 at obj-7. Hence the assertions. */
+_Static_assert(sizeof(ScrCycHdr) == 16, "LLVM backend expects a 16-byte cycle header");
+_Static_assert(sizeof(ScrCycHdr) - offsetof(ScrCycHdr, buffered) == 6 &&
+ sizeof(((ScrCycHdr *)0)->buffered) == 2,
+ "the emitted release fast path reads `buffered` as an i16 at obj-6");
+_Static_assert(sizeof(ScrCycHdr) - offsetof(ScrCycHdr, gen) == 7,
+ "LLVM backend's inline free reads the generation as an i8 at obj-7");
+_Static_assert(sizeof(ScrCycHdr) - offsetof(ScrCycHdr, type) == 16 ||
+ UINTPTR_MAX == UINT32_MAX,
+ "LLVM backend stores the descriptor at the block start on 64-bit targets");
_Static_assert(sizeof(ScrCycHdr) % _Alignof(double) == 0,
"cycle-headered payloads must preserve double alignment");
-_Static_assert(offsetof(ScrCycHdr, buffered) == offsetof(ScrCycHdr, color) + 4 &&
- sizeof(((ScrCycHdr *)0)->buffered) == 2,
- "the emitted release fast path reads `buffered` as an i16 four bytes "
- "after color");
static inline ScrCycHdr *scr_cyc_hdr(void *obj) { return (ScrCycHdr *)obj - 1; }
/* Zeroed allocation with a cycle header in front; returns the OBJECT
- * pointer (header at scr_cyc_hdr). Aborts on OOM. */
-void *scr_cyc_alloc(size_t size, ScrTraceFn trace, ScrCycFreeFn free_fn);
+ * pointer (header at scr_cyc_hdr). `type` must outlive the object (a
+ * static descriptor). Aborts on OOM. */
+void *scr_cyc_alloc(size_t size, const ScrCycType *type);
void scr_cyc_free(void *obj); /* frees the block, header included */
/* ── small-object allocator (scr_alloc.c) ───────────────────────────────
@@ -672,12 +706,11 @@ extern SCR_TL size_t scr_cyc_live;
extern SCR_TL size_t scr_cyc_old_freed;
extern SCR_TL void (*scr_weak_dispose_hook)(void *);
#endif
-static inline void *scr_cyc_alloc_inline(size_t size, ScrTraceFn trace, ScrCycFreeFn free_fn) {
+static inline void *scr_cyc_alloc_inline(size_t size, const ScrCycType *type) {
if (size > SIZE_MAX - sizeof(ScrCycHdr)) scr_trap("scriptc: out of memory\n");
ScrCycHdr *h = (ScrCycHdr *)scr_mem_calloc(sizeof(ScrCycHdr) + size);
if (!h) scr_trap("scriptc: out of memory\n");
- h->trace = trace;
- h->free_fn = free_fn;
+ h->type = type;
scr_cyc_live++;
return h + 1;
}
@@ -688,8 +721,8 @@ static inline void scr_cyc_free_inline(void *obj) {
scr_mem_free(scr_cyc_hdr(obj));
}
_Static_assert(SCR_CYC_NURSERY == 0, "zeroed cycle headers start in the nursery");
-_Static_assert(sizeof(((ScrCycHdr *)0)->gen) == 2 && SCR_CYC_OLD == 2,
- "LLVM backend's inline free compares an i16 generation against 2");
+_Static_assert(sizeof(((ScrCycHdr *)0)->gen) == 1 && SCR_CYC_OLD == 2,
+ "LLVM backend's inline free compares an i8 generation against 2");
/* Dispose an object whose reference count already reached zero. The caller
* removes its cycle candidate first. Nested disposals keep bounded stack
@@ -1560,6 +1593,13 @@ typedef struct ScrArr {
ScrTraceFn elem_trace;
uint64_t *data;
uint8_t *present; /* one state per dense slot; heap arrays share data allocation */
+ /* Rarely used storage (sparse indices, noncanonical numeric properties,
+ * match-result metadata) lives out of line: NULL until first needed, so
+ * the common dense array stays small. */
+ struct ScrArrExt *ext;
+} ScrArr;
+
+typedef struct ScrArrExt {
struct ScrArrSparseSlot *sparse;
size_t sparse_len;
size_t sparse_cap;
@@ -1567,7 +1607,7 @@ typedef struct ScrArr {
size_t prop_len;
size_t prop_cap;
struct ScrDyn *metadata; /* heterogeneous named properties of match results */
-} ScrArr;
+} ScrArrExt;
/* The LLVM backend's inline retain/release read elem_trace through its
* mirrored %ScrArr type (field 6) to decide mark-live / candidate buffering
* exactly like scr_arr_retain and scr_arr_release. */
@@ -1609,6 +1649,11 @@ static inline ScrArr *scr_arr_retain(ScrArr *a) {
* non-NULL) feed the candidate-root buffer like every headered type. */
void scr_arr_release(ScrArr *a);
void scr_arr_copy_metadata(ScrArr *a, struct ScrDyn *target);
+/* The array's out-of-line storage, allocated (zeroed) on first use. */
+ScrArrExt *scr_arr_ext(ScrArr *a);
+/* Swaps the sparse slots and numeric properties of two arrays; each keeps
+ * its own match metadata (refresh of a cached typed view, llvm/dyn.ts). */
+void scr_arr_swap_indexed_ext(ScrArr *a, ScrArr *b);
/* Trace entry point for cycle-capable arrays — stored in their collector
* header and passed wherever a container stores an ARRAY payload's RC entry
@@ -2188,6 +2233,23 @@ typedef struct ScrBox {
uint64_t slot; /* double/bool/pointer via memcpy and casts */
} ScrBox;
+/* A box can lie on a cycle only when its trace visits its payload: a
+ * closure, or a class instance that carries a cycle header. Every other box
+ * is a leaf of the object graph, so its releases buffer no candidate. The
+ * emitted inline release (llvm/shapes.ts, the box row) mirrors this test,
+ * reading `kind` and `obj_trace` at the asserted offsets. */
+static inline bool scr_box_traced(const ScrBox *b) {
+ return b->kind == SCR_BOX_FUNC || b->obj_trace != NULL;
+}
+_Static_assert(SCR_BOX_FUNC == 4, "the emitted box release compares kind against 4");
+#if UINTPTR_MAX == UINT64_MAX
+_Static_assert(offsetof(ScrBox, kind) == 8 && offsetof(ScrBox, obj_trace) == 32,
+ "the emitted box release reads kind at 8 and obj_trace at 32");
+#else
+_Static_assert(offsetof(ScrBox, kind) == 4 && offsetof(ScrBox, obj_trace) == 16,
+ "the emitted box release reads kind at 4 and obj_trace at 16");
+#endif
+
ScrBox *scr_box_new(ScrBoxKind kind); /* returns +1, slot zeroed */
ScrBox *scr_box_new_obj(void *(*retain)(void *), void (*release)(void *),
ScrTraceFn trace);
diff --git a/packages/runtime/src/scr_stream.c b/packages/runtime/src/scr_stream.c
index 760c3341b..dc7647090 100644
--- a/packages/runtime/src/scr_stream.c
+++ b/packages/runtime/src/scr_stream.c
@@ -1230,7 +1230,8 @@ static ScrStreamState *scr_stream_state_new(bool has_r, bool has_w,
static ScrStream *scr_stream_alloc(const ScrVt *vt, const char *cls, bool has_r, bool has_w,
double rhwm, double whwm, bool auto_destroy,
bool emit_close, bool allow_half_open) {
- ScrStream *s = scr_cyc_alloc(sizeof *s, &scr_stream_trace, &scr_stream_gcfree);
+ static const ScrCycType cyc_type = { &scr_stream_trace, &scr_stream_gcfree, SCR_CYC_KIND_STREAM };
+ ScrStream *s = scr_cyc_alloc(sizeof *s, &cyc_type);
s->rc = 1;
s->vt = vt;
s->reg = NULL;
diff --git a/packages/runtime/src/scr_union.c b/packages/runtime/src/scr_union.c
index a5e981de7..c4f3b41f6 100644
--- a/packages/runtime/src/scr_union.c
+++ b/packages/runtime/src/scr_union.c
@@ -39,7 +39,8 @@ static void scr_union_gcfree(void *o) {
}
static ScrUnion *scr_union_alloc(uint32_t tag) {
- ScrUnion *u = scr_cyc_alloc_inline(sizeof(ScrUnion), &scr_union_trace, &scr_union_gcfree);
+ static const ScrCycType cyc_type = { &scr_union_trace, &scr_union_gcfree, SCR_CYC_KIND_UNION };
+ ScrUnion *u = scr_cyc_alloc_inline(sizeof(ScrUnion), &cyc_type);
u->rc = 1;
u->tag = tag;
#ifdef SCR_RC_AUDIT
diff --git a/packages/runtime/src/scr_worker_events.c b/packages/runtime/src/scr_worker_events.c
index 663d7116a..5cd68b558 100644
--- a/packages/runtime/src/scr_worker_events.c
+++ b/packages/runtime/src/scr_worker_events.c
@@ -77,7 +77,8 @@ static void scr_worker_port_listeners(void *ptr, const ScrStr *name, bool presen
}
static ScrWorkerView *scr_worker_view_new(bool port) {
- ScrWorkerView *view = scr_cyc_alloc(sizeof(*view), scr_worker_view_trace, scr_worker_view_gcfree);
+ static const ScrCycType cyc_type = { &scr_worker_view_trace, &scr_worker_view_gcfree, SCR_CYC_KIND_WORKER_VIEW };
+ ScrWorkerView *view = scr_cyc_alloc(sizeof(*view), &cyc_type);
view->rc = 1;
view->port = port;
view->referenced = !port;
diff --git a/packages/runtime/test/cycle.test.ts b/packages/runtime/test/cycle.test.ts
index 0e73752c7..47c699b24 100644
--- a/packages/runtime/test/cycle.test.ts
+++ b/packages/runtime/test/cycle.test.ts
@@ -17,6 +17,7 @@ test("cycle traversal preserves counts, deep graphs, configured thresholds and g
"-Wall",
"-Wextra",
"-fsanitize=address,undefined",
+ "-DSCR_CYC_TESTING",
"-o",
bin,
join(testDir, "test_cycle.c"),
diff --git a/packages/runtime/test/test_array.c b/packages/runtime/test/test_array.c
index b92f5ba75..f707b4486 100644
--- a/packages/runtime/test/test_array.c
+++ b/packages/runtime/test/test_array.c
@@ -18,6 +18,9 @@
#include
#include
+/* Sparse entries live in the array's out-of-line storage. */
+static size_t arr_sparse_len(const ScrArr *a) { return a->ext ? a->ext->sparse_len : 0; }
+
#ifdef SCR_RC_AUDIT
long scr_str_live_count(void); /* provided by scr_string.c */
long scr_arr_live_count(void); /* provided by scr_array.c */
@@ -385,7 +388,8 @@ static void mock_cyc_release(void *p) {
}
static MockRec *mock_cyc_new(int value) {
- MockRec *r = scr_cyc_alloc(sizeof *r, &mock_trace, &mock_gcfree);
+ static const ScrCycType cyc_type = { &mock_trace, &mock_gcfree, 0 };
+ MockRec *r = scr_cyc_alloc(sizeof *r, &cyc_type);
r->rc = 1;
r->value = value;
mock_live++;
@@ -648,11 +652,11 @@ static void test_sparse_holes(void) {
scr_arr_set_f64(high, high_index, 11);
check(scr_arr_has(high, high_index), "high sparse index is present");
check(high->cap <= ((size_t)1 << 20), "high sparse index avoids dense allocation");
- check(high->sparse_len == 1, "high sparse index uses side storage");
+ check(arr_sparse_len(high) == 1, "high sparse index uses side storage");
check_f64(scr_arr_next_present(high, 0), high_index,
"next-present traversal jumps to side storage");
scr_arr_set_len(high, 0);
- check(high->sparse_len == 0 && scr_arr_len(high) == 0,
+ check(arr_sparse_len(high) == 0 && scr_arr_len(high) == 0,
"length shrink removes sparse entries");
scr_arr_release(high);
@@ -706,7 +710,7 @@ static void test_large_storage(void) {
ScrArr *pushed = scr_arr_new(SCR_ELEM_F64, 0);
for (double i = 0; i < n; i++) scr_arr_push_f64(pushed, i);
- check(pushed->sparse_len == 0 && pushed->cap >= (size_t)n,
+ check(arr_sparse_len(pushed) == 0 && pushed->cap >= (size_t)n,
"appends past the cutoff stay dense");
check_f64(scr_arr_get_f64(pushed, n - 1), n - 1, "large append round-trips");
scr_arr_release(pushed);
@@ -714,7 +718,7 @@ static void test_large_storage(void) {
ScrArr *filled = scr_arr_new(SCR_ELEM_BOOL, 0);
scr_arr_set_len(filled, n);
scr_arr_release(scr_arr_fill_bool(filled, true, 0, INFINITY));
- check(filled->sparse_len == 0 && filled->cap >= (size_t)n,
+ check(arr_sparse_len(filled) == 0 && filled->cap >= (size_t)n,
"fill of a large holey array allocates dense storage");
scr_arr_set_bool(filled, n - 1, false);
check(!scr_arr_get_bool(filled, n - 1) && scr_arr_get_bool(filled, n - 2),
@@ -724,7 +728,7 @@ static void test_large_storage(void) {
ScrArr *forward = scr_arr_new(SCR_ELEM_F64, 0);
scr_arr_set_len(forward, n);
for (double i = 0; i < n; i++) scr_arr_set_f64(forward, i, i * 2);
- check(forward->sparse_len == 0, "forward indexed writes extend dense storage");
+ check(arr_sparse_len(forward) == 0, "forward indexed writes extend dense storage");
scr_arr_release(forward);
/* Every 64th index past the cutoff is too sparse for packed slots, even
@@ -734,14 +738,14 @@ static void test_large_storage(void) {
scr_arr_release(scr_arr_fill_f64(strided, 0, 0, INFINITY));
size_t entries = 0;
for (double i = (double)cutoff + 4095; i < n; i += 64, entries++) scr_arr_set_f64(strided, i, i);
- check(strided->cap == cutoff && strided->sparse_len == entries,
+ check(strided->cap == cutoff && arr_sparse_len(strided) == entries,
"strided indices past the cutoff keep side storage");
for (int round = 0; round < 3; round++) {
for (double i = (double)cutoff + 4095; i < n; i += 64) {
scr_arr_set_f64(strided, i, scr_arr_get_f64(strided, i) + 1);
}
}
- check(strided->sparse_len == entries &&
+ check(arr_sparse_len(strided) == entries &&
scr_arr_get_f64(strided, (double)cutoff + 4159) == (double)cutoff + 4162,
"rewriting sparse entries updates them in place");
check(!scr_arr_has(strided, (double)cutoff + 1) && scr_arr_len(strided) > (double)cutoff + 4095,
@@ -760,7 +764,7 @@ static void test_large_storage(void) {
for (double i = n - 2; i >= (double)cutoff; i -= 2) {
scr_arr_set_ref(backward, i, scr_str_new("v", 1));
}
- check(backward->sparse_len == 0 && backward->cap >= (size_t)n,
+ check(arr_sparse_len(backward) == 0 && backward->cap >= (size_t)n,
"a populated sparse tail densifies");
check(scr_arr_state(backward, n - 1) == SCR_ARR_UNDEFINED &&
scr_arr_state(backward, n - 3) == SCR_ARR_HOLE &&
@@ -777,7 +781,7 @@ static void test_large_storage(void) {
scr_arr_set_ref(deque, middle - k * 1000, scr_str_new("d", 1));
scr_arr_set_ref(deque, middle + 1 + k * 1000, scr_str_new("a", 1));
}
- check(deque->sparse_len == 6000 && deque->cap == 0, "spread first touches stay sparse");
+ check(arr_sparse_len(deque) == 6000 && deque->cap == 0, "spread first touches stay sparse");
for (double k = 0; k < 3000; k += 3) {
scr_arr_delete(deque, middle - k * 1000);
scr_arr_delete(deque, middle + 1 + k * 1000);
@@ -807,7 +811,7 @@ static void test_large_storage(void) {
ScrArr *holes = scr_arr_slice(high, 1, INFINITY);
check(holes->cap <= cutoff && scr_arr_len(holes) == 4294967294.0,
"slicing a huge hole run keeps a bounded allocation");
- check(holes->sparse_len == 1 && scr_arr_get_f64(holes, 4294967293.0) == 2,
+ check(arr_sparse_len(holes) == 1 && scr_arr_get_f64(holes, 4294967293.0) == 2,
"a sliced hole run keeps its sparse value");
scr_arr_release(holes);
scr_arr_release(high);
diff --git a/packages/runtime/test/test_cycle.c b/packages/runtime/test/test_cycle.c
index ae05db24a..ad19035a7 100644
--- a/packages/runtime/test/test_cycle.c
+++ b/packages/runtime/test/test_cycle.c
@@ -63,7 +63,8 @@ static Node *make_ring(size_t count) {
Node **nodes = calloc(count, sizeof(*nodes));
if (!nodes) scr_trap("cycle test: out of memory\n");
for (size_t i = 0; i < count; i++) {
- nodes[i] = scr_cyc_alloc(sizeof(*nodes[i]), node_trace, node_free);
+ static const ScrCycType cyc_type = { &node_trace, &node_free, 0 };
+ nodes[i] = scr_cyc_alloc(sizeof(*nodes[i]), &cyc_type);
nodes[i]->rc = 1; /* the ring edge that will point at this node */
}
for (size_t i = 0; i < count; i++)
@@ -118,7 +119,8 @@ static void check_age_reset_when_last_mature_root_dies(void) {
ring = make_ring(32);
ring->rc++;
release_live(ring);
- leaf = scr_cyc_alloc(sizeof(*leaf), node_trace, node_free);
+ static const ScrCycType cyc_type = { &node_trace, &node_free, 0 };
+ leaf = scr_cyc_alloc(sizeof(*leaf), &cyc_type);
leaf->rc = 1;
leaf->rc++; /* temporary release makes it a nursery candidate */
release_live(leaf);
@@ -153,7 +155,8 @@ static void check_dead_backlog_rearms_release_trigger(void) {
Node *leaves[BACKLOG];
for (size_t i = 0; i < BACKLOG; i++) {
- leaves[i] = scr_cyc_alloc(sizeof(*leaves[i]), node_trace, node_free);
+ static const ScrCycType cyc_type = { &node_trace, &node_free, 0 };
+ leaves[i] = scr_cyc_alloc(sizeof(*leaves[i]), &cyc_type);
leaves[i]->rc = 2; /* external owner plus a temporary reference */
release_live(leaves[i]);
}
@@ -180,7 +183,8 @@ static void check_dead_backlog_rearms_release_trigger(void) {
}
static Node *make_leaf(void) {
- Node *leaf = scr_cyc_alloc(sizeof(*leaf), node_trace, node_free);
+ static const ScrCycType cyc_type = { &node_trace, &node_free, 0 };
+ Node *leaf = scr_cyc_alloc(sizeof(*leaf), &cyc_type);
leaf->rc = 1;
return leaf;
}
@@ -609,7 +613,8 @@ static void check_deep_cross_generation_chain(void) {
size_t before = freed;
Node *chain = NULL;
for (size_t i = 0; i < DEPTH; i++) {
- Node *node = scr_cyc_alloc(sizeof(*node), node_trace, node_free);
+ static const ScrCycType cyc_type = { &node_trace, &node_free, 0 };
+ Node *node = scr_cyc_alloc(sizeof(*node), &cyc_type);
node->rc = 1;
node->next = chain;
chain = node;
@@ -661,6 +666,259 @@ static void check_deferred_white_restoration(void) {
check(freed == before + DEPTH, "restored deferred nodes leaked");
}
+/* Nursery bypass (scr_cycle.c): candidates of an emitted type (kind != 0)
+ * that keep surviving their nursery passes are promoted on release instead
+ * of walked; dead cycles of that type are still found by older passes; and
+ * once the type's objects die in nursery cycles again, it is walked again.
+ * Window, ratio and sample mirror SCR_CYC_BYPASS_*. */
+/* Tests only (SCR_CYC_TESTING): one pass at exactly `gen`. */
+size_t scr_cyc_pass_at(unsigned gen);
+static size_t bypass_traced; /* trace calls on bypass-kind nodes */
+static size_t bypass_freed; /* bypass-kind nodes freed */
+static void bypass_trace(void *obj, ScrTraceVisit visit, void *ctx) {
+ bypass_traced++;
+ node_trace(obj, visit, ctx);
+}
+static void bypass_free(void *obj) {
+ bypass_freed++;
+ node_free(obj);
+}
+static const ScrCycType bypass_type = { &bypass_trace, &bypass_free, 7 };
+
+static Node *make_bypass_node(void) {
+ Node *node = scr_cyc_alloc(sizeof(*node), &bypass_type);
+ node->rc = 1;
+ return node;
+}
+
+/* Run release-triggered passes, as a program does by releasing: a pass's
+ * level then follows growth and backlogs only, never the scheduled age. */
+static void run_release_passes(void) {
+ for (size_t i = 0; i <= configured_nursery_threshold(); i++)
+ release_live(make_ring(1));
+}
+
+/* Trace calls spent on PROBE fresh, externally owned bypass-kind
+ * candidates. A full pass first resets the growth baselines, so the probe
+ * never trips a mature or full pass: only nursery passes walk. */
+static size_t probe_bypass_walks(Node **owned, size_t probe) {
+ scr_collect_cycles();
+ size_t before = bypass_traced;
+ for (size_t i = 0; i < probe; i++) {
+ owned[i] = make_bypass_node();
+ read_live(owned[i]);
+ }
+ run_release_passes();
+ return bypass_traced - before;
+}
+
+static void check_nursery_bypass(void) {
+ enum { WINDOW = 8192, SAMPLE = 16, LIVE = 3 * WINDOW, PROBE = 4096, RINGS = 32 };
+ Node **live = calloc(LIVE + 3 * PROBE, sizeof(*live));
+ if (!live) scr_trap("cycle test: out of memory\n");
+ size_t nlive = 0;
+
+ /* A walked candidate costs two trace calls (markGray and the restoring
+ * scan). Every survivor of the first window is walked; afterwards only
+ * one in SAMPLE is. */
+ for (; nlive < LIVE; nlive++) {
+ live[nlive] = make_bypass_node();
+ read_live(live[nlive]);
+ }
+ size_t walks = probe_bypass_walks(live + nlive, PROBE);
+ nlive += PROBE;
+ check(walks < PROBE / 2, "a type whose candidates always survive was not bypassed");
+
+ /* Dead rings of the bypassed type wait in the mature buffer for an older
+ * pass, which finds them. */
+ size_t before = bypass_freed;
+ for (size_t r = 0; r < RINGS; r++) {
+ Node *a = make_bypass_node(), *b = make_bypass_node();
+ a->next = b;
+ b->next = a;
+ a->rc++; /* the outside owner */
+ release_live(a);
+ }
+ run_release_passes();
+ check(bypass_freed - before < 2 * RINGS, "bypassed dead rings were all walked by the nursery");
+ scr_collect_cycles();
+ check(bypass_freed - before == 2 * RINGS, "an older pass missed dead rings of a bypassed type");
+
+ /* Now the type's objects die in nursery cycles. The sampled candidates
+ * keep measuring the yield, so after one window the type is walked again.
+ * A nursery pass after each release walks every sampled candidate at its
+ * own level: the promoted rings pile up in older buffers, and an older
+ * pass the growth schedule runs there would otherwise sweep sampled
+ * candidates too and leave the nursery's window unfinished. */
+ for (size_t r = 0; r < (size_t)WINDOW * SAMPLE; r++) {
+ Node *a = make_bypass_node(), *b = make_bypass_node();
+ a->next = b;
+ b->next = a;
+ a->rc++;
+ release_live(a);
+ scr_cyc_pass_at(SCR_CYC_NURSERY);
+ }
+ scr_collect_cycles();
+ walks = probe_bypass_walks(live + nlive, PROBE);
+ nlive += PROBE;
+ check(walks >= PROBE, "a type whose objects die in nursery cycles stayed bypassed");
+
+ for (size_t i = 0; i < nlive; i++) {
+ live[i]->rc = 0;
+ scr_cyc_on_dead(live[i]);
+ node_free(live[i]);
+ }
+ free(live);
+ scr_collect_cycles();
+}
+
+/* The mature level's bypass: a type whose mature candidates keep surviving
+ * goes straight to the old generation, its dead cycles are found by full
+ * passes, and once its objects die at the mature level again it is judged
+ * there again. Passes run at exact levels through the test entry point. */
+static size_t mature_kind_freed;
+static void mature_kind_free(void *obj) {
+ mature_kind_freed++;
+ node_free(obj);
+}
+static const ScrCycType mature_kind_type = { &node_trace, &mature_kind_free, 8 };
+
+static Node *make_mature_kind_node(void) {
+ Node *node = scr_cyc_alloc(sizeof(*node), &mature_kind_type);
+ node->rc = 1;
+ return node;
+}
+
+/* How many of PROBE fresh, externally owned candidates a release buffers
+ * straight into the old generation. */
+static size_t probe_old_on_release(Node **owned, size_t probe) {
+ scr_collect_cycles(); /* reset the growth baselines: no pass runs below */
+ size_t old = 0;
+ for (size_t i = 0; i < probe; i++) {
+ owned[i] = make_mature_kind_node();
+ read_live(owned[i]);
+ old += scr_cyc_hdr(owned[i])->gen == SCR_CYC_OLD;
+ }
+ return old;
+}
+
+static void check_mature_bypass(void) {
+ enum { WINDOW = 8192, SAMPLE = 16, LIVE = 3 * WINDOW, PROBE = 4096, BATCH = 1024 };
+ Node **live = calloc(LIVE + 2 * PROBE, sizeof(*live));
+ Node **batch = calloc(BATCH, sizeof(*batch));
+ if (!live || !batch) scr_trap("cycle test: out of memory\n");
+ size_t nlive = 0;
+
+ for (; nlive < LIVE; nlive++) {
+ live[nlive] = make_mature_kind_node();
+ read_live(live[nlive]);
+ }
+ /* Whatever passes those releases ran, put every node in the mature
+ * generation as a candidate (any generation is sound), and walk them all
+ * at that level: none dies. */
+ scr_collect_cycles();
+ for (size_t i = 0; i < nlive; i++) {
+ scr_cyc_hdr(live[i])->gen = SCR_CYC_MATURE;
+ read_live(live[i]);
+ }
+ scr_cyc_pass_at(SCR_CYC_MATURE);
+ size_t old = probe_old_on_release(live + nlive, PROBE);
+ nlive += PROBE;
+ check(old >= PROBE / 2, "a type whose mature candidates always survive kept its mature passes");
+
+ /* Dead rings of it wait for a full pass, which finds them. */
+ size_t before = mature_kind_freed;
+ for (size_t r = 0; r < 32; r++) {
+ Node *a = make_mature_kind_node(), *b = make_mature_kind_node();
+ a->next = b;
+ b->next = a;
+ a->rc++;
+ release_live(a);
+ }
+ scr_cyc_pass_at(SCR_CYC_NURSERY);
+ scr_cyc_pass_at(SCR_CYC_MATURE);
+ check(mature_kind_freed - before < 64, "rings past the mature level were all walked by it");
+ scr_collect_cycles();
+ check(mature_kind_freed - before == 64, "a full pass missed dead rings of a type past the mature level");
+
+ /* Now its rings survive the nursery and die before the next mature pass.
+ * The candidates kept at the mature level measure that, and after one
+ * window the type is judged there again. */
+ for (size_t round = 0; round < (size_t)WINDOW * SAMPLE / BATCH; round++) {
+ for (size_t r = 0; r < BATCH; r++) {
+ Node *a = make_mature_kind_node(), *b = make_mature_kind_node();
+ a->next = b;
+ b->next = a;
+ a->rc++; /* the outside owner */
+ read_live(a);
+ batch[r] = a;
+ }
+ scr_cyc_pass_at(SCR_CYC_NURSERY);
+ for (size_t r = 0; r < BATCH; r++) release_live(batch[r]);
+ scr_cyc_pass_at(SCR_CYC_MATURE);
+ }
+ scr_collect_cycles();
+ old = probe_old_on_release(live + nlive, PROBE);
+ nlive += PROBE;
+ check(old < PROBE / 4, "a type whose objects die at the mature level stayed past it");
+
+ for (size_t i = 0; i < nlive; i++) {
+ live[i]->rc = 0;
+ scr_cyc_on_dead(live[i]);
+ node_free(live[i]);
+ }
+ free(live);
+ free(batch);
+ scr_collect_cycles();
+}
+
+/* A nursery pass re-buffers its survivors past the mature level when
+ * their type bypasses it, even if the type's nursery candidates are
+ * walked (here half of them die young, in rings). */
+static const ScrCycType young_dying_type = { &node_trace, &node_free, 9 };
+
+static void check_rebuffer_past_mature(void) {
+ enum { BATCH = 1024, ROUNDS = 24, PROBE = 1024 };
+ Node **live = calloc((size_t)BATCH * ROUNDS + PROBE, sizeof(*live));
+ if (!live) scr_trap("cycle test: out of memory\n");
+ size_t nlive = 0;
+ for (size_t round = 0; round < ROUNDS; round++) {
+ for (size_t i = 0; i < BATCH; i++) {
+ Node *keep = scr_cyc_alloc(sizeof(*keep), &young_dying_type);
+ keep->rc = 1;
+ read_live(keep);
+ live[nlive++] = keep;
+ Node *a = scr_cyc_alloc(sizeof(*a), &young_dying_type);
+ Node *b = scr_cyc_alloc(sizeof(*b), &young_dying_type);
+ a->rc = 2; /* b's edge and the outside owner */
+ b->rc = 1;
+ a->next = b;
+ b->next = a;
+ release_live(a); /* dead, buffered in the nursery */
+ }
+ scr_cyc_pass_at(SCR_CYC_NURSERY); /* frees the rings, re-buffers the rest */
+ scr_cyc_pass_at(SCR_CYC_MATURE); /* finds nothing */
+ }
+ scr_collect_cycles();
+ for (size_t i = 0; i < PROBE; i++) {
+ Node *keep = scr_cyc_alloc(sizeof(*keep), &young_dying_type);
+ keep->rc = 1;
+ read_live(keep);
+ live[nlive++] = keep;
+ }
+ scr_cyc_pass_at(SCR_CYC_NURSERY);
+ size_t old = 0;
+ for (size_t i = nlive - PROBE; i < nlive; i++) old += scr_cyc_hdr(live[i])->gen == SCR_CYC_OLD;
+ check(old >= PROBE / 2, "nursery survivors were re-buffered into a level their type bypasses");
+ for (size_t i = 0; i < nlive; i++) {
+ live[i]->rc = 0;
+ scr_cyc_on_dead(live[i]);
+ node_free(live[i]);
+ }
+ free(live);
+ scr_collect_cycles();
+}
+
int main(void) {
check_old_garbage_reclaimed_while_growing();
check_older_rings_into_old();
@@ -680,6 +938,9 @@ int main(void) {
check_teardown_rebuffers_and_immortals();
check_growth_schedule_follows_yield();
check_growth_cap_bounds_floating_garbage();
+ check_nursery_bypass();
+ check_mature_bypass();
+ check_rebuffer_past_mature();
printf("cycle collection checks passed: threshold=%zu growth_cap=%zu\n",
configured_nursery_threshold(), configured_growth_cap());
return 0;
diff --git a/packages/runtime/test/test_lib.c b/packages/runtime/test/test_lib.c
index a30eb884f..2bed0d099 100644
--- a/packages/runtime/test/test_lib.c
+++ b/packages/runtime/test/test_lib.c
@@ -20,6 +20,9 @@
#include
#ifndef _WIN32
#include
+
+/* Sparse entries live in the array's out-of-line storage. */
+static size_t arr_sparse_len(const ScrArr *a) { return a->ext ? a->ext->sparse_len : 0; }
#endif
static int checks = 0;
@@ -53,7 +56,7 @@ static void test_large_set_materialization(void) {
for (size_t i = 0; i < count; i++) scr_map_set_f64_f64(set, (double)i, 0);
scr_map_delete_f64(set, 1);
ScrArr *values = scr_set_to_arr_f64(set);
- check(values->len == count - 1 && values->cap >= values->len && values->sparse_len == 0,
+ check(values->len == count - 1 && values->cap >= values->len && arr_sparse_len(values) == 0,
"large Set materializes into dense storage past the 2^20 cutoff");
check(scr_arr_get_f64(values, 0) == 0 && scr_arr_get_f64(values, 1) == 2,
"large Set materialization skips tombstones in order");
diff --git a/tests/corpus/4610-array-ext-dyn-refresh.ts b/tests/corpus/4610-array-ext-dyn-refresh.ts
new file mode 100644
index 000000000..584501dfe
--- /dev/null
+++ b/tests/corpus/4610-array-ext-dyn-refresh.ts
@@ -0,0 +1,44 @@
+// Sparse slots, noncanonical numeric properties and regex match metadata
+// live in an array's out-of-line storage: they keep their values through
+// typed views of dynamically typed arrays, writes through live references,
+// truncation and copies.
+
+function asNumbers(value: any): number[] {
+ return value as number[];
+}
+
+const parsed: any = JSON.parse("[1, 2, 3]");
+const first = asNumbers(parsed);
+console.log("first", first.length, first[0], first[2]);
+
+parsed[-1] = 9;
+parsed["1.5"] = 11;
+parsed.push(4);
+const second = asNumbers(parsed);
+console.log("properties", second.length, second[3], parsed[-1], parsed["1.5"], "-1" in parsed, Object.keys(parsed).length);
+
+delete parsed[-1];
+parsed.push(5);
+const third = asNumbers(parsed);
+console.log("third", third.length, third[4], parsed[-1], "-1" in parsed, "1.5" in parsed);
+
+const sparse: number[] = [1, 2, 3];
+sparse[4_000_000] = 7;
+console.log("sparse", sparse.length, sparse[4_000_000], 3_999_999 in sparse, 4_000_000 in sparse);
+sparse.length = 3;
+console.log("sparse truncated", sparse.length, sparse[4_000_000], 4_000_000 in sparse);
+
+const typed: number[] = [5, 6];
+typed[3_000_000] = 1;
+const viaAny: any = typed;
+viaAny[3_000_001] = 2;
+console.log("typed", typed.length, typed[3_000_000], typed[3_000_001], 3_000_001 in typed);
+typed.length = 2;
+console.log("typed truncated", typed.length, typed[3_000_000], viaAny.length);
+
+const match = /(b)(c)?/.exec("abd")!;
+console.log("match", match.index, match.input, match.length, match[1], match[2]);
+const loose: any = match;
+console.log("match via any", loose.index, loose.input, loose[1], loose.length);
+const copy = match.slice();
+console.log("copy", copy.length, copy[0], copy[1]);
diff --git a/tests/corpus/4660-typed-rest-spread-forwarding.ts b/tests/corpus/4660-typed-rest-spread-forwarding.ts
new file mode 100644
index 000000000..14c6f218c
--- /dev/null
+++ b/tests/corpus/4660-typed-rest-spread-forwarding.ts
@@ -0,0 +1,76 @@
+// Runtime-length spreads that land in a typed rest parameter, including a
+// `T[]` into a `(T | undefined)[]` rest: forwarding `...args` preserves
+// source evaluation order, copies the spread (the callee's rest is fresh),
+// and reads holes as undefined.
+class Node {
+ kind: number;
+ pos: number;
+ constructor(kind: number, pos: number) {
+ this.kind = kind;
+ this.pos = pos;
+ }
+}
+
+class Checker {
+ count = 0;
+ report(node: Node | undefined, message: string, ...args: string[]): string {
+ this.count++;
+ const at = node === undefined ? "-" : `${node.kind}@${node.pos}`;
+ return `${at} ${message} [${args.join(",")}] ${args.length}`;
+ }
+}
+
+function grammarError(c: Checker, node: Node, message: string, ...args: string[]): string {
+ return c.report(node, message, ...args);
+}
+
+function diagnosticAt(file: Node | undefined, node: Node | undefined, message: string, ...args: (string | undefined)[]): string {
+ const shown = args.map((a) => (a === undefined ? "" : a));
+ return `${file?.kind ?? "none"} ${node?.pos ?? "none"} ${message} ${shown.join("|")}`;
+}
+
+function forward(file: Node, node: Node | undefined, message: string, ...args: string[]): string {
+ return diagnosticAt(file, node, message, ...args);
+}
+
+function mutateRest(...items: number[]): number {
+ items.push(99);
+ items[0] = -1;
+ return items.length;
+}
+
+const order: string[] = [];
+function note(label: string): string {
+ order.push(label);
+ return label;
+}
+function noteAll(label: string, values: string[]): string[] {
+ order.push(label);
+ return values;
+}
+
+function join(head: string, ...rest: string[]): string {
+ return [head, ...rest].join("/");
+}
+
+const c = new Checker();
+const n = new Node(80, 12);
+console.log(grammarError(c, n, "missing", "a", "b"));
+console.log(grammarError(c, n, "none"));
+console.log(c.report(undefined, "direct", ...["x", "y", "z"]));
+console.log(forward(n, n, "fwd", "p", "q"));
+console.log(forward(n, undefined, "fwd-empty"));
+
+const source = [1, 2, 3];
+console.log(mutateRest(...source), source.join(","));
+console.log(mutateRest(0, ...source, 4), source.join(","));
+
+const sparse: string[] = [];
+sparse[2] = "late";
+console.log(diagnosticAt(n, n, "holes", ...sparse));
+
+const xs = ["x1", "x2"];
+const ys = ["y1"];
+console.log(join(note("head"), ...noteAll("xs", xs), note("mid"), ...noteAll("ys", ys)));
+console.log(order.join(" "));
+console.log(c.count);
diff --git a/tests/corpus/4671-scoped-array-read-borrows.ts b/tests/corpus/4671-scoped-array-read-borrows.ts
new file mode 100644
index 000000000..2cb4fbbda
--- /dev/null
+++ b/tests/corpus/4671-scoped-array-read-borrows.ts
@@ -0,0 +1,152 @@
+export {};
+
+// Optional reads of `T | undefined` element arrays, and local reads whose
+// live range may or may not remove the element's last reference.
+class Item {
+ name: string;
+ hits = 0;
+ constructor(name: string) {
+ this.name = name;
+ }
+}
+
+type Page = (Item | undefined)[] | undefined;
+
+class Store {
+ readonly pages: Page[] = [];
+ items: (Item | undefined)[] = [];
+ created = 0;
+
+ // A get-or-create store in pages of 4 ids: the page read is dead before
+ // the creating path mutates the page table.
+ item(id: number): Item {
+ const page = this.pages[id >> 2];
+ let item = page !== undefined ? page[id & 3] : undefined;
+ if (item === undefined) {
+ item = new Item(`item${id}`);
+ this.created++;
+ store(this.pages, id, item);
+ }
+ return item;
+ }
+
+ // The same through an optional chain on the page read.
+ peek(id: number): string {
+ const item = this.pages[id >> 2]?.[id & 3];
+ return item === undefined ? "none" : item.name;
+ }
+
+ // The local is still used after the slot is overwritten: it keeps its own
+ // reference, so the old element survives until the read.
+ replaceThenRead(i: number): string {
+ const old = this.items[i];
+ this.items[i] = new Item(`new${i}`);
+ return old === undefined ? "none" : `${old.name}:${old.hits}`;
+ }
+
+ // Truncation and field replacement between the read and its last use.
+ truncateThenRead(i: number): string {
+ const old = this.items[i];
+ this.items.length = 0;
+ return old === undefined ? "none" : old.name;
+ }
+
+ swapThenRead(i: number): string {
+ const old = this.items[i];
+ this.items = [new Item("swapped")];
+ return old === undefined ? "none" : old.name;
+ }
+
+ // Mutation after the last use: the read may borrow.
+ readThenMutate(i: number): string {
+ const it = this.items[i];
+ const label = it === undefined ? "none" : it.name;
+ this.items.push(new Item(`pushed${i}`));
+ return label;
+ }
+
+ // A preserving call consumes the element; a mutating one keeps an owner.
+ describe(i: number): string {
+ return label(this.items[i]);
+ }
+
+ clearAndDescribe(i: number): string {
+ return clearing(this, this.items[i]);
+ }
+}
+
+function store(pages: Page[], id: number, item: Item): void {
+ const index = id >> 2;
+ while (pages.length <= index) pages.push(undefined);
+ let page = pages[index];
+ if (page === undefined) {
+ page = [];
+ for (let i = 0; i < 4; i++) page.push(undefined);
+ pages[index] = page;
+ }
+ page[id & 3] = item;
+}
+
+function label(item: Item | undefined): string {
+ return item === undefined ? "none" : `${item.name}#${item.hits}`;
+}
+
+function clearing(s: Store, item: Item | undefined): string {
+ s.items = [];
+ return item === undefined ? "none" : `${item.name} after clear`;
+}
+
+const s = new Store();
+for (let round = 0; round < 3; round++)
+ for (let id = 0; id < 11; id += 3) s.item(id).hits++;
+console.log("created", s.created, s.item(0).hits, s.item(9).hits);
+for (const id of [0, 1, 3, 9, 10, 40]) console.log("peek", id, s.peek(id));
+
+function fill(): void {
+ s.items = [new Item("a"), undefined, new Item("c")];
+ s.items.push(undefined);
+ s.items[6] = new Item("g");
+ s.items[0]!.hits = 5;
+}
+fill();
+for (const i of [0, 1, 2, 3, 4, 6, 7, 1.5, -1]) console.log("describe", i, s.describe(i));
+for (const i of [0, 1, 2]) console.log("readThenMutate", s.readThenMutate(i), s.items.length);
+fill();
+for (const i of [0, 1, 6]) console.log("replace", s.replaceThenRead(i), s.items[i]?.name);
+fill();
+console.log("truncate", s.truncateThenRead(2), s.items.length);
+fill();
+console.log("swap", s.swapThenRead(6), s.items.length, s.items[0]?.name);
+fill();
+console.log("clear", s.clearAndDescribe(2), s.items.length);
+
+// An optional chain whose body empties the array that held its receiver.
+class Node2 {
+ name: string;
+ owner: Holder;
+ constructor(name: string, owner: Holder) {
+ this.name = name;
+ this.owner = owner;
+ }
+ detach(): string {
+ this.owner.nodes = [];
+ return `${this.name} detached, ${this.owner.nodes.length} left`;
+ }
+}
+class Holder {
+ nodes: (Node2 | undefined)[] = [];
+}
+const h = new Holder();
+h.nodes.push(new Node2("n0", h), undefined, new Node2("n2", h));
+console.log("chain", h.nodes[1]?.detach(), h.nodes.length);
+console.log("chain", h.nodes[2]?.detach(), h.nodes.length);
+console.log("chain", h.nodes[0]?.detach());
+
+// Many reads, so leaked or freed elements would show up under the sanitizers.
+let total = 0;
+for (let round = 0; round < 500; round++) {
+ fill();
+ for (let i = 0; i < 8; i++) total += s.describe(i).length + s.readThenMutate(i).length;
+ total += s.replaceThenRead(round % 7).length;
+}
+console.log("total", total, s.items.length);
diff --git a/tests/corpus/4690-cycle-bypass-long-lived.ts b/tests/corpus/4690-cycle-bypass-long-lived.ts
new file mode 100644
index 000000000..5ddae341b
--- /dev/null
+++ b/tests/corpus/4690-cycle-bypass-long-lived.ts
@@ -0,0 +1,138 @@
+// Collector types that bypass the younger generations.
+//
+// The cycle collector keeps per-type statistics: a class whose objects keep
+// surviving their nursery passes has its candidates promoted on release
+// instead of walked, and one whose mature candidates keep surviving goes
+// straight to the old generation. This program builds a large long-lived
+// declaration tree with parent and symbol back edges (tens of thousands of
+// candidates of each class, enough for both levels to bypass), then lets
+// objects of the same classes die in cycles, first after they are long
+// past the nursery, then young again, which turns the bypass back off.
+// WeakMap entries keyed by long-lived declarations must survive every pass.
+//
+// The output pins the surviving structure and the weak entries. The
+// sanitized lane's exit RC audit proves the cycles of bypassing classes
+// were reclaimed, and AddressSanitizer that no pass touched a freed one.
+class Sym {
+ name: string;
+ decls: Decl[] = [];
+ value: number;
+ constructor(name: string, value: number) {
+ this.name = name;
+ this.value = value;
+ }
+}
+
+class Decl {
+ parent: Decl | null = null;
+ kids: Decl[] = [];
+ sym: Sym | null = null;
+ depth: number;
+ constructor(depth: number) {
+ this.depth = depth;
+ }
+}
+
+function declare(parent: Decl | null, depth: number, name: string, value: number): Decl {
+ const decl = new Decl(depth);
+ const sym = new Sym(name, value);
+ decl.sym = sym;
+ sym.decls.push(decl);
+ if (parent !== null) {
+ decl.parent = parent;
+ parent.kids.push(decl);
+ }
+ return decl;
+}
+
+function build(parent: Decl | null, depth: number, fanout: number, prefix: string): Decl {
+ const decl = declare(parent, depth, prefix, prefix.charCodeAt(prefix.length - 1) * 7 + prefix.length + depth);
+ if (depth > 0) {
+ for (let i = 0; i < fanout; i = i + 1) build(decl, depth - 1, fanout, `${prefix}${i}`);
+ }
+ return decl;
+}
+
+function walk(decl: Decl, visit: (d: Decl) => void): void {
+ visit(decl);
+ for (const kid of decl.kids) walk(kid, visit);
+}
+
+function summary(root: Decl): string {
+ let nodes = 0;
+ let sum = 0;
+ let linked = true;
+ walk(root, (d) => {
+ nodes = nodes + 1;
+ const sym = d.sym;
+ if (sym === null || sym.decls[0] !== d) linked = false;
+ else sum = (sum + sym.value * (d.depth + 1)) % 1000003;
+ for (const kid of d.kids) if (kid.parent !== d) linked = false;
+ });
+ return `nodes=${nodes} sum=${sum} linked=${linked}`;
+}
+
+// A detached subtree of the same classes: plain reference counting cannot
+// free it (children point back at their parent, symbols at their decls).
+function garbageTree(depth: number, fanout: number, tag: string): number {
+ const root = build(null, depth, fanout, tag);
+ let n = 0;
+ walk(root, () => {
+ n = n + 1;
+ });
+ return n;
+}
+
+const roots: Decl[] = [];
+for (let r = 0; r < 4; r = r + 1) roots.push(build(null, 6, 5, `r${r}`));
+console.log("built", roots.map(summary).join(" | "));
+
+const weak = new WeakMap();
+let keyed = 0;
+for (const root of roots) {
+ walk(root, (d) => {
+ if (d.depth === 3) {
+ weak.set(d, `w${keyed}`);
+ keyed = keyed + 1;
+ }
+ });
+}
+console.log("weak keys", keyed);
+
+// Long-lived garbage: subtrees that survive a while, held by a list that
+// is then dropped all at once.
+let made = 0;
+for (let round = 0; round < 6; round = round + 1) {
+ const held: Decl[] = [];
+ for (let i = 0; i < 40; i = i + 1) held.push(build(null, 4, 4, `h${round}.${i}`));
+ for (const root of roots) {
+ walk(root, (d) => {
+ if (d.depth === 6) d.sym!.value = d.sym!.value + round;
+ });
+ }
+ for (const h of held) made = made + h.kids.length;
+}
+console.log("held and dropped", made, roots.map(summary).join(" | "));
+
+// Young garbage of the same classes: dies before any pass promotes it.
+let young = 0;
+for (let i = 0; i < 3000; i = i + 1) young = young + garbageTree(2, 3, `y${i % 17}`);
+console.log("young garbage", young);
+
+// Grow the live tree again after the bypass turned back off.
+roots.push(build(null, 6, 4, "late"));
+let found = 0;
+let present = 0;
+for (const root of roots) {
+ walk(root, (d) => {
+ if (d.depth === 3) {
+ const w = weak.get(d);
+ if (w !== undefined) {
+ found = found + 1;
+ if (w.startsWith("w")) present = present + 1;
+ }
+ }
+ });
+}
+console.log("weak found", found, present);
+console.log("final", roots.map(summary).join(" | "));
diff --git a/tests/fixtures/self-hosting/llvm-layouts.ts b/tests/fixtures/self-hosting/llvm-layouts.ts
index d5685fcb6..d767defab 100644
--- a/tests/fixtures/self-hosting/llvm-layouts.ts
+++ b/tests/fixtures/self-hosting/llvm-layouts.ts
@@ -3,6 +3,7 @@ import { deserializeModule } from "../../../packages/compiler/src/ir/serialize.j
import { computeTraced } from "../../../packages/compiler/src/backend/cycle-analysis.js";
import { buildClassGraph, classFieldIndex, type ClassHost } from "../../../packages/compiler/src/backend/llvm/classes.js";
import { emitLlvmLayouts } from "../../../packages/compiler/src/backend/llvm/layouts.js";
+import { FIRST_EMITTED_CYC_KIND } from "../../../packages/compiler/src/backend/llvm/alloc.js";
import { LlvmDebugInfo } from "../../../packages/compiler/src/backend/llvm/debug-info.js";
import { LlWalkers, type WalkerHost } from "../../../packages/compiler/src/backend/llvm/walkers.js";
import { BlockBuilder } from "../../../packages/compiler/src/backend/llvm/blocks.js";
@@ -36,6 +37,7 @@ try {
const units: { union: string; tag: number }[] = [];
let needsOom = false;
let needsBadTag = false;
+ let cycKinds = 0;
const traced = computeTraced(mod);
const nullableUnions = new NullableUnions(mod);
const unionsById = new Map((mod.unions ?? []).map((union) => [union.id, union]));
@@ -50,7 +52,9 @@ try {
return `@native_literal_${index}`;
},
sizeType: request.bits === 32 ? "i32" : "i64",
- cycleColorOffset: request.bits === 32 ? 12 : 16,
+ cycleBufferedOffset: 6,
+ // Collector statistics indices, numbered as LlEmitter numbers them.
+ nextCycKind: () => FIRST_EMITTED_CYC_KIND + cycKinds++,
tracedShapes: traced.shapes,
tracedUnions: traced.unions,
recordsById: new Map((mod.records ?? []).map((record) => [record.id, record])),
diff --git a/tests/harness/self-hosting-native-toolchain.test.ts b/tests/harness/self-hosting-native-toolchain.test.ts
index 92197243b..43a230682 100644
--- a/tests/harness/self-hosting-native-toolchain.test.ts
+++ b/tests/harness/self-hosting-native-toolchain.test.ts
@@ -100,7 +100,7 @@ for (const backend of ["llvm"] as const) {
object_format: target.objectFormat,
minimum_os: target.minimumOs,
},
- runtime_abi: { version: 9, marker: "scr_runtime_abi_v9" },
+ runtime_abi: { version: 12, marker: "scr_runtime_abi_v12" },
compiler: { command: "fixture", identity: "fixture", target: target.llvmTriple },
macros: { executable: [], excluded: [], sanitizer: "external-toolchain-required" },
flavors: {