Merge main

This commit is contained in:
Chris Tate
2026-10-25 15:18:27 -05:00
161 changed files with 4374 additions and 482 deletions
+36 -2
View File
@@ -6,6 +6,42 @@ All notable changes to scriptc will be documented in this file.
<!-- release:start -->
## 0.4.5
### Highlights
- **Fixes:**
- Release builds read a typed array's length without a header load that could read a freed view.
- `net.Socket` `write()` returns Node's backpressure boolean and fires `'drain'`.
- Super calls in JavaScript method values call the declared base method.
- Calling a non-function through an asserted callee throws Node's TypeError.
- **Faster programs:**
- Exceptions propagate by unwinding from directly called functions on Linux, instead of status checks after every call.
- Typed-array loops prove their bounds and keep their views disjoint in more cases, including views over shared memory in worker programs; integer loops stay in integers across nested counted loops, pure helper calls, powers of two and byte reads.
- Lone array callbacks run as loops with their bodies inlined, and row sums run two rows per pass.
- Number-key maps look keys up on their integer view and write updates through the entry the read found; small number-key maps use a direct index.
- Strings: base64 decodes eight characters at a time; Buffers fill, swap byte order and decode `utf16le` a word at a time; readline splits input in one scan; string scans over non-ASCII strings use their UTF-16 units.
- Untyped code: property caches also cache absent properties, object literal members go through the store caches, calls on plain arrays skip the method ladder, and dynamic values dispose without key-cache walks.
- JSON parses into discriminated unions, and dynamic values convert into nested unions, in one pass.
- BigInt division settles quotients from the top limbs, and limb shifts are vectorized.
- **Faster builds:**
- Function bodies are emitted on up to eight workers, including in dev builds, and repeated work in LLVM module emission is cut.
- Linux executables link with lld when the driver finds it, and unchanged executables are reused without selecting the runtime pack.
- Dev builds reuse their frontend output after edits that keep token positions, and unchanged frontend inputs are checked by file identity before hashing.
- Lowering fetches only the syntax trees it reads, and the large compiler tables are built on first use.
- Release builds of the native compiler use a profile collected on the release commit.
- Type checking can use a tsc-ts server shared by later builds (opt-in).
- **Node.js parity:**
- Crypto: RSA key generation lengths, RSA-PSS keys, encrypted private keys, `X509Certificate` fields and methods, and TLS certificates and `tls.Server` contexts.
- zlib: Brotli compression and decompression.
- Dates are mutable objects with Node's setters and conversions; URL hosts and paths parse as the URL Standard does, including non-ASCII hosts with UTS #46.
- `Math`, `JSON` and `Reflect` are real objects in JavaScript; objects can inherit from JavaScript class prototypes; typed arrays held as untyped values support named properties and `Symbol.species`.
- Error messages: `JSON.parse` syntax errors use V8's messages and positions; `Object.getPrototypeOf` of an Error answers `Error.prototype`.
- `util.inspect` of dynamic classes, functions and `%o` arguments, and `Symbol.toStringTag` fields and getters, match Node.
- **npm packages:** more npm and CommonJS dependencies compile statically, including namespace-style packages with guarded declarations and `var` exports, and untyped Maps, Sets and arrays are typed from their uses.
<!-- release:end -->
## 0.4.4
### Highlights
@@ -29,8 +65,6 @@ All notable changes to scriptc will be documented in this file.
- Split CommonJS `require` bindings, `__`-prefixed exports, string-key destructuring of `any` values, and record key order for values JavaScript built.
- **npm packages:** zod 4 and `@hono/zod-validator` values are projected at the declaration boundary.
<!-- release:end -->
## 0.4.3
### Highlights
+1
View File
@@ -32,6 +32,7 @@ Application-shaped workloads for measuring the speed of compiled executables. Ea
| `regex-logs` | cpu | regex `exec` with groups, `test`, global `replace` |
| `regex-replacers` | cpu | regex `replace`/`replaceAll` with inline callbacks whose parameters have no annotations: dictionary placeholders, snake_case to camelCase, date reordering, optional captures, capitalization, and a string search that reads the offset |
| `region-builds` | cpu | a watch-mode build loop over `LifetimeRegion`: each build binds a program in a new region and releases the previous one, which a cache keeps referenced for two builds |
| `string-iteration` | cpu | code point iteration over text that mixes ASCII, Latin-1 letters, CJK and emoji: a for-of word tokenizer, spread reversal, and `Array.from` over words |
| `template-render` | cpu | template literals, escaping, `map`/`join` |
| `throw-chains` | cpu | exceptions caught 2, 5 and 20 recursive calls above the throw, through direct calls and through a callback, and a backtracking parser that throws on a failed alternative |
| `thread-slots`, `thread-slots-cache` | cpu | worker threads reading one published type graph from an `InternTable` and keeping per-thread state for each type they visit: a `ThreadSlots` family, or a direct-mapped cache by id in front of a `Map` by id (the `-cache` entry); under Node.js both are per thread |
@@ -0,0 +1,6 @@
{
"entry": "workloads/string-iteration.ts",
"args": [
"1"
]
}
@@ -0,0 +1,59 @@
// Code point iteration over mixed text: a word tokenizer over for-of, a
// character histogram, reversal through spread, and Array.from over
// words. The text mixes ASCII, Latin-1 letters, CJK and emoji.
let seed = 11;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
const pieces = [
"the", "quick", "brown", "fox", "jumps", "over", "lazy", "dog",
"café", "naïve", "über", "señor", "straße", "déjà", "vu",
"日本", "語", "テキスト", "中文", "한국어",
"😀", "🚀", "👍", "🎉",
];
const punctuation = [" ", " ", " ", ", ", ". ", "! ", "\n"];
const scale = Number(process.argv[2] ?? "1");
const lines: string[] = [];
for (let i = 0; i < Math.floor(4000 * scale); i++) {
let line = "";
const words = 6 + Math.floor(random() * 10);
for (let w = 0; w < words; w++) {
line += pieces[Math.floor(random() * pieces.length)]!;
line += punctuation[Math.floor(random() * punctuation.length)]!;
}
lines.push(line);
}
function isWordChar(c: string): boolean {
const code = c.codePointAt(0)!;
return (code >= 0x61 && code <= 0x7a) || (code >= 0x41 && code <= 0x5a) || code >= 0xc0;
}
let tokens = 0;
let tokenUnits = 0;
let reversedLength = 0;
let fromLength = 0;
const histogram = new Map<string, number>();
for (let round = 0; round < 30; round++) {
for (const line of lines) {
let word = "";
for (const c of line) {
if (isWordChar(c)) {
word += c;
continue;
}
if (word !== "") {
tokens++;
tokenUnits += word.length;
if (round === 0) histogram.set(word, (histogram.get(word) ?? 0) + 1);
fromLength += Array.from(word).length;
word = "";
}
}
reversedLength += [...line].reverse().join("").length;
}
}
const top = [...histogram].sort((a, b) => b[1] - a[1] || (a[0] < b[0] ? -1 : 1)).slice(0, 5);
console.log(top.map(([word, count]) => `${word}:${count}`).join(" "));
console.log("tokens", tokens, "units", tokenUnits, "reversed", reversedLength, "from", fromLength);
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-darwin-arm64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Native scriptc compiler for darwin-arm64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-darwin-x64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Native scriptc compiler for darwin-x64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-arm64-gnu",
"version": "0.4.4",
"version": "0.4.5",
"description": "Native scriptc compiler for linux-arm64-gnu",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-arm64-musl",
"version": "0.4.4",
"version": "0.4.5",
"description": "Native scriptc compiler for linux-arm64-musl",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-x64-gnu",
"version": "0.4.4",
"version": "0.4.5",
"description": "Native scriptc compiler for linux-x64-gnu",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-linux-x64-musl",
"version": "0.4.4",
"version": "0.4.5",
"description": "Native scriptc compiler for linux-x64-musl",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/cli-win32-x64-msvc",
"version": "0.4.4",
"version": "0.4.5",
"description": "Native scriptc compiler for win32-x64-msvc",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "scriptc",
"version": "0.4.4",
"version": "0.4.5",
"description": "Compile ordinary TypeScript and JavaScript to small, fast native executables and WebAssembly modules that run without Node.js",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+2 -2
View File
@@ -72,7 +72,7 @@ describe.runIf(supported)("macOS arm64 native link info", () => {
minimum_os: "14.0",
},
program: { object, entry_symbol: "main" },
runtime_abi: { version: 44, marker: "scr_runtime_abi_v44" },
runtime_abi: { version: 45, marker: "scr_runtime_abi_v45" },
runtime_pack: {
kind: "precompiled",
package: "@scriptc/runtime-darwin-arm64",
@@ -165,6 +165,6 @@ describe.runIf(supported)("macOS arm64 native link info", () => {
() => null,
(failure: { stderr?: string }) => failure,
);
expect(error?.stderr).toContain("scr_runtime_abi_v44");
expect(error?.stderr).toContain("scr_runtime_abi_v45");
});
});
+14 -5
View File
@@ -5345,10 +5345,10 @@ declare module "node:zlib" {
* with the real port discoverable as address().port (the composed read
* is the lowered form — bare address() has no lowering); close([cb])
* firing once connections drain; socket write/end/destroy/pipe and the
* data/end/close/error/connect events (on and once — listeners take at
* most the declared parameter). write/end return void here (Node returns
* the backpressure boolean / the socket); on/once return void (chaining
* is fenced). connect's host is a numeric IP or "localhost" (pinned to
* data/end/close/error/connect/drain events (on and once — listeners
* take at most the declared parameter). write returns Node's
* backpressure boolean; end returns void here (Node returns the socket);
* on/once return void (chaining is fenced). connect's host is a numeric IP or "localhost" (pinned to
* 127.0.0.1 — SEMANTICS.md); every failure is the async 'error' event,
* and an 'error' with no listener exits 1 like an unhandled EventEmitter
* 'error'. */
@@ -5380,7 +5380,13 @@ declare module "net" {
readonly bytesWritten: number;
/* write's callback runs once the data is flushed, with null; end's
* once the socket finishes. */
write(data: string | Uint8Array, callback?: (err?: Error | null) => void): void;
/* false when the bytes this write left unsent reach the 65536-byte
* high-water mark, as in Node; 'drain' fires when they are all sent. */
write(data: string | Uint8Array, callback?: (err?: Error | null) => void): boolean;
/* the bytes written and not yet sent */
readonly writableLength: number;
/* a write answered false and 'drain' has not fired yet */
readonly writableNeedDrain: boolean;
end(callback?: () => void): void;
end(data: string | Uint8Array, callback?: () => void): void;
destroy(): void;
@@ -5428,6 +5434,7 @@ declare module "net" {
| "connect"
| "ready"
| "timeout"
| "drain"
| "readable"
| "finish"
| "secureConnect"
@@ -5452,6 +5459,7 @@ declare module "net" {
| "connect"
| "ready"
| "timeout"
| "drain"
| "readable"
| "finish"
| "secureConnect"
@@ -5475,6 +5483,7 @@ declare module "net" {
| "connect"
| "ready"
| "timeout"
| "drain"
| "readable"
| "finish"
| "secureConnect"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/compiler",
"version": "0.4.4",
"version": "0.4.5",
"description": "The scriptc compiler — TypeScript/JavaScript frontend, typed IR, LLVM backend",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
@@ -453,7 +453,11 @@ export function emitDenseArrayRead(
B.br(join);
B.startBlock(slow);
const slowValue = B.tmp();
const attrs = acc === "ref" && retain ? " cold" : READ_FALLBACK;
// ToNumber of a foreign element (SCR_ARR_FOREIGN) can run user code.
const attrs =
(acc === "ref" && retain) || (getter === "scr_arr_get_number" && host.mayHoldForeign(element))
? " cold"
: READ_FALLBACK;
B.line(`${slowValue} = call ${ty} @${getter}(ptr ${array}, double ${index.name})${attrs}`);
let slowInt: string | null = null;
if (fastInt !== null) {
@@ -610,6 +614,28 @@ export function emitDenseArraySet(
B.line(`store ptr ${value}, ptr ${valuePtr}${host.fieldAliasAttachment(valuePtr)}`);
}
};
if ((acc === "f64" || kind === "ref") && host.mayHoldForeign(element)) {
// Past the hole-free prefix a slot may hold a foreign element, whose
// value the runtime setter releases; below it every slot is a value.
B.br(slow);
B.startBlock(filledLabel);
const filledValuePtr = valuePointer(host, guard, kind);
let filledOld: string | null = null;
if (kind === "ref") {
filledOld = B.tmp();
B.line(
`${filledOld} = load ptr, ptr ${filledValuePtr}${host.fieldAliasAttachment(filledValuePtr)}`,
);
}
storeValue(filledValuePtr);
if (filledOld) host.releaseHeap(filledOld, element);
B.br(join);
B.startBlock(slow);
call(" cold");
B.br(join);
B.startBlock(join);
return;
}
const stated = B.newLabel("arr.set.states"),
plain = B.newLabel("arr.set.plain"),
plainStore = B.newLabel("arr.set.values"),
+100 -1
View File
@@ -1980,6 +1980,87 @@ export class LlDyn {
/** `sc_dc_<n>(ptr d, ptr path) -> T` — validate the checked-dynamic tree against T and
* BUILD the typed value (+1), or throw the catchable path-annotated
* TypeError and return a dummy with the pending flag set. */
/** The array a live view's snapshot commits back into, for element types
* whose arrays can hold foreign elements (LlEmitter.mayHoldForeign):
* like dynCheckHelper's array case on a checked array `%d`, except that
* an element the element type cannot hold is stored as a foreign element
* (undefined as a present undefined) instead of failing the conversion.
* Boolean arrays, whose slots cannot hold one, keep the failing check. */
dynViewArrayHelper(t: IrType & { kind: "array" }): string {
const builderKey = `view:${typeKey(t)}`;
const existing = this.dynBuilders.get(builderKey);
if (existing) return existing;
const name = `sc_dva_${this.dynBuilders.size}`;
this.dynBuilders.set(builderKey, name);
const host = this.host;
const elem = t.elem;
const check = this.dynCheckHelper(elem);
const match = this.dynMatchHelper(elem);
const B = new BlockBuilder();
const n = this.lenOf(B, "%d");
const a = B.tmp();
B.line(`${a} = ${arrNewCall(host, elem, n)}`);
const items = this.itemsOf(B, "%d");
const acc = elemAccess(elem);
const accTy = acc === "f64" ? "double" : acc === "bool" ? "i1" : "ptr";
host.declare(
`declare double @scr_arr_push_${acc}(ptr, ${acc === "bool" ? "i1 zeroext" : accTy})`,
);
host.declare(`declare zeroext i1 @scr_dyn_arr_has_index(ptr, ${this.S})`);
host.declare(`declare void @scr_arr_set_len(ptr, double)`);
host.declare(`declare zeroext i1 @scr_arr_set_foreign(ptr, double, ptr)`);
this.i64Loop(B, "dva", n, (i) => {
const index = B.tmp();
B.line(`${index} = ${sizeToDouble(this.S)} ${this.S} ${i} to double`);
const present = B.tmp();
B.line(`${present} = call zeroext i1 @scr_dyn_arr_has_index(ptr %d, ${this.S} ${i})`);
const valueLabel = B.newLabel("dva.present");
const holeLabel = B.newLabel("dva.hole");
const typedLabel = B.newLabel("dva.typed");
const foreignLabel = B.newLabel("dva.foreign");
const doneLabel = B.newLabel("dva.done");
B.condBr(present, valueLabel, holeLabel);
B.startBlock(holeLabel);
const length = B.tmp();
B.line(`${length} = fadd double ${index}, 1.0`);
B.line(`call void @scr_arr_set_len(ptr ${a}, double ${length})`);
B.br(doneLabel);
B.startBlock(valueLabel);
const e = this.itemAt(B, items, i);
const fits = B.tmp();
B.line(`${fits} = call zeroext i1 @${match}(ptr ${e})`);
B.condBr(fits, typedLabel, foreignLabel);
B.startBlock(foreignLabel);
const kept = B.tmp();
B.line(`${kept} = call zeroext i1 @scr_arr_set_foreign(ptr ${a}, double ${index}, ptr ${e})`);
B.condBr(kept, doneLabel, typedLabel);
B.startBlock(typedLabel);
const v = B.tmp();
B.line(`${v} = call ${this.valTy(elem)} @${check}(ptr ${e}, ptr null)`);
this.pendingBail(
B,
"dva",
() => {
host.declare(`declare void @scr_arr_release(ptr)`);
B.line(`call void @scr_arr_release(ptr ${a})`);
},
"ptr null",
);
const pushed = B.tmp();
B.line(`${pushed} = call double @scr_arr_push_${acc}(ptr ${a}, ${accTy} ${v})`);
B.br(doneLabel);
B.startBlock(doneLabel);
});
B.terminate(`ret ptr ${a}`);
this.defs.push(
`define internal ptr @${name}(ptr %d, ptr %path) ${FN_ATTRS} { ; view commit ${typeKey(t)}`,
B.render(),
`}`,
``,
);
return name;
}
dynCheckHelper(t: IrType, preserveRefs = false): string {
// A `(T | undefined)[]` stored as T elements also takes undefined ones.
const key = checkedConversionKey(t);
@@ -3686,7 +3767,25 @@ export class LlDyn {
B.br(doneLabel);
B.startBlock(presentLabel);
B.line(`${undefinedState} = fcmp oeq double ${state}, 2.0`); // SCR_ARR_UNDEFINED
B.terminate(`br i1 ${undefinedState}, label %${undefinedLabel}, label %${valueLabel}`);
if (host.mayHoldForeign(t.elem)) {
// A foreign element (SCR_ARR_FOREIGN) holds its own value.
const typedLabel = B.newLabel("tda.typed");
const foreignLabel = B.newLabel("tda.foreign");
B.terminate(`br i1 ${undefinedState}, label %${undefinedLabel}, label %${typedLabel}`);
B.startBlock(typedLabel);
const foreign = B.tmp();
B.line(`${foreign} = fcmp oeq double ${state}, 3.0`);
B.condBr(foreign, foreignLabel, valueLabel);
B.startBlock(foreignLabel);
host.declare(`declare ptr @scr_arr_get_foreign(ptr, double)`);
host.declare(`declare void @scr_dyn_arr_push(ptr, ptr)`);
const payload = B.tmp();
B.line(`${payload} = call ptr @scr_arr_get_foreign(ptr %v, double ${i}) ; +1`);
B.line(`call void @scr_dyn_arr_push(ptr ${d}, ptr ${payload})`);
B.br(doneLabel);
} else {
B.terminate(`br i1 ${undefinedState}, label %${undefinedLabel}, label %${valueLabel}`);
}
B.startBlock(undefinedLabel);
const retained = this.retainDyn(B, this.undef(B));
B.line(`call void @scr_dyn_arr_push(ptr ${d}, ptr ${retained})`);
@@ -600,6 +600,23 @@ export function emitModuleParts(emitter: LlEmitter): string[] {
regexMatchers.set(key, code);
for (const decl of code.decls) emitter.declare(decl);
}
// Replacers of global substitutions with a literal replacement; the call
// sites register only patterns inside the compiled subset.
const regexReplacers: RegexMatcherCode[] = [];
for (const rr of emitter.regexReplacers.values()) {
const code = compileRegexMatcher(
rr.sym,
rr.pattern,
rr.flags,
emitter.sizeType,
!emitter.wasi,
rr.replacement,
);
if (code === null)
throw new InternalCompilerError(`llvm emitter bug: replacer ${rr.sym} did not compile`);
regexReplacers.push(code);
for (const decl of code.decls) emitter.declare(decl);
}
const out: string[] = [
`; Generated by scriptc (LLVM backend) from ${emitter.mod.sourceFile}. Do not edit.`,
@@ -882,6 +899,7 @@ export function emitModuleParts(emitter: LlEmitter): string[] {
}
if (emitter.regexInstances.size > 0) out.push(``);
for (const code of regexMatchers.values()) out.push(...code.lines, ``);
for (const code of regexReplacers) out.push(...code.lines, ``);
for (const [, inst] of emitter.templateStringsInstances) {
// One immortal ScrArr per tagged-template site: a data global of
// interned cooked-string literals, and the ScrArr header over it
@@ -1303,6 +1303,12 @@ export class LlEmitter {
string,
{ sym: string; src: string; fl: string; pattern: string; flags: string }
>();
/** Build-time replacers: one per (pattern, flags, replacement) of a
* global substitution with a literal replacement (regex-matchers.ts). */
readonly regexReplacers = new Map<
string,
{ sym: string; pattern: string; flags: string; replacement: string }
>();
/** Interned tagged-template strings objects: per-site key → { symbol,
* interned cooked-literal refs }. One immortal ScrArr of string slots
* per template SITE (the spec's per-occurrence identity — the C
@@ -1633,6 +1639,9 @@ export class LlEmitter {
* commit back into their objects. Library builds keep their commits:
* their values cross a native boundary. */
readonly liveViewsReadOnly: boolean;
/** IrModule.foreignElementKeys: element keys whose arrays can hold
* SCR_ARR_FOREIGN elements. */
readonly foreignElementKeys: ReadonlySet<string>;
/** The program freezes, seals or prevents extensions of untyped values
* (IrModule.untypedIntegrity): the commits of class views store the
* view's integrity state in the instance's integrity word. */
@@ -1925,6 +1934,7 @@ export class LlEmitter {
!this.untypedIntegrity &&
!dynamicStores.all &&
dynamicStores.names.size === 0;
this.foreignElementKeys = new Set(mod.foreignElementKeys ?? []);
this.int32Abis = this.debug === null ? findInt32Abis(mod, this.int32Slots) : new Map();
this.immortalGlobals = new Set(mod.contracts?.immortalGlobals ?? []);
this.programLifetime = new Set(mod.contracts?.programLifetime ?? []);
@@ -2219,6 +2229,7 @@ export class LlEmitter {
declare: (decl) => this.declare(decl),
needOom: () => this.needOom(),
needBadTag: () => this.needBadTag(),
mayHoldForeign: (elem) => this.mayHoldForeign(elem),
internLiteral: (text) => this.internLiteral(text),
cstr: (text) => this.cstr(text),
mapOps: (operands) => this.mapOps(operands),
@@ -5439,6 +5450,8 @@ export class LlEmitter {
return addNumbered(this.threadSlotsBlocks, key, value as string, "@sc_slots_", names);
case "regex":
return addNumberedSymbol(this.regexInstances, key, value as never, "sc_re_", names);
case "regexReplacer":
return addNumberedSymbol(this.regexReplacers, key, value as never, "sc_rr_", names);
case "templateStrings":
return addNumberedSymbol(
this.templateStringsInstances,
@@ -6716,6 +6729,12 @@ export class LlEmitter {
}
}
/** Arrays of this element type can hold SCR_ARR_FOREIGN elements. */
mayHoldForeign(elem: IrType): boolean {
if (this.foreignElementKeys.size === 0) return false;
return this.foreignElementKeys.has("*") || this.foreignElementKeys.has(typeKey(elem));
}
/** Lifetime regions in modules with lifetimeRegions (llvm/regions.ts).
* Created with the class graph; null elsewhere. */
regions: LifetimeRegions | null = null;
@@ -1,3 +1,4 @@
import { dynDesc } from "../../ir/analysis.js";
/* Focused LLVM expression emission extracted from emitter.ts. */
import { arraySearchHelperName, arraySearchKind } from "./array-search.js";
import { InternalCompilerError } from "../../errors.js";
@@ -49,11 +50,13 @@ import { emitFusedMapSet, rememberMapEntry } from "./map-entry-fusion.js";
import { emitScratchStringSlice, emitStringSliceRead } from "./string-slices.js";
import {
STRING_CHAR_AT_DECL,
STRING_CP_AT_BYTE_DECL,
STRING_CHAR_AT_INT_MARKER,
STRING_CHAR_CODE_AT_DECL,
STRING_CHAR_CODE_AT_INT_MARKER,
emitAsciiChar,
emitAsciiCharCode,
emitStringByteLength,
emitUnitCharCode,
STRING_UTF16_LEN_DECL,
} from "./string-reads.js";
@@ -564,16 +567,30 @@ export function emitStrIntrinsic(
return call("scr_str_is_well_formed", "zeroext i1 (ptr)", `ptr ${r.name}`, "i1", false);
case "toWellFormed":
return call("scr_str_to_well_formed", "ptr (ptr)", `ptr ${r.name}`, "ptr", true);
case "cpAt":
// The code point AT an index as a one-code-point string (+1) —
// the string-for-of desugar's read.
return call(
"scr_str_cp_at",
"ptr (ptr, double)",
`ptr ${r.name}, double ${args[0]!.name}`,
"ptr",
true,
);
case "byteLength": {
const t = B.tmp();
// A byte count fits in a signed size, whose conversion is one instruction.
B.line(`${t} = sitofp ${host.sizeType} ${emitStringByteLength(host, r.name)} to double`);
return { name: t, type: e.type };
}
case "cpAtByte": {
// sc_str_cp_at_byte (string-reads.ts) answers ASCII bytes from the
// charAt table inline. The cursor is a character start, an exact
// integer below the byte count.
host.declare(STRING_CHAR_AT_DECL);
host.declare(STRING_CP_AT_BYTE_DECL);
const index = e.args[0] ? exactInteger(host, args[0]!, e.args[0]) : null;
let wide: string;
if (index) {
wide = widenInteger(host, index);
} else {
wide = B.tmp();
B.line(`${wide} = fptosi double ${args[0]!.name} to i64`);
}
const t = B.tmp();
B.line(`${t} = call ptr @sc_str_cp_at_byte(ptr ${r.name}, i64 ${wide})`);
return host.own({ name: t, type: e.type });
}
default: {
const _exhaustive: never = method;
void _exhaustive;
@@ -790,6 +807,9 @@ export function emitArrIntrinsic(
case "shift": {
host.emitPublishedArrayGuard(r.name, e.receiver.type, method === "pop" ? 2 : 6);
const checkedValue = elem.kind === "dyn" && e.type.kind === "dyn";
// A foreign element (SCR_ARR_FOREIGN) cannot be the typed result:
// removing one throws the checked conversion's TypeError.
const foreignElements = !checkedValue && host.mayHoldForeign(elem);
if (!checkedValue && e.type.kind !== "union")
throw new InternalCompilerError("llvm emitter bug: array removal result is not a union");
const def = e.type.kind === "union" ? host.unionsById.get(e.type.unionId) : undefined;
@@ -832,7 +852,25 @@ export function emitArrIntrinsic(
);
B.line(`${slowRaw} = load i64, ptr ${rawSlot}`);
B.line(`${has} = icmp eq i8 ${state}, 1`);
B.condBr(has, lp, la);
const lf = foreignElements ? B.newLabel("remove.f") : la;
B.condBr(has, lp, lf);
if (foreignElements) {
// Only the runtime call removes a foreign element; its value moves
// out with the slot, and the throw releases it.
B.startBlock(lf);
const foreign = B.tmp();
const throwLabel = B.newLabel("remove.foreign");
B.line(`${foreign} = icmp eq i8 ${state}, 3`);
B.condBr(foreign, throwLabel, la);
B.startBlock(throwLabel);
const payload = B.tmp();
B.line(`${payload} = inttoptr i64 ${slowRaw} to ptr`);
host.declare(`declare void @scr_dyn_foreign_throw(ptr, ptr)`);
const want = host.cstr(dynDesc(e.type, host.recordsById, host.unionsById));
B.line(`call void @scr_dyn_foreign_throw(ptr ${payload}, ptr ${want})`);
host.emitAlwaysPendingUnwind();
B.br(la);
}
B.startBlock(lp);
let raw = slowRaw;
const value = B.tmp();
@@ -661,6 +661,29 @@ export function regexLiteralTemplate(
return `@${re.sym}`;
}
/** The build-time replacer of a global substitution of regex literal `e`
* with the literal `replacement` (regex-matchers.ts), registered on first
* use. */
export function regexLiteralReplacer(
host: LlvmEmitterContext,
e: IrExpr & { kind: "regexLit" },
replacement: string,
): string {
const key = `${e.flags}/${e.pattern}\0${replacement}`;
let rr = host.regexReplacers.get(key);
if (!rr) {
rr = {
sym: `sc_rr_${host.regexReplacers.size}`,
pattern: e.pattern,
flags: e.flags,
replacement,
};
host.regexReplacers.set(key, rr);
}
host.journal?.entry("regexReplacer", key, rr);
return `@${rr.sym}`;
}
/** A string equality or comparison between `s.charAt(i)` and a literal of
* exactly one UTF-16 code unit: the charAt call, the unit, and whether the
* charAt is the left operand. */
@@ -6,7 +6,8 @@ import { LlvmUnsupportedError } from "./unsupported.js";
import type { LlvmEmitterContext, LlValue } from "./expr-context.js";
import { f64Lit, llvmCommentText } from "./common.js";
import { borrowableInputs, emitBorrowedInputs, emitBorrowedInput } from "./borrowed-inputs.js";
import { regexLiteralTemplate } from "./expr-primitives.js";
import { regexLiteralReplacer, regexLiteralTemplate } from "./expr-primitives.js";
import { canReplaceWith, regexLiteralCompiles } from "./regex-matchers.js";
import { isScalarUnionDef, loadInlineUnion, presentBits } from "./inline-unions.js";
import { preservesRegexInputs } from "./reference-effects.js";
@@ -160,6 +161,39 @@ function directRegexOperandMethod(method: string): boolean {
}
}
/** The build-time replacer of a substitution: a regex literal running on
* its template, with a literal replacement the replacer can write as is.
* Null when the substitution takes the runtime's loop. */
function literalReplacer(
host: LlvmEmitterContext,
literal: IrExpr | null,
template: string | null,
replacement: IrExpr,
): string | null {
if (literal === null || template === null || literal.kind !== "regexLit") return null;
if (replacement.kind !== "strLit") return null;
if (!canReplaceWith(literal.pattern, literal.flags, replacement.value)) return null;
if (!regexLiteralCompiles(literal.pattern, literal.flags)) return null;
return regexLiteralReplacer(host, literal, replacement.value);
}
/** scr_regex_replace_with: the runtime runs `replacer` on an ASCII subject
* and its own loop on any other. */
function emitReplaceWith(
host: LlvmEmitterContext,
e: IrExpr,
r: LlValue,
args: LlValue[],
replacer: string,
): LlValue {
host.declare(`declare ptr @scr_regex_replace_with(ptr, ptr, ptr, ptr)`);
const t = host.B.tmp();
host.B.line(
`${t} = call ptr @scr_regex_replace_with(ptr ${r.name}, ptr ${args[0]!.name}, ptr ${args[1]!.name}, ptr ${replacer})`,
);
return host.own({ name: t, type: e.type });
}
export function emitRegexIntrinsic(
host: LlvmEmitterContext,
e: IrExpr & { kind: "regexIntrinsic" },
@@ -234,11 +268,16 @@ export function emitRegexIntrinsic(
"scr_regex_match_all_captures_into",
`ptr ${r.name}, ptr ${args[0]!.name}, ptr ${args[1]!.name}`,
);
case "replaceAll":
case "replaceAll": {
// A global literal (the only kind a replacer is built for) never takes
// replaceAll's non-global TypeError.
const replacer = literalReplacer(host, literal, template, e.args[1]!);
if (replacer !== null) return emitReplaceWith(host, e, r, args, replacer);
return fallible(
"scr_regex_replace_all",
`ptr ${r.name}, ptr ${args[0]!.name}, ptr ${args[1]!.name}`,
);
}
case "split":
host.declare(`declare ptr @scr_regex_split_limit(ptr, ptr, double)`);
{
@@ -323,6 +362,8 @@ export function emitRegexIntrinsic(
return host.own({ name: t, type: e.type });
}
case "replace": {
const replacer = literalReplacer(host, literal, template, e.args[1]!);
if (replacer !== null) return emitReplaceWith(host, e, r, args, replacer);
host.declare(`declare ptr @scr_regex_replace(ptr, ptr, ptr)`);
const t = B.tmp();
B.line(
@@ -184,7 +184,11 @@ export function streamTypedRefCommitAdapter(
}
if (t.kind === "array") {
const commit = `${snapshot}_commit`;
const check = host.dyn.dynCheckHelper(t);
// A value the element type cannot hold stays in the array as a foreign
// element where the program can store one (LlEmitter.mayHoldForeign).
const check = host.mayHoldForeign(t.elem)
? host.dyn.dynViewArrayHelper(t)
: host.dyn.dynCheckHelper(t);
host.declare(`declare void @scr_arr_release(ptr)`);
host.declare(`declare void @scr_arr_outline(ptr)`);
host.declare(`declare void @scr_arr_adopt_view_properties(ptr, ptr, ptr)`);
@@ -1235,6 +1239,21 @@ export function streamTypedRefMaterializeAdapter(
B.line(`call void @scr_dyn_arr_push_hole(ptr ${out})`);
B.br(doneLabel);
B.startBlock(undefinedLabel);
if (host.mayHoldForeign(elem)) {
// A foreign element (SCR_ARR_FOREIGN) holds its own value.
const foreign = B.tmp();
const foreignLabel = B.newLabel("live.array.foreign");
const presentUndefined = B.newLabel("live.array.present.undefined");
B.line(`${foreign} = fcmp oeq double ${state}, 3.0`);
B.condBr(foreign, foreignLabel, presentUndefined);
B.startBlock(foreignLabel);
host.declare(`declare ptr @scr_arr_get_foreign(ptr, double)`);
const payload = B.tmp();
B.line(`${payload} = call ptr @scr_arr_get_foreign(ptr %p, double ${index}) ; +1`);
B.line(`call void @scr_dyn_arr_push(ptr ${out}, ptr ${payload})`);
B.br(doneLabel);
B.startBlock(presentUndefined);
}
const undefinedValue = B.tmp();
const retained = B.tmp();
B.line(`${undefinedValue} = call ptr @scr_dyn_undefined()`);
@@ -1333,7 +1352,16 @@ function liveArrayOps(
const refuse = S.newLabel("live.array.refuse");
S.condBr(fits, store, refuse);
S.startBlock(refuse);
S.terminate(`ret i1 false`);
if (host.mayHoldForeign(elem)) {
// The array keeps a value its element type cannot hold as a foreign
// element, and undefined as a present undefined.
host.declare(`declare zeroext i1 @scr_arr_set_foreign(ptr, double, ptr)`);
const kept = S.tmp();
S.line(`${kept} = call zeroext i1 @scr_arr_set_foreign(ptr %p, double %i, ptr %v)`);
S.terminate(`ret i1 ${kept}`);
} else {
S.terminate(`ret i1 false`);
}
S.startBlock(store);
const next = S.tmp();
S.line(
+6 -2
View File
@@ -406,9 +406,13 @@ function emitGet(host: Host, e: ExprOf<"call">, callee: IrFunction, key: string)
inlineLoad(host, receiver.name, ic, INLINE_WAYS, INLINE_PROTO, (cell) => cell, hits, hit, probe);
B.startBlock(probe);
// The read's probe also answers keys the receiver's plain chain lacks.
host.declare(`declare ptr @scr_js_get_probe(ptr, ptr, ptr)`);
// Only `length` and `next` reads can be answered for a native array or
// iterator, so only those sites take the probe that tests them.
const probeFn =
key === "length" || key === "next" ? "scr_js_get_probe_native" : "scr_js_get_probe";
host.declare(`declare ptr @${probeFn}(ptr, ptr, ptr)`);
const probed = B.tmp();
B.line(`${probed} = call ptr @scr_js_get_probe(ptr ${receiver.name}, ptr ${name}, ptr ${ic})`);
B.line(`${probed} = call ptr @${probeFn}(ptr ${receiver.name}, ptr ${name}, ptr ${ic})`);
const found = B.tmp();
B.line(`${found} = icmp ne ptr ${probed}, null`);
const probeEnd = B.currentLabel();
@@ -1143,6 +1143,8 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue {
case "blob":
case "wasm":
case "array":
if (e.fn === "array.foreignThrow") return emitArrayForeignThrow(host, e);
return host.emitGenericLibCall(e);
case "collection":
case "record":
case "disposable":
@@ -1164,6 +1166,29 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue {
}
}
/** `array.foreignThrow`: the checked conversion's TypeError for a foreign
* element read into a typed result. Always throws; the result is a dummy
* of the replaced read's type (the error.argTypeThrow pattern). */
function emitArrayForeignThrow(host: LlvmEmitterContext, e: LibCallExpr): LlValue {
const B = host.B;
const arr = host.emitExpr(e.args[0]!);
const index = host.emitExpr(e.args[1]!);
const want = host.emitExpr(e.args[2]!);
host.declare(`declare void @scr_arr_foreign_throw(ptr, double, ptr)`);
B.line(
`call void @scr_arr_foreign_throw(ptr ${arr.name}, double ${index.name}, ptr ${want.name})`,
);
const ty = host.llType(e.type);
if (ty === "void") {
host.emitAlwaysPendingUnwind();
return { name: "", type: e.type };
}
const dummy = ty === "double" ? "0.0" : ty === "i1" ? "false" : "null";
const out = host.own({ name: dummy, type: e.type });
host.emitAlwaysPendingUnwind();
return out;
}
/** The settlement adapter of Promise.all's tuple form for the builder
* `fnName`, which fulfills the result with the builder's tuple. */
function allTupleAdapterFor(host: LlvmEmitterContext, fnName: string, tuple: IrType): string {
@@ -309,6 +309,7 @@ export function emitNetworkHttpLibCall(host: LlvmEmitterContext, e: LibCallExpr)
e.fn === "net.sockOnClose" ||
e.fn === "net.sockOnConnect" ||
e.fn === "net.sockOnTimeout" ||
e.fn === "net.sockOnDrain" ||
e.fn === "net.sockOnReadable" ||
e.fn === "net.sockOnReady" ||
e.fn === "http.reqOnEnd" ||
@@ -329,6 +330,7 @@ export function emitNetworkHttpLibCall(host: LlvmEmitterContext, e: LibCallExpr)
"net.sockOnClose": "scr_net_sock_on_close",
"net.sockOnConnect": "scr_net_sock_on_connect",
"net.sockOnTimeout": "scr_net_sock_on_timeout",
"net.sockOnDrain": "scr_net_sock_on_drain",
"net.sockOnReadable": "scr_net_sock_on_readable",
"net.sockOnReady": "scr_net_sock_on_ready",
"http.reqOnEnd": "scr_http_req_on_end",
@@ -104,6 +104,7 @@ export const LIB_FN_SYMS: Record<string, string> = {
"str.localeCase": "scr_str_locale_case",
"str.replaceLiteral": "scr_str_replace_literal",
"str.replaceChars": "scr_str_replace_chars",
"str.codePoints": "scr_str_code_points",
"num.fromString": "scr_string_to_number",
"math.round": "scr_math_round",
"math.sin": "sin",
@@ -1337,6 +1338,8 @@ export const LIB_FN_SYMS: Record<string, string> = {
"net.sockResetAndDestroy": "scr_net_sock_reset_and_destroy",
"net.sockAddress": "scr_net_sock_address",
"net.sockBytesWritten": "scr_net_sock_bytes_written",
"net.sockWritableLength": "scr_net_sock_writable_length",
"net.sockWritableNeedDrain": "scr_net_sock_writable_need_drain",
"net.sockAddrField": "scr_net_sock_addr_field",
"net.sockBytesRead": "scr_net_sock_bytes_read",
"net.sockConnecting": "scr_net_sock_connecting",
@@ -1661,6 +1664,7 @@ export const LIB_FN_SYMS: Record<string, string> = {
"regex.groupParseInt": "scr_regex_group_parse_int",
"regex.resultProp": "scr_regex_result_prop",
"array.namedKeys": "scr_arr_named_keys",
"array.foreign": "scr_arr_get_foreign",
"array.hasNamedKey": "scr_arr_has_named_key",
"array.hasProperties": "scr_arr_has_properties",
"array.undefinedKey": "scr_arr_undefined_key",
@@ -107,6 +107,7 @@ const INERT_LIB_CALLS = new Set([
"regex.groupParseInt",
"regex.resultProp",
"array.namedKeys",
"array.foreign",
"array.hasNamedKey",
"array.hasProperties",
"process.cwd",
@@ -1,5 +1,6 @@
import { describe, expect, test } from "vitest";
import {
canReplaceWith,
compileRegexMatcher,
regexLiteralBailReason,
regexLiteralCaptures,
@@ -152,3 +153,46 @@ describe("regex literal matcher code", () => {
expect(text).not.toContain("constant [128 x i8]");
});
});
describe("regex literal replacers", () => {
test("take global, non-sticky, non-plain-text patterns and ASCII replacements without $", () => {
expect(canReplaceWith("a[bc]", "g", "<2>")).toBe(true);
expect(canReplaceWith("a[bc]", "gi", "")).toBe(true);
expect(canReplaceWith("a[bc]", "", "x")).toBe(false);
expect(canReplaceWith("a[bc]", "gy", "x")).toBe(false);
expect(canReplaceWith("a[bc]", "g", "$1")).toBe(false);
expect(canReplaceWith("a[bc]", "g", "é")).toBe(false);
// Plain text is a string search at run time.
expect(canReplaceWith("&", "g", "&amp;")).toBe(false);
expect(canReplaceWith("a\\.b", "g", "x")).toBe(false);
// Letters under i are not plain text.
expect(canReplaceWith("ab", "gi", "x")).toBe(true);
});
test("append the gap and the replacement and continue the search", () => {
const code = compileRegexMatcher("r", "<[^>]*>", "g", "i64", true, "|");
const text = code!.lines.join("\n");
expect(text).toContain('@r.rep = private unnamed_addr constant [1 x i8] c"|"');
expect(text).toContain("call void @scr_re_replace_out_reserve(ptr %out, i64 %m.need)");
expect(text).toContain("load <16 x i8>, ptr %m.src, align 1");
expect(text).toMatch(/br i1 %m\.past, label %nomatch, label %scan/);
expect(text).not.toContain("m.ret:");
expect(text).toContain("ret i32 %n.found");
// The search after a match is a 16-byte scan, not a memchr call.
expect(text).not.toContain("@memchr");
expect(code!.decls).toContain("declare void @scr_re_replace_out_reserve(ptr, i64)");
// Without vectors the gap is a memcpy.
const scalar = compileRegexMatcher("r", "<[^>]*>", "g", "i64", false, "|")!.lines.join("\n");
expect(scalar).not.toContain("<16 x i8>");
expect(scalar).toContain(
"call void @llvm.memcpy.p0.p0.i64(ptr %m.dst, ptr %m.src, i64 %m.gapw, i1 false)",
);
});
test("escape the replacement bytes and skip the copy of an empty one", () => {
const quoted = compileRegexMatcher("r", "a[bc]", "g", "i64", true, '"\\\n')!.lines.join("\n");
expect(quoted).toContain('c"\\22\\5C\\0A"');
const empty = compileRegexMatcher("r", "a[bc]", "g", "i64", true, "")!.lines.join("\n");
expect(empty).not.toContain("@r.rep");
});
});
@@ -24,6 +24,8 @@
* `len`, it returns the capture count (whole match included) so the runtime
* can check it against the compiled bytecode. */
import { InternalCompilerError } from "../../errors.js";
const ASCII = 128;
const INF = -1;
/** Larger patterns keep the interpreter (binary size). */
@@ -1853,10 +1855,13 @@ export function compileRegexMatcher(
flags: string,
sizeType: string,
vector: boolean,
replacement: string | null = null,
): RegexMatcherCode | null {
const parsed = parseSubset(pattern, flags);
if (parsed === null) return null;
const sticky = flags.includes("y");
if (replacement !== null && !canReplaceWith(pattern, flags, replacement))
throw new InternalCompilerError(`llvm emitter bug: no replacer for /${pattern}/${flags}`);
const { root, ncap } = parsed;
const gen = new RxGen(sym, ncap, vector);
const end: FirstInfo = { set: emptySet(), nullable: true, bol: false, uncond: true };
@@ -1873,6 +1878,12 @@ export function compileRegexMatcher(
fn.push(` %stk = alloca ptr`);
fn.push(` %sbuf = alloca [64 x i32]`);
for (let r = 0; r < gen.regs; r++) fn.push(` %r${r} = alloca i32`);
if (replacement !== null) {
fn.push(` %next = alloca i32`);
fn.push(` %nfound = alloca i32`);
fn.push(` store i32 0, ptr %next`);
fn.push(` store i32 0, ptr %nfound`);
}
fn.push(` store ptr %sbuf, ptr %stk`);
fn.push(` store i32 64, ptr %scap`);
fn.push(` store i32 %start, ptr %at`);
@@ -1900,7 +1911,9 @@ export function compileRegexMatcher(
} else {
fn.push(` br label %seek`);
}
if (firstCount === 1) {
// A replacer seeks again after every match, often a few bytes on: the
// 16-byte scan avoids a memchr call per match.
if (firstCount === 1 && replacement === null) {
fn.push(`seek:`);
const byte = intervals(first.set)[0]![0]!;
fn.push(` %rest = sub i32 %len, %a`);
@@ -1944,6 +1957,98 @@ export function compileRegexMatcher(
fn.push(` br i1 %over, label %nomatch, label %scan`);
}
if (replacement !== null) {
emitReplaceMatch(fn, sym, replacement, vector);
} else {
emitReturnMatch(fn, ncap);
}
const result = replacement !== null ? "%n.found" : "0";
fn.push(`nomatch:`);
if (replacement !== null) {
fn.push(` %n.next = load i32, ptr %next`);
fn.push(` %n.nextp = getelementptr inbounds i8, ptr %out, i64 16`);
fn.push(` %n.nextw = sext i32 %n.next to i64`);
fn.push(` store i64 %n.nextw, ptr %n.nextp`);
fn.push(` %n.found = load i32, ptr %nfound`);
}
fn.push(` %n.stk = load ptr, ptr %stk`);
fn.push(` %n.heap = icmp ne ptr %n.stk, %sbuf`);
fn.push(` br i1 %n.heap, label %n.free, label %n.ret`);
fn.push(`n.free:`);
fn.push(` call void @scr_regex_stack_free(ptr %n.stk)`);
fn.push(` br label %n.ret`);
fn.push(`n.ret:`);
fn.push(` ret i32 ${result}`);
gen.emitTail();
for (const l of gen.body) fn.push(l);
fn.push(`}`);
const decls = REGEX_MATCHER_DECLS.slice();
if (gen.body.some((l) => l.includes("@llvm.cttz.i16")))
decls.push(`declare i16 @llvm.cttz.i16(i16, i1 immarg)`);
if (!sticky && !first.nullable && firstCount === 1 && replacement === null)
decls.push(`declare ptr @memchr(ptr, i32, ${sizeType})`);
if (replacement !== null) {
decls.push(`declare void @scr_re_replace_out_reserve(ptr, i64)`);
decls.push(`declare void @llvm.memcpy.p0.p0.i64(ptr, ptr, i64, i1 immarg)`);
}
const globals = [...gen.globals];
if (replacement !== null && replacement.length > 0)
globals.push(
`@${sym}.rep = private unnamed_addr constant [${replacement.length} x i8] c"${llvmBytes(replacement)}"`,
);
return { lines: [...globals, ...fn], decls };
}
/** Whether a substitution can run through a build-time replacer: a global,
* non-sticky pattern that is not plain text (the runtime searches those as
* strings) and an ASCII replacement without `$` patterns. */
export function canReplaceWith(pattern: string, flags: string, replacement: string): boolean {
if (!flags.includes("g") || flags.includes("y") || isPlainTextPattern(pattern, flags))
return false;
for (let i = 0; i < replacement.length; i++) {
const c = replacement.charCodeAt(i);
if (c >= ASCII || c === 0x24) return false;
}
return true;
}
/** scr_regex_plain_text's rule: a pattern of literal characters and escaped
* metacharacters (no letters under `i`) that the runtime searches as text. */
function isPlainTextPattern(pattern: string, flags: string): boolean {
const fold = flags.includes("i");
if (pattern.length === 0 || pattern.length > 256) return false;
for (let i = 0; i < pattern.length; i++) {
let c = pattern[i]!;
if (c === "\\") {
if (i + 1 >= pattern.length) return false;
c = pattern[++i]!;
if (!"^$\\.*+?()[]{}|/".includes(c)) return false;
} else {
const code = c.charCodeAt(0);
if (code < 0x20 || code >= 0x7f || "^$.*+?()[]{}|".includes(c)) return false;
}
if (fold && /[a-z]/i.test(c)) return false;
}
return true;
}
function llvmBytes(text: string): string {
let out = "";
for (let i = 0; i < text.length; i++) {
const c = text.charCodeAt(i);
out +=
c >= 0x20 && c < 0x7f && c !== 0x22 && c !== 0x5c
? text[i]
: `\\${c.toString(16).toUpperCase().padStart(2, "0")}`;
}
return out;
}
/** The match block of a matcher: store the capture pointers, return 1. */
function emitReturnMatch(fn: string[], ncap: number): void {
fn.push(`match:`);
fn.push(` %m.end = load i32, ptr %pos`);
fn.push(` %m.start = load i32, ptr %at`);
@@ -1969,25 +2074,72 @@ export function compileRegexMatcher(
fn.push(` br label %m.ret`);
fn.push(`m.ret:`);
fn.push(` ret i32 1`);
}
fn.push(`nomatch:`);
fn.push(` %n.stk = load ptr, ptr %stk`);
fn.push(` %n.heap = icmp ne ptr %n.stk, %sbuf`);
fn.push(` br i1 %n.heap, label %n.free, label %n.ret`);
fn.push(`n.free:`);
fn.push(` call void @scr_regex_stack_free(ptr %n.stk)`);
fn.push(` br label %n.ret`);
fn.push(`n.ret:`);
fn.push(` ret i32 0`);
gen.emitTail();
for (const l of gen.body) fn.push(l);
fn.push(`}`);
const decls = REGEX_MATCHER_DECLS.slice();
if (gen.body.some((l) => l.includes("@llvm.cttz.i16")))
decls.push(`declare i16 @llvm.cttz.i16(i16, i1 immarg)`);
if (!sticky && !first.nullable && firstCount === 1)
decls.push(`declare ptr @memchr(ptr, i32, ${sizeType})`);
return { lines: [...gen.globals, ...fn], decls };
/** The match block of a replacer: append the gap since the previous match
* and the replacement to the output (ScrReReplaceOut: len, cap, next at
* 0, 8 and 16, then data), and continue the search at the match end, or one past it for an
* empty match, as the runtime's replace loop does. */
function emitReplaceMatch(fn: string[], sym: string, replacement: string, vector: boolean): void {
const n = replacement.length;
fn.push(`match:`);
fn.push(` %m.end = load i32, ptr %pos`);
fn.push(` %m.start = load i32, ptr %at`);
fn.push(` %m.next = load i32, ptr %next`);
fn.push(` %m.gap = sub i32 %m.start, %m.next`);
fn.push(` %m.gapw = zext i32 %m.gap to i64`);
fn.push(` %m.lenp = getelementptr inbounds i8, ptr %out, i64 0`);
fn.push(` %m.capp = getelementptr inbounds i8, ptr %out, i64 8`);
fn.push(` %m.datap = getelementptr inbounds i8, ptr %out, i64 24`);
fn.push(` %m.len0 = load i64, ptr %m.lenp`);
fn.push(` %m.cap = load i64, ptr %m.capp`);
fn.push(` %m.room = sub i64 %m.cap, %m.len0`);
// 16 bytes of slack past the gap and the replacement for the 16-byte copy.
fn.push(` %m.need = add i64 %m.gapw, ${n + 32}`);
fn.push(` %m.fits = icmp uge i64 %m.room, %m.need`);
fn.push(` br i1 %m.fits, label %m.put, label %m.grow`);
fn.push(`m.grow:`);
fn.push(` call void @scr_re_replace_out_reserve(ptr %out, i64 %m.need)`);
fn.push(` br label %m.put`);
fn.push(`m.put:`);
fn.push(` %m.data = load ptr, ptr %m.datap`);
fn.push(` %m.len = load i64, ptr %m.lenp`);
fn.push(` %m.dst = getelementptr inbounds i8, ptr %m.data, i64 %m.len`);
fn.push(` %m.nextw = sext i32 %m.next to i64`);
fn.push(` %m.src = getelementptr inbounds i8, ptr %s, i64 %m.nextw`);
if (vector) {
// A gap of at most 16 bytes with 16 readable subject bytes is one
// unaligned 16-byte copy.
fn.push(` %m.small = icmp ule i32 %m.gap, 16`);
fn.push(` %m.rest = sub i32 %len, %m.next`);
fn.push(` %m.readable = icmp sge i32 %m.rest, 16`);
fn.push(` %m.fast = and i1 %m.small, %m.readable`);
fn.push(` br i1 %m.fast, label %m.copy16, label %m.copy`);
fn.push(`m.copy16:`);
fn.push(` %m.v = load <16 x i8>, ptr %m.src, align 1`);
fn.push(` store <16 x i8> %m.v, ptr %m.dst, align 1`);
fn.push(` br label %m.rep`);
} else {
fn.push(` br label %m.copy`);
}
fn.push(`m.copy:`);
fn.push(` call void @llvm.memcpy.p0.p0.i64(ptr %m.dst, ptr %m.src, i64 %m.gapw, i1 false)`);
fn.push(` br label %m.rep`);
fn.push(`m.rep:`);
fn.push(` %m.rdst = getelementptr inbounds i8, ptr %m.dst, i64 %m.gapw`);
if (n > 0)
fn.push(` call void @llvm.memcpy.p0.p0.i64(ptr %m.rdst, ptr @${sym}.rep, i64 ${n}, i1 false)`);
fn.push(` %m.len1 = add i64 %m.len, %m.gapw`);
fn.push(` %m.len2 = add i64 %m.len1, ${n}`);
fn.push(` store i64 %m.len2, ptr %m.lenp`);
fn.push(` store i32 %m.end, ptr %next`);
fn.push(` %m.found0 = load i32, ptr %nfound`);
fn.push(` %m.found1 = add i32 %m.found0, 1`);
fn.push(` store i32 %m.found1, ptr %nfound`);
fn.push(` %m.empty = icmp eq i32 %m.end, %m.start`);
fn.push(` %m.after = add i32 %m.end, 1`);
fn.push(` %m.at = select i1 %m.empty, i32 %m.after, i32 %m.end`);
fn.push(` store i32 %m.at, ptr %at`);
fn.push(` %m.past = icmp sgt i32 %m.at, %len`);
fn.push(` br i1 %m.past, label %nomatch, label %scan`);
}
@@ -146,11 +146,19 @@ describe("LLVM runtime RC symbols", () => {
// The dyn fast path buffers surviving values by kind, as scr_dyn_release does.
const ir = emitInlineRcHelpers(host).join("\n");
expect(ir).toContain("call void @scr_dyn_release_survivor(ptr %o)");
expect(ir).toMatch(
/switch i32 %kind, label %done \[ i32 4, label %container i32 5, label %container i32 8, label %root/,
// Every kind that can be a candidate takes one path; scalars stay inline.
expect(ir).toContain(
"switch i32 %kind, label %done [ i32 4, label %candidate i32 5, label %candidate i32 8, label %candidate i32 9, label %candidate i32 12, label %candidate i32 13, label %candidate ]",
);
// Arrays and objects not yet buffered ask the runtime, which skips intact documents.
// Values not yet buffered ask the runtime, which decides per value.
expect(ir).toMatch(/br i1 %cqueued, label %done, label %survivor/);
// The runtime buffers dyn candidates, so the dyn helper has no inline append.
const dynHelper = ir.slice(
ir.indexOf("@sc_rc_release_dyn("),
ir.indexOf("\n}", ir.indexOf("@sc_rc_release_dyn(")),
);
expect(dynHelper).not.toContain("sc_cyc_append");
expect(dynHelper).not.toContain("scr_cyc_on_release");
});
test("sanitized builds call the fast-path helpers out of line", () => {
@@ -208,36 +216,42 @@ describe("LLVM runtime RC symbols", () => {
});
test("optimized builds buffer candidates through the module's append helper", () => {
const emit = (inlineRc: true | "release", extra: Partial<ShapeHost>) => {
const emit = (inlineRc: true | "release", extra: Partial<ShapeHost>, kind: "union" | "dyn") => {
const { host } = declarationHost(64, inlineRc);
const configured: ShapeHost = { ...host, inlineAlloc: true, ...extra };
releaseSym(configured, { kind: "dyn" });
releaseSym(configured, kind === "union" ? ({ kind, unionId: "u0" } as never) : { kind });
return {
ir: emitInlineRcHelpers(configured).join("\n"),
decls: inlineRcDecls(configured),
};
};
const { ir, decls } = emit(true, {}, "union");
expect(ir).toContain("call preserve_mostcc void @sc_cyc_append(ptr %o)");
expect(ir).toContain("define internal preserve_mostcc void @sc_cyc_append(ptr %o) noinline");
// The young test reads the generation byte at obj-5; the append fills
// scr_cyc_buffers' nursery buffer and the header's position at obj-4.
expect(ir).toContain("getelementptr i8, ptr %o, i64 -5");
expect(ir).toContain("getelementptr inbounds i8, ptr @scr_cyc_buffers, i64 96");
expect(ir).toContain("getelementptr inbounds i8, ptr @scr_cyc_buffers, i64 104");
expect(ir).toContain("load i64, ptr @scr_cyc_live");
expect(ir).toContain("or i32 %i, -2147483648");
expect(ir).toContain("call void @scr_cyc_on_release(ptr %o)");
expect(decls).toContain(
"@scr_cyc_buffers = external global { [3 x { ptr, i64, i64, i64 }], i64, i64 }",
);
expect(decls).toContain("@scr_cyc_live = external global i64");
// The dyn release buffers through scr_dyn_release_survivor, so a module
// whose only headered release is dyn's has no append helper.
for (const mode of [true, "release"] as const) {
const { ir, decls } = emit(mode, {});
expect(ir).toContain("call preserve_mostcc void @sc_cyc_append(ptr %o)");
expect(ir).toContain("define internal preserve_mostcc void @sc_cyc_append(ptr %o) noinline");
// The young test reads the generation byte at obj-5; the append fills
// scr_cyc_buffers' nursery buffer and the header's position at obj-4.
expect(ir).toContain("getelementptr i8, ptr %o, i64 -5");
expect(ir).toContain("getelementptr inbounds i8, ptr @scr_cyc_buffers, i64 96");
expect(ir).toContain("getelementptr inbounds i8, ptr @scr_cyc_buffers, i64 104");
expect(ir).toContain("load i64, ptr @scr_cyc_live");
expect(ir).toContain("or i32 %i, -2147483648");
expect(ir).toContain("call void @scr_cyc_on_release(ptr %o)");
expect(decls).toContain(
"@scr_cyc_buffers = external global { [3 x { ptr, i64, i64, i64 }], i64, i64 }",
);
expect(decls).toContain("@scr_cyc_live = external global i64");
const dyn = emit(mode, {}, "dyn");
expect(dyn.ir).not.toContain("sc_cyc_append");
expect(dyn.ir).not.toContain("scr_cyc_on_release");
expect(dyn.decls.join("\n")).not.toContain("scr_cyc_buffers");
}
// Worker executables and targets without inline allocation call
// scr_cyc_on_release, which tries the same append first.
for (const extra of [{ threadLocalAlloc: true }, { inlineAlloc: false }]) {
const { ir, decls } = emit(true, extra);
const { ir, decls } = emit(true, extra, "union");
expect(ir).not.toContain("sc_cyc_append");
expect(ir).toContain("call void @scr_cyc_on_release(ptr %o)");
expect(decls.join("\n")).not.toContain("scr_cyc_buffers");
+23 -16
View File
@@ -589,9 +589,10 @@ interface InlineRcFamily {
* repeats its own checks, so the fast path never duplicates it.
*
* The dyn release buffers a surviving value by kind (scr_dyn_release):
* the fast path decides the always and never cases and calls
* scr_dyn_release_survivor for kinds whose candidacy is per value,
* including arrays and objects of intact documents (ScrDynRegion).
* the fast path skips scalars and values already in a candidate buffer or
* never candidates, and calls scr_dyn_release_survivor for the rest, which
* decides per value (intact documents, ScrDynRegion; capsules and handles
* without edges).
*
* The array release inlines the decrement of an array that stays alive
* and is not headered (`a->cyc`): those are most array releases in
@@ -698,9 +699,10 @@ function requestRcHelper(host: ShapeHost, family: string, op: RcOp): string {
for (const decl of rcHelperDecls(family, op)) host.declare(decl);
// Its body is written last (emitInlineRcHelpers), after the requests are
// collected: a release that can buffer a candidate requests the append
// helper now. The jsval row (JS_RC_ROW) has no collector header; its
// cells release through dyn's helper, which makes its own request.
if (op === "release" && row.cycle === "always") requestCycleAppend(host);
// helper now. The dyn release buffers through scr_dyn_release_survivor
// instead. The jsval row (JS_RC_ROW) has no collector header; its cells
// release through dyn's helper.
if (op === "release" && row.cycle === "always" && family !== "dyn") requestCycleAppend(host);
return `@${inlineRcName(family, op)}`;
}
@@ -820,8 +822,11 @@ function rcHelperDecls(family: string, op: RcOp): string[] {
if (op === "retain" || row.release === null) return [];
return [
`declare void @${row.release}(ptr)`,
...(row.cycle === "always" ? [`declare void @scr_cyc_on_release(ptr)`] : []),
...(family === "dyn" ? [`declare void @scr_dyn_release_survivor(ptr)`] : []),
...(family === "dyn"
? [`declare void @scr_dyn_release_survivor(ptr)`]
: row.cycle === "always"
? [`declare void @scr_cyc_on_release(ptr)`]
: []),
];
}
@@ -931,21 +936,21 @@ function releaseHelper(host: ShapeHost, family: string, row: InlineRcFamily): st
// scr_dyn_release buffers a surviving value by kind: always for
// functions and proxies, per value for arrays and objects (not
// while their document is intact, ScrDynRegion), capsules and
// handles (scr_dyn_release_survivor), never for scalars. An
// array or object already in a candidate buffer stays inline.
// handles, never for scalars. Every kind that can be buffered
// takes one path, which keeps each inlined site small: a value
// already in a candidate buffer (or never one) stays inline, and
// scr_dyn_release_survivor decides the rest.
` %kp = getelementptr i8, ptr %o, ${S} ${S === "i64" ? 8 : 4} ; ScrDyn.kind`,
` %kind = load i32, ptr %kp`,
// SCR_DYN_ARR 4, OBJ 5, FUNC 8, PROXY 13; HANDLE 9, TYPED_REF 12.
` switch i32 %kind, label %done [ i32 4, label %container i32 5, label %container i32 8, label %root i32 13, label %root i32 9, label %survivor i32 12, label %survivor ]`,
`container:`,
// SCR_DYN_ARR 4, OBJ 5, FUNC 8, HANDLE 9, TYPED_REF 12, PROXY 13.
` switch i32 %kind, label %done [ i32 4, label %candidate i32 5, label %candidate i32 8, label %candidate i32 9, label %candidate i32 12, label %candidate i32 13, label %candidate ]`,
`candidate:`,
` %cbufp = getelementptr i8, ptr %o, ${S} -${host.cycleBufOffset}`,
` %cbuf = load i32, ptr %cbufp`,
` %cqueued = icmp slt i32 %cbuf, 0`,
` br i1 %cqueued, label %done, label %survivor`,
`survivor:`,
` call void @scr_dyn_release_survivor(ptr %o)`,
` br label %done`,
`root:`,
]
: family === "box"
? [
@@ -964,7 +969,9 @@ function releaseHelper(host: ShapeHost, family: string, row: InlineRcFamily): st
`root:`,
]
: []),
...(cycle === "always" ? cycleRootLines(host, "done") : [` br label %done`]),
...(cycle === "always" && family !== "dyn"
? cycleRootLines(host, "done")
: [` br label %done`]),
`done:`,
` ret void`,
`}`,
@@ -341,7 +341,7 @@ const methodCases: {
{ method: "normalize", args: [ref("argument")], result: STRING, target: "normalize" },
{ method: "isWellFormed", args: [], result: BOOL, target: "is_well_formed" },
{ method: "toWellFormed", args: [], result: STRING, target: "to_well_formed" },
{ method: "cpAt", args: [num(1)], result: STRING, target: "cp_at" },
{ method: "cpAtByte", args: [num(1)], result: STRING, target: "cp_at_byte", inline: true },
];
test.each(methodCases)(
@@ -30,7 +30,8 @@ export function borrowsStringInputs(method: IrStrIntrinsicMethod): boolean {
case "normalize":
case "isWellFormed":
case "toWellFormed":
case "cpAt":
case "byteLength":
case "cpAtByte":
return true;
default:
return false;
@@ -43,6 +43,10 @@ export const STRING_CHAR_CODE_AT_INT_MARKER = "; uses @sc_str_char_code_at_int";
export const STRING_CHAR_AT_DECL = "declare ptr @scr_str_char_at(ptr, double)";
/** Requests the integer-index charAt helper, as the marker above does. */
export const STRING_CHAR_AT_INT_MARKER = "; uses @sc_str_char_at_int";
/** The code-point character starting at a byte offset (the string
* iterator's step). Requested together with STRING_CHAR_AT_DECL, which
* defines the ASCII table. */
export const STRING_CP_AT_BYTE_DECL = "declare ptr @scr_str_cp_at_byte(ptr, i64)";
/** The proven-ASCII flag of string `s`, as an i1. */
export function emitStringAsciiFlag(host: LlvmEmitterContext, s: string): string {
@@ -351,6 +355,29 @@ export const STRING_READ_HELPERS: readonly StringReadHelper[] = [
];
},
},
{
// The string iterator's step at byte `%b`, a character start below the
// byte count: an ASCII byte is the table entry for it; any other lead
// byte takes the runtime, which copies the character's bytes.
decl: STRING_CP_AT_BYTE_DECL,
define: (sz, attrs) => [
`define internal ptr @sc_str_cp_at_byte(ptr %s, i64 %b) ${attrs} {`,
`entry:`,
` %data = getelementptr inbounds %ScrStr, ptr %s, i32 1`,
` %p = getelementptr inbounds i8, ptr %data, i64 %b`,
` %byte = load i8, ptr %p`,
` %ascii = icmp sge i8 %byte, 0`,
` br i1 %ascii, label %read, label %slow`,
`read:`,
` %code = zext i8 %byte to i64`,
` %char = getelementptr inbounds [128 x { ${sz}, ${sz}, ${sz}, [2 x i8] }], ptr @sc_ascii_chars, i64 0, i64 %code`,
` ret ptr %char`,
`slow:`,
` %r = call ptr @scr_str_cp_at_byte(ptr %s, i64 %b) nounwind willreturn`,
` ret ptr %r`,
`}`,
],
},
{
// An exact integer index, as for sc_str_char_code_at_int. Requested
// together with STRING_CHAR_AT_DECL, which defines the table.
+98 -16
View File
@@ -26,6 +26,7 @@ import {
llFieldType,
loadStructField,
releaseSym,
retainSym,
traceAdapter,
type ShapeHost,
} from "./shapes.js";
@@ -41,6 +42,8 @@ import { undefinedArmTag } from "../../ir/analysis.js";
* scr_error-style label texts). */
export interface WalkerHost extends ShapeHost {
readonly unionsById: Map<string, IrUnionDef>;
/** Arrays of this element type can hold SCR_ARR_FOREIGN elements. */
mayHoldForeign(elem: IrType): boolean;
/** `@`-ref of an interned immortal ScrStr literal. */
internLiteral(text: string): string;
/** `@`-ref of an interned NUL-terminated byte-array constant. */
@@ -325,9 +328,7 @@ export class LlWalkers {
if (i > 0) this.putc(B, "%b", "44"); // ','
if (edgeable(f.type)) this.jbEdgeIdx(B, String(i));
const v = this.loadField(B, "%v", shapeId, fieldIndex.get(f.name)!, f.type);
B.line(
`call void @${this.jsonWriteHelper(f.type)}(ptr %b, ${this.valTy(f.type)} ${v}) ; [${llvmCommentText(f.name)}]`,
);
this.writeField(B, v, f.type, `[${llvmCommentText(f.name)}]`);
});
this.putc(B, "%b", "93"); // ']'
if (cyclic) this.jbLeave(B);
@@ -359,9 +360,7 @@ export class LlWalkers {
this.puts(B, "%b", `${i > 0 ? "," : ""}${jsonObjectKeyLabel(f.name)}`);
if (edgeable(f.type)) this.jbEdgeProp(B, f.name);
const v = this.loadField(B, "%v", shapeId, fieldIndex.get(f.name)!, f.type);
B.line(
`call void @${this.jsonWriteHelper(f.type)}(ptr %b, ${this.valTy(f.type)} ${v}) ; ${llvmCommentText(f.name)}`,
);
this.writeField(B, v, f.type, `${llvmCommentText(f.name)}`);
});
} else {
const first = B.slot();
@@ -396,9 +395,7 @@ export class LlWalkers {
comma();
this.puts(B, "%b", jsonObjectKeyLabel(f.name));
if (edgeable(f.type)) this.jbEdgeProp(B, f.name);
B.line(
`call void @${this.jsonWriteHelper(f.type)}(ptr %b, ${this.valTy(f.type)} ${v}) ; ${llvmCommentText(f.name)}`,
);
this.writeField(B, v, f.type, `${llvmCommentText(f.name)}`);
if (skip !== null) {
B.br(skip);
B.startBlock(skip);
@@ -520,9 +517,7 @@ export class LlWalkers {
B.startBlock(lKey);
this.puts(B, "%b", jsonObjectKeyLabel(fieldName));
if (cyclic && traceAdapter(this.host, f.type) !== null) this.jbEdgeProp(B, fieldName);
B.line(
`call void @${this.jsonWriteHelper(f.type)}(ptr %b, ${this.valTy(f.type)} ${v}) ; ${llvmCommentText(fieldName)}`,
);
this.writeField(B, v, f.type, `${llvmCommentText(fieldName)}`);
B.terminate("ret i1 true");
});
B.startBlock(lSkip);
@@ -539,14 +534,15 @@ export class LlWalkers {
/** Whether the walker for `t` can leave an exception pending: cyclic
* values, records serialized through a kept checked source, and
* checked-dynamic values (toJSON methods, getters, BigInts). */
* checked-dynamic values (toJSON methods, getters, BigInts), including
* the foreign elements of arrays that can hold them. */
jsonWriteMayThrow(t: IrType, seen = new Set<string>()): boolean {
if (traceAdapter(this.host, t) !== null) return true;
switch (t.kind) {
case "dyn":
return true;
case "array":
return this.jsonWriteMayThrow(t.elem, seen);
return this.host.mayHoldForeign(t.elem) || this.jsonWriteMayThrow(t.elem, seen);
case "union":
return (this.host.unionsById.get(t.unionId)?.arms ?? []).some((arm) =>
this.jsonWriteMayThrow(arm, seen),
@@ -678,6 +674,47 @@ export class LlWalkers {
B.line(`call void @scr_arr_release(ptr ${ks})`);
}
/** Serializing a value of `t` can call user code that changes the
* arrays or records holding it: it reaches an array that can hold
* foreign elements (their toJSON and getters), or a value whose own
* serialization may (a class instance, a checked-dynamic value), in a
* program that has foreign elements at all. */
private serializationMayCallUserCode(t: IrType, seen = new Set<string>()): boolean {
const host = this.host;
if (!host.mayHoldForeign({ kind: "f64" }) && !host.mayHoldForeign({ kind: "string" }))
return false;
const key = typeKey(t);
if (seen.has(key)) return false;
seen.add(key);
switch (t.kind) {
case "dyn":
case "object":
return true;
case "array":
return host.mayHoldForeign(t.elem) || this.serializationMayCallUserCode(t.elem, seen);
case "record":
return (host.recordsById.get(t.shapeId)?.fields ?? []).some((field) =>
this.serializationMayCallUserCode(field.type, seen),
);
case "union":
return (host.unionsById.get(t.unionId)?.arms ?? []).some((arm) =>
this.serializationMayCallUserCode(arm, seen),
);
default:
return false;
}
}
/** Write a field value loaded borrowed from its record. When its
* serialization can call user code that replaces the field, the value
* is held across the write. */
private writeField(B: BlockBuilder, v: string, t: IrType, comment: string): void {
const held = isRefCounted(t) && this.serializationMayCallUserCode(t);
if (held) B.line(`call ptr ${retainSym(this.host, t)}(ptr ${v})`);
B.line(`call void @${this.jsonWriteHelper(t)}(ptr %b, ${this.valTy(t)} ${v}) ; ${comment}`);
if (held) B.line(`call void ${releaseSym(this.host, t)}(ptr ${v})`);
}
private emitArrayWriter(B: BlockBuilder, elem: IrType, cyclic: boolean): void {
const host = this.host;
const w = this.jsonWriteHelper(elem);
@@ -687,7 +724,12 @@ export class LlWalkers {
if (cyclic) this.jbEnter(B, true);
this.putc(B, "%b", "91"); // '['
const finish = B.newLabel("jwa.end");
if (elem.kind !== "bool") this.emitDenseArrayElements(B, elem, w, cyclic, finish);
// The dense walk borrows the storage and each element. When an
// element's serialization can call user code, which can grow or
// shrink the array or replace the element, the general walk below
// reads each index afresh and holds its element instead.
if (elem.kind !== "bool" && !this.serializationMayCallUserCode(elem))
this.emitDenseArrayElements(B, elem, w, cyclic, finish);
const len = B.tmp();
B.line(`${len} = call double @scr_arr_len(ptr %v)`);
B.countedLoop(len, (i) => {
@@ -710,6 +752,25 @@ export class LlWalkers {
B.line(`${present} = fcmp oeq double ${state}, ${f64Lit(1)}`);
B.condBr(present, valueBlock, missingBlock);
B.startBlock(missingBlock);
if (host.mayHoldForeign(elem)) {
// A foreign element (SCR_ARR_FOREIGN) serializes its own value,
// after its toJSON runs with the index as its key.
const foreign = B.tmp();
const foreignBlock = B.newLabel("jwa.foreign");
const nullBlock = B.newLabel("jwa.null");
B.line(`${foreign} = fcmp oeq double ${state}, ${f64Lit(3)}`);
B.condBr(foreign, foreignBlock, nullBlock);
B.startBlock(foreignBlock);
host.declare(`declare ptr @scr_arr_get_foreign(ptr, double)`);
host.declare(`declare void @scr_dyn_release(ptr)`);
const payload = B.tmp();
B.line(`${payload} = call ptr @scr_arr_get_foreign(ptr %v, double ${i}) ; +1`);
host.declare(`declare void @scr_jb_put_dyn_element(ptr, ptr, double)`);
B.line(`call void @scr_jb_put_dyn_element(ptr %b, ptr ${payload}, double ${i})`);
B.line(`call void @scr_dyn_release(ptr ${payload})`);
B.br(doneBlock);
B.startBlock(nullBlock);
}
host.declare(`declare void @scr_jb_puts(ptr, ptr)`);
B.line(`call void @scr_jb_puts(ptr %b, ptr ${host.cstr("null")})`);
B.br(doneBlock);
@@ -911,7 +972,28 @@ export class LlWalkers {
const lvalue = B.newLabel("uj.p");
B.line(`${state} = call double @scr_arr_state(ptr %a, double ${i})`);
B.line(`${isValue} = fcmp oeq double ${state}, ${f64Lit(1)}`);
B.condBr(isValue, lvalue, next);
if (host.mayHoldForeign({ kind: "union", unionId })) {
// A foreign element (SCR_ARR_FOREIGN) joins as its ToString, like
// every element but undefined and null.
const notValue = B.newLabel("uj.f");
const foreignLabel = B.newLabel("uj.foreign");
B.condBr(isValue, lvalue, notValue);
B.startBlock(notValue);
const foreign = B.tmp();
B.line(`${foreign} = fcmp oeq double ${state}, ${f64Lit(3)}`);
B.condBr(foreign, foreignLabel, next);
B.startBlock(foreignLabel);
host.declare(`declare ptr @scr_arr_get_foreign(ptr, double)`);
host.declare(`declare void @scr_dyn_release(ptr)`);
const payload = B.tmp();
B.line(`${payload} = call ptr @scr_arr_get_foreign(ptr %a, double ${i}) ; +1`);
host.declare(`declare void @scr_jb_join_dyn(ptr, ptr)`);
B.line(`call void @scr_jb_join_dyn(ptr ${buf}, ptr ${payload})`);
B.line(`call void @scr_dyn_release(ptr ${payload})`);
B.br(next);
} else {
B.condBr(isValue, lvalue, next);
}
B.startBlock(lvalue);
const u = B.tmp();
B.line(`${u} = call ptr @scr_arr_peek_ref(ptr %a, double ${i}) ; borrowed scalar union`);
+2 -2
View File
@@ -2,8 +2,8 @@
* of the link contract: a program object keeps this symbol undefined and the
* matching runtime defines it, so incompatible manual links fail before the
* program can start. */
export const RUNTIME_ABI_VERSION = 44 as const;
export const RUNTIME_ABI_MARKER = "scr_runtime_abi_v44" as const;
export const RUNTIME_ABI_VERSION = 45 as const;
export const RUNTIME_ABI_MARKER = "scr_runtime_abi_v45" as const;
/** Object consumption is useful today, but the complete scr_* surface may
* still change before 1.0. The versioned marker prevents accidental mixing;
@@ -52,7 +52,7 @@ async function fixture(target: NativeTargetSpec, workers = false) {
object_format: target.objectFormat,
minimum_os: target.minimumOs,
},
runtime_abi: { version: 44, marker: "scr_runtime_abi_v44" },
runtime_abi: { version: 45, marker: "scr_runtime_abi_v45" },
compiler: { command: "fixture", identity: "fixture", target: target.llvmTriple },
macros: { executable: [], excluded: ["SCR_LIB"], sanitizer: "external-toolchain-required" },
flavors: {
@@ -129,7 +129,7 @@ async function fixture(target: NativeTargetSpec = MACOS_ARM64_TARGET) {
object_format: target.objectFormat,
minimum_os: target.minimumOs,
},
runtime_abi: { version: 44, marker: "scr_runtime_abi_v44" },
runtime_abi: { version: 45, marker: "scr_runtime_abi_v45" },
compiler: {
command: "clang",
identity: "fixture clang",
@@ -58,7 +58,33 @@ export function arrayNumberRead(arr: IrExpr, index: IrExpr, loc: SrcLoc): IrExpr
};
}
/** Read one stabilized receiver/index pair. Both a hole and present undefined yield undefined. */
/** A foreign element read into a typed result: it never fits, so the read
* throws the checked conversion's TypeError (array.foreignThrow). */
export function foreignElementThrow(
lowerer: Lowerer,
arr: IrExpr,
index: IrExpr,
type: IrType,
loc: SrcLoc,
): IrExpr {
return {
kind: "libCall",
fn: "array.foreignThrow",
args: [arr, index, { kind: "strLit", value: lowerer.fmt(type), type: STRING, loc }],
type,
loc,
};
}
/** Arrays of this element type can hold elements outside it, stored
* through dynamic views (Lowerer.foreignElementKeys). */
export function elementMayBeForeign(lowerer: Lowerer, elem: IrType): boolean {
const keys = lowerer.foreignElementKeys;
return keys.size > 0 && elem.kind !== "dyn" && (keys.has("*") || keys.has(typeKey(elem)));
}
/** Read one stabilized receiver/index pair. Both a hole and present undefined yield undefined;
* a foreign element (state 3) takes the checked conversion into the typed result. */
export function arrayValueRead(
lowerer: Lowerer,
arr: IrExpr,
@@ -68,8 +94,25 @@ export function arrayValueRead(
): IrExpr {
const type = arrayValueType(lowerer, elem);
const read: IrExpr = { kind: "arrayGet", arr, index, type: elem, loc };
const missing: IrExpr =
let missing: IrExpr =
type.kind === "dyn" ? dynUndefinedExpr(loc) : lowerer.wrappedUndefined(type, loc)!;
if (elementMayBeForeign(lowerer, elem)) {
missing = {
kind: "ternary",
cond: {
kind: "bin",
op: "===",
left: { kind: "arrayState", arr, index, type: F64, loc },
right: { kind: "numLit", value: 3, type: F64, loc },
type: BOOL,
loc,
},
then: foreignElementThrow(lowerer, arr, index, type, loc),
else_: missing,
type,
loc,
};
}
return {
kind: "ternary",
cond: {
@@ -684,6 +727,9 @@ export function tryLowerNumericIndexRead(
operand.callee !== lowerer.arrHofHelpers.get(`idxOr:${typeKey(F64)}`)
)
return null;
// ToNumber of a foreign element differs from its own value under `+`,
// comparisons and ToBoolean.
if (elementMayBeForeign(lowerer, F64)) return null;
const [arr, index] = operand.args;
if (
operand.args.length !== 2 ||
@@ -777,6 +823,8 @@ export function tryFuseIndexRead(
if (arr?.type.kind !== "array" || index?.type.kind !== "f64") return null;
const elem = arr.type.elem;
if (elem.kind === "union" || elem.kind === "void") return null;
// A foreign element is neither the present value nor the missing one.
if (elementMayBeForeign(lowerer, elem)) return null;
if (operand.callee !== lowerer.arrHofHelpers.get(`idxOr:${typeKey(elem)}`)) return null;
if (arr.kind !== "varRef" || (index.kind !== "varRef" && index.kind !== "numLit")) return null;
const loc = operand.loc;
@@ -42,6 +42,8 @@ import {
arrayValueStore,
arrayValueType,
currentArrayIndexPresent,
elementMayBeForeign,
foreignElementThrow,
primitiveElement,
} from "../array-values.js";
import { typeKey } from "../../type-mapper.js";
@@ -2531,6 +2533,9 @@ export function lowerArraySortCall(
loc,
native,
keyed && { order, key: reads !== null, keys: keyed.length },
elementMayBeForeign(lowerer, elem)
? (arr, index) => foreignElementThrow(lowerer, arr, index, elem, loc)
: null,
),
);
}
@@ -1,5 +1,6 @@
import { keyedHelperName } from "../stable-names.js";
import { identityPreservingWidening } from "../coercions/identity.js";
import { elementMayBeForeign, foreignElementThrow } from "../array-values.js";
import { countedFor, numLit, varRef } from "../../../ir/build.js";
import {
BOOL,
@@ -131,7 +132,33 @@ export function arrayConcatHelper(
loc,
},
then: [{ kind: "arraySetUndefined", arr: out, index: destination, loc }],
else_: null,
else_: elementMayBeForeign(lowerer, sourceElem)
? [
{
// A foreign element takes the checked conversion's TypeError.
kind: "if",
cond: {
kind: "bin",
op: "===",
left: { kind: "arrayState", arr: source, index, type: F64, loc },
right: numLit(3, loc),
type: BOOL,
loc,
},
then: [
{
kind: "arraySet",
arr: out,
index: destination,
value: foreignElementThrow(lowerer, source, index, elem, loc),
loc,
},
],
else_: null,
loc,
},
]
: null,
loc,
},
],
@@ -1081,9 +1081,7 @@ export function lowerArrayFromCall(
if ((args.length === 1 || args.length === 2) && !ts.isObjectLiteralExpression(args[0]!)) {
const source = lowerer.lowerExpr(args[0]!);
let src =
source.type.kind === "string" && args.length === 2
? strCharsCall(lowerer, source, loc)
: source;
source.type.kind === "string" && args.length === 2 ? strCharsCall(source, loc) : source;
if (src.type.kind === "array") {
const arrT = src.type;
if (args.length === 1) {
@@ -1171,7 +1169,7 @@ export function lowerArrayFromCall(
}
// String iteration advances by code point; the same helper serves
// `[...s]` and both Array.from(s) forms.
if (src.type.kind === "string" && args.length === 1) return strCharsCall(lowerer, src, loc);
if (src.type.kind === "string" && args.length === 1) return strCharsCall(src, loc);
if (src.type.kind === "dyn" && args.length === 1) {
return lowerCheckedArrayFrom(lowerer, src, loc);
}
@@ -1318,112 +1316,9 @@ function buildArrayFromArrayFn(
}
/** `Array.from(s)` / `[...s]` on a STRING: the code-point split into a
* fresh string[], through one interned helper per module. */
export function strCharsCall(lowerer: Lowerer, src: IrExpr, loc: SrcLoc): IrExpr {
const key = "strChars";
let helper = lowerer.arrHofHelpers.get(key);
if (!helper) {
helper = "%str.chars";
lowerer.arrHofHelpers.set(key, helper);
lowerer.liftedFns.push(buildStrCharsFn(helper, loc));
}
return { kind: "call", callee: helper, args: [src], type: arrayOf(STRING), loc };
}
/** The code-point split, from existing IR nodes — the string for-of
* desugar's UTF-16 cursor as a function:
*
* out = [];
* i = 0;
* while (i < s.length) { ch = cpAt(s, i); i += ch.length; out.push(ch); }
* return out;
*/
function buildStrCharsFn(name: string, loc: SrcLoc): IrFunction {
const outT = arrayOf(STRING);
const sLen = (recv: IrExpr): IrExpr => ({
kind: "strIntrinsic",
method: "length",
receiver: recv,
args: [],
type: F64,
loc,
});
const body: IrStmt[] = [
{
kind: "varDecl",
localId: "out.0",
init: { kind: "arrayLit", elems: [], type: outT, loc },
loc,
},
{ kind: "varDecl", localId: "i.0", init: { kind: "numLit", value: 0, type: F64, loc }, loc },
{
kind: "while",
cond: {
kind: "bin",
op: "<",
left: varRef("i.0", F64, loc),
right: sLen(varRef("s.0", STRING, loc)),
type: BOOL,
loc,
},
body: [
{
kind: "varDecl",
localId: "ch.0",
init: {
kind: "strIntrinsic",
method: "cpAt",
receiver: varRef("s.0", STRING, loc),
args: [varRef("i.0", F64, loc)],
type: STRING,
loc,
},
loc,
},
{
kind: "assign",
localId: "i.0",
value: {
kind: "bin",
op: "+",
left: varRef("i.0", F64, loc),
right: sLen(varRef("ch.0", STRING, loc)),
type: F64,
loc,
},
loc,
},
{
kind: "exprStmt",
expr: {
kind: "arrIntrinsic",
method: "push",
receiver: varRef("out.0", outT, loc),
args: [varRef("ch.0", STRING, loc)],
type: F64,
loc,
},
loc,
},
],
loc,
},
{ kind: "return", value: varRef("out.0", outT, loc), loc },
];
return {
name,
params: [{ localId: "s.0", name: "s", type: STRING }],
returnType: outT,
locals: [
{ id: "s.0", name: "s", type: STRING, mutable: true },
{ id: "out.0", name: "out", type: outT, mutable: false },
{ id: "i.0", name: "i", type: F64, mutable: true },
{ id: "ch.0", name: "ch", type: STRING, mutable: false },
],
body,
loc,
};
* fresh string[] (scr_str_code_points). */
export function strCharsCall(src: IrExpr, loc: SrcLoc): IrExpr {
return { kind: "libCall", fn: "str.codePoints", args: [src], type: arrayOf(STRING), loc };
}
/** The lowered `length` value of the one-property source literal, or null
@@ -1886,17 +1886,27 @@ function arraySearchHelper(
},
loc,
};
const visitPresent: IrStmt[] = [
{ kind: "varDecl", localId: "value.0", init: value, loc },
{
kind: "if",
cond: match,
then: [{ kind: "return", value: method === "includes" ? boolLit(true, loc) : indexRef, loc }],
else_: null,
loc,
},
advance,
];
const found: IrStmt = {
kind: "if",
cond: directNeedle ? equal : match,
then: [{ kind: "return", value: method === "includes" ? boolLit(true, loc) : indexRef, loc }],
else_: null,
loc,
};
// A direct needle reads only value slots: an undefined or foreign
// element written through a dynamic view has no element-typed payload.
const visitPresent: IrStmt[] = directNeedle
? [
{
kind: "if",
cond: { kind: "bin", op: "===", left: stateRef, right: numLit(1, loc), type: BOOL, loc },
then: [{ kind: "varDecl", localId: "value.0", init: value, loc }, found],
else_: null,
loc,
},
advance,
]
: [{ kind: "varDecl", localId: "value.0", init: value, loc }, found, advance];
const skipHole: IrStmt[] =
method === "lastIndexOf"
? [advance]
@@ -34,6 +34,7 @@ import {
arrayValueRead,
arrayValueStore,
arrayValueType,
elementMayBeForeign,
} from "../array-values.js";
import type { Lowerer } from "../lowerer.js";
@@ -156,7 +157,8 @@ const PURE_STRING = new Set<string>([
"toUpperCase",
"isWellFormed",
"toWellFormed",
"cpAt",
"byteLength",
"cpAtByte",
]);
function primitive(t: IrType): boolean {
@@ -1643,7 +1645,9 @@ class FusedLoop {
];
}
const valueT = arrayValueType(this.lowerer, arrT.elem);
if (stage.fast) {
// A foreign element is neither the plain element nor undefined: the
// ordinary read below converts it.
if (stage.fast && !elementMayBeForeign(this.lowerer, arrT.elem)) {
// The split visit: a value goes on as the plain element, a present
// undefined as the undefined arm.
const missing = this.lowerer.wrappedUndefined(valueT, loc);
@@ -1135,7 +1135,7 @@ function setFromSeedValue(
if (source.type.kind === "set" && typeEquals(source.type, setT)) {
return { kind: "setIntrinsic", method: "clone", receiver: source, args: [], type: setT, loc };
} else if (source.type.kind === "string" && setT.elem.kind === "string") {
seed = strCharsCall(lowerer, source, loc);
seed = strCharsCall(source, loc);
} else if (source.type.kind === "array" && typeEquals(source.type.elem, setT.elem)) {
seed = source;
} else if (
@@ -504,7 +504,7 @@ export function lowerArrayLiteral(
// interned helper as Array.from(s); the result rides the array
// machinery below like any string[] source.
if (src.type.kind === "string") {
src = strCharsCall(lowerer, src, locOf(el));
src = strCharsCall(src, locOf(el));
}
// `[...typedArray]`: represented typed arrays are dense numeric
// iterables, so drain their elements into the fresh number[] that
@@ -59,6 +59,7 @@ import {
arrayValueRead,
arrayValueStore,
arrayValueType,
elementMayBeForeign,
lowerSafeBytesRead,
lowerSafeIndexRead,
primitiveElement,
@@ -1623,11 +1624,14 @@ export function lowerElementCompoundParts(
const indexRef = (): IrExpr => varRef(indexLocal.id, F64, loc);
const oldValue: IrExpr =
receiver.type.kind === "array"
? receiver.type.elem.kind === "f64"
? receiver.type.elem.kind === "f64" &&
!(op === "+" && elementMayBeForeign(lowerer, receiver.type.elem))
? // Every numeric operator applies ToNumber to the old value, so a
// missing element is NaN without materializing `number | undefined`.
arrayNumberRead(receiverRef(), indexRef(), locOf(target))
: receiver.type.elem.kind === "string" && op === "+"
: receiver.type.elem.kind === "string" &&
op === "+" &&
!elementMayBeForeign(lowerer, receiver.type.elem)
? arrayStringOperand(receiverRef(), indexRef(), locOf(target))
: arrayValueRead(lowerer, receiverRef(), indexRef(), receiver.type.elem, locOf(target))
: {
@@ -14,7 +14,12 @@ import {
import { varRef } from "../../../ir/build.js";
import { locOf } from "../../program.js";
import type { Lowerer } from "../lowerer.js";
import { arrayNumberRead, arrayValueRead, arrayValueStore } from "../array-values.js";
import {
arrayNumberRead,
arrayValueRead,
arrayValueStore,
elementMayBeForeign,
} from "../array-values.js";
import { fenceNodeModuleMutation } from "../lower-node-module.js";
import { globalSymbolKey } from "./global-symbols.js";
import { tryLowerExpression } from "./try-lower-expression.js";
@@ -185,7 +190,12 @@ export function lowerShortCircuitAssignment(lowerer: Lowerer, expr: ts.BinaryExp
const element = storage;
write = (value) => arrayValueStore(lowerer, receiver, key, value, element, loc);
const elementLoc = locOf(target);
if (nullish && NON_NULLISH_ELEMENTS.has(element.kind)) {
// A foreign element can be any value, nullish or not.
if (
nullish &&
NON_NULLISH_ELEMENTS.has(element.kind) &&
!elementMayBeForeign(lowerer, element)
) {
// Only a missing element is nullish: keep the present value or
// store the right side, without the optional element value.
rhs = lowerer.lowerExprExpecting(expr.right, element);
@@ -577,10 +577,11 @@ export function lowerBuiltinDelegation(
loc,
});
} else if (sourceT.kind === "string") {
// A byte cursor over the WTF-8 data, as for-of over a string walks.
const ch = lowerer.declareHiddenLocal("%delegateElement", STRING);
const length = (receiver: IrExpr): IrExpr => ({
kind: "strIntrinsic",
method: "length",
method: "byteLength",
receiver,
args: [],
type: F64,
@@ -595,7 +596,7 @@ export function lowerBuiltinDelegation(
localId: ch.id,
init: {
kind: "strIntrinsic",
method: "cpAt",
method: "cpAtByte",
receiver: src(),
args: [at()],
type: STRING,
@@ -743,7 +743,7 @@ export function lowerStringIteratorValue(
left: index,
right: {
kind: "strIntrinsic",
method: "length",
method: "byteLength",
receiver: text,
args: [],
type: F64,
@@ -758,7 +758,7 @@ export function lowerStringIteratorValue(
localId: "point.0",
init: {
kind: "strIntrinsic",
method: "cpAt",
method: "cpAtByte",
receiver: text,
args: [index],
type: STRING,
@@ -775,7 +775,7 @@ export function lowerStringIteratorValue(
left: index,
right: {
kind: "strIntrinsic",
method: "length",
method: "byteLength",
receiver: point,
args: [],
type: F64,
@@ -926,6 +926,10 @@ export function buildArraySortFn(
loc: SrcLoc,
native = false,
keyed: { order: number; key: boolean; keys: number } | null = null,
/** The throw for a foreign element (SCR_ARR_FOREIGN) at `index` of
* `arr`, which the comparator's element type cannot hold; for element
* types whose arrays can hold one. */
checkForeign: ((arr: IrExpr, index: IrExpr) => IrExpr) | null = null,
): IrFunction {
const b = new SortIr(loc);
const arrT = arrayOf(elem),
@@ -1035,7 +1039,21 @@ export function buildArraySortFn(
b.branch(
b.eq(state, b.num(1)),
[compactValue()],
[b.branch(b.eq(state, b.num(2)), [b.step(undefinedCount)])],
[
b.branch(
b.eq(state, b.num(2)),
[b.step(undefinedCount)],
checkForeign
? [
// A foreign element does not fit the comparator's element
// type: the checked conversion's TypeError.
b.branch(b.eq(state, b.num(3)), [
{ kind: "exprStmt", expr: checkForeign(src, index), loc },
]),
]
: null,
),
],
),
]),
// Sort only the compacted prefix. The tail is overwritten with undefined
@@ -1,5 +1,6 @@
import { keyedHelperName } from "./stable-names.js";
import { perKeyOrder } from "./key-order-witness.js";
import { elementMayBeForeign } from "./array-values.js";
import { classPrototypeView } from "./class-prototypes.js";
import {
dynUndefinedExpr,
@@ -1421,11 +1422,29 @@ function inspectHelper(lowerer: Lowerer, t: IrType, loc: SrcLoc): string {
const i = (): IrExpr => varRef("i.0", F64, loc);
const next = (): IrExpr => varRef("next.0", F64, loc);
const hasMore = (): IrExpr => lt(i(), n());
// A foreign element (state 3) holds a value outside the element type.
const foreignElements = elementMayBeForeign(lowerer, t.elem);
const foreign = (): IrExpr => ({
kind: "libCall",
fn: "array.foreign",
args: [v(), i()],
type: DYN,
loc,
});
const isNumber = (): IrExpr => ({
kind: "ternary",
cond: eq(state(i()), numLit(1, loc)),
then: isNumberFlag(lowerer, t.elem, () => at(i()), loc),
else_: boolLit(false, loc),
else_: foreignElements
? {
kind: "ternary",
cond: eq(state(i()), numLit(3, loc)),
then: isNumberFlag(lowerer, DYN, foreign, loc),
else_: boolLit(false, loc),
type: BOOL,
loc,
}
: boolLit(false, loc),
type: BOOL,
loc,
});
@@ -1504,7 +1523,16 @@ function inspectHelper(lowerer: Lowerer, t: IrType, loc: SrcLoc): string {
kind: "ternary",
cond: eq(state(i()), numLit(1, loc)),
then: child(t.elem, at(i())),
else_: strLit("undefined", loc),
else_: foreignElements
? {
kind: "ternary",
cond: eq(state(i()), numLit(3, loc)),
then: child(DYN, foreign()),
else_: strLit("undefined", loc),
type: STRING,
loc,
}
: strLit("undefined", loc),
type: STRING,
loc,
},
@@ -2425,7 +2425,9 @@ function lowerNetSocketMethodCall(
};
}
if (name === "write" || name === "end") {
requireStatementPosition(lowerer, call, `socket.${name}(...)`);
// write() answers Node's backpressure boolean; end() stays a statement
if (name === "end") requireStatementPosition(lowerer, call, `socket.${name}(...)`);
const resultT = name === "write" ? BOOL : VOID;
// write(chunk, encoding) — the two-argument encoding form: 'buffer'
// beside a string chunk is Writable's encoding check (a synchronous
// ERR_UNKNOWN_ENCODING TypeError, "Unknown encoding: buffer" — the
@@ -2451,7 +2453,7 @@ function lowerNetSocketMethodCall(
const socket = coerceToHandle(lowerer, access.expression, NETSOCKET_T);
const chunk = lowerer.lowerExpr(args[0]!);
const fn: IrLibFn = chunk.type.kind === "string" ? "net.sockWrite" : "net.sockWriteBytes";
return { kind: "libCall", fn, args: [socket, chunk], type: VOID, loc };
return { kind: "libCall", fn, args: [socket, chunk], type: BOOL, loc };
}
}
}
@@ -2510,7 +2512,7 @@ function lowerNetSocketMethodCall(
NARROW_DATA_HINT,
);
}
main = { kind: "libCall", fn, args: [receiver(), data], type: VOID, loc };
main = { kind: "libCall", fn, args: [receiver(), data], type: resultT, loc };
}
const cb = lowerer.lowerExpr(last);
let register: IrExpr;
@@ -2531,8 +2533,24 @@ function lowerNetSocketMethodCall(
loc,
};
}
stmts.push({ kind: "exprStmt", expr: main, loc });
return { kind: "seqExpr", stmts, result: register, type: VOID, loc };
if (resultT.kind === "void") {
stmts.push({ kind: "exprStmt", expr: main, loc });
return { kind: "seqExpr", stmts, result: register, type: VOID, loc };
}
// write(data, cb): the data is written, the callback registered, and
// the write's answer is the value
const okLocal = lowerer.declareHiddenLocal("%sockWriteOk", BOOL);
stmts.push(
{ kind: "varDecl", localId: okLocal.id, init: main, loc },
{ kind: "exprStmt", expr: register, loc },
);
return {
kind: "seqExpr",
stmts,
result: { kind: "varRef", localId: okLocal.id, type: BOOL, loc },
type: BOOL,
loc,
};
}
const maxArgs = name === "write" ? 1 : 1;
const minArgs = name === "write" ? 1 : 0;
@@ -2552,17 +2570,17 @@ function lowerNetSocketMethodCall(
const data = lowerer.lowerExpr(args[0]!);
if (data.type.kind === "string") {
const fn: IrLibFn = name === "write" ? "net.sockWrite" : "net.sockEndStr";
return { kind: "libCall", fn, args: [receiver, data], type: VOID, loc };
return { kind: "libCall", fn, args: [receiver, data], type: resultT, loc };
}
if (data.type.kind === "bytes" && data.type.elem === "u8") {
const fn: IrLibFn = name === "write" ? "net.sockWriteBytes" : "net.sockEndBytes";
return { kind: "libCall", fn, args: [receiver, data], type: VOID, loc };
return { kind: "libCall", fn, args: [receiver, data], type: resultT, loc };
}
if (data.type.kind === "dyn") {
// An untyped JS payload (a helper's parameter): the runtime
// dispatches STR/BYTES and throws Node's chunk TypeError otherwise.
const fn: IrLibFn = name === "write" ? "net.sockWriteDyn" : "net.sockEndDyn";
return { kind: "libCall", fn, args: [receiver, data], type: VOID, loc };
return { kind: "libCall", fn, args: [receiver, data], type: resultT, loc };
}
lowerer.noLowering(
`${name} of '${lowerer.fmt(data.type)}' data`,
@@ -2944,6 +2962,7 @@ function lowerNetSocketMethodCall(
event === "connect" ||
event === "ready" ||
event === "timeout" ||
event === "drain" ||
event === "readable"
) {
const { cb } = lowerCallbackArg(
@@ -2962,11 +2981,13 @@ function lowerNetSocketMethodCall(
? "net.sockOnClose"
: event === "timeout"
? "net.sockOnTimeout"
: event === "readable"
? "net.sockOnReadable"
: event === "ready"
? "net.sockOnReady"
: "net.sockOnConnect";
: event === "drain"
? "net.sockOnDrain"
: event === "readable"
? "net.sockOnReadable"
: event === "ready"
? "net.sockOnReady"
: "net.sockOnConnect";
return { kind: "libCall", fn, args: [receiver, cb, once], type: VOID, loc };
}
if (event === "lookup") {
@@ -3957,6 +3978,18 @@ export function lowerServerProperty(
const receiver = coerceToHandle(lowerer, expr.expression, NETSOCKET_T);
return { kind: "libCall", fn: "net.sockBytesWritten", args: [receiver], type: F64, loc };
}
if (
recvKind === "netSocket" &&
lowerer.isStdlibMember(expr) &&
(expr.name.text === "writableLength" || expr.name.text === "writableNeedDrain")
) {
// The write buffer's unsent bytes, and whether a write answered false
// and 'drain' has not fired yet.
const receiver = coerceToHandle(lowerer, expr.expression, NETSOCKET_T);
return expr.name.text === "writableLength"
? { kind: "libCall", fn: "net.sockWritableLength", args: [receiver], type: F64, loc }
: { kind: "libCall", fn: "net.sockWritableNeedDrain", args: [receiver], type: BOOL, loc };
}
if (recvKind === "netSocket" && lowerer.isStdlibMember(expr) && expr.name.text === "readable") {
// true until the read half is done (peer FIN / destroy).
const receiver = coerceToHandle(lowerer, expr.expression, NETSOCKET_T);
@@ -2668,7 +2668,7 @@ export function lowerBindingPattern(
lowerer.unsupported("SC1031", el, "rest elements over a union of array or tuple layouts");
const loc = locOf(el);
let value = unionArrayValueRead(lowerer, srcRef(), numLit(i, loc), loc, (text) =>
strCharsCall(lowerer, text, loc),
strCharsCall(text, loc),
);
if (!value)
lowerer.unsupported(
@@ -2798,7 +2798,7 @@ export function lowerBindingPattern(
}
// STRING sources (`const [a, b] = str`): array destructuring walks
// the STRING ITERATOR — code points, astral characters whole —
// which is Array.from(string)'s own machinery (%str.chars), so the
// which is Array.from(string)'s own machinery (str.codePoints), so the
// source splits once into a hidden chars array and the pattern
// lowers as an array pattern over string[]. Positions, holes,
// defaults (with the bounds test), rest (the remaining code points
@@ -2817,7 +2817,7 @@ export function lowerBindingPattern(
out.push({
kind: "varDecl",
localId: chars.id,
init: strCharsCall(lowerer, srcRef(), loc),
init: strCharsCall(srcRef(), loc),
loc,
});
lowerer.lowerBindingPattern(
@@ -9826,7 +9826,7 @@ function lowerArrayAssignInto(
let srcType = tmpRef().type;
// A STRING source (`[a, b] = s`): the declaration path's rule in
// assignment position — the string iterator's code-point split
// (%str.chars, astral characters whole) into a hidden chars array,
// (str.codePoints, astral characters whole) into a hidden chars array,
// then the pattern proceeds as an array pattern over string[]
// (defaults with the bounds test, rest packing the remaining code
// points, elisions; positions past the last code point inherit the
@@ -9839,7 +9839,7 @@ function lowerArrayAssignInto(
out.push({
kind: "varDecl",
localId: chars.id,
init: strCharsCall(lowerer, tmpRef(), loc),
init: strCharsCall(tmpRef(), loc),
loc,
});
tmpRef = () => ({ kind: "varRef", localId: chars.id, type: charsT, loc });
@@ -11383,16 +11383,18 @@ function lowerForOfMatchAll(
/** `for (const ch of s)` over a STRING: JS's string iterator, which walks
* code POINTS — each pass binds the whole character (a two-unit string
* for astral chars, where charAt/slice would return one surrogate half).
* Desugars over the UTF-16 cursor machinery: a hidden unit
* index reads the character with the cpAt intrinsic and advances by ITS
* length before the body runs (so `continue` never skips the advance):
* Desugars over a byte cursor into the WTF-8 data: a hidden byte offset
* reads the character that starts there with the cpAtByte intrinsic and
* advances by ITS byte count before the body runs (so `continue` never
* skips the advance):
*
* { const %s = <expr>; let %i = 0;
* while (%i < %s.length) { const ch = cpAt(%s, %i); %i += ch.length; <body> } }
* while (%i < byteLength(%s)) { const ch = cpAtByte(%s, %i); %i += byteLength(ch); <body> } }
*
* Strings are immutable, so the length is stable and there is no
* Strings are immutable, so the byte count is stable and there is no
* iterator state to unwind on throw; a lone surrogate is a one-unit
* character, so every step lands on a character start. */
* character with its own three-byte encoding, so every step lands on a
* character start. */
/** for-of over a CLASS ITERABLE (classIteratorOf's shape) — the
* iterator protocol as ordinary method calls:
*
@@ -12087,7 +12089,7 @@ function lowerForOfString(
try {
const s = lowerer.declareHiddenLocal("%strof", STRING);
const i = lowerer.declareHiddenLocal("%iterof", F64);
i.mutable = true; // the unit index reassigns (hidden locals default const)
i.mutable = true; // the byte cursor reassigns (hidden locals default const)
const sRef = (): IrExpr => ({ kind: "varRef", localId: s.id, type: STRING, loc });
const iRef = (): IrExpr => ({ kind: "varRef", localId: i.id, type: F64, loc });
// `for (var c of str)`: the character mechanics stay on a hidden
@@ -12118,7 +12120,7 @@ function lowerForOfString(
localId: ch.id,
init: {
kind: "strIntrinsic",
method: "cpAt",
method: "cpAtByte",
receiver: sRef(),
args: [iRef()],
type: STRING,
@@ -12145,7 +12147,7 @@ function lowerForOfString(
left: iRef(),
right: {
kind: "strIntrinsic",
method: "length",
method: "byteLength",
receiver: chRef,
args: [],
type: F64,
@@ -12177,7 +12179,7 @@ function lowerForOfString(
left: iRef(),
right: {
kind: "strIntrinsic",
method: "length",
method: "byteLength",
receiver: sRef(),
args: [],
type: F64,
@@ -2399,6 +2399,11 @@ export class Lowerer {
/** for-of loops over arrays that can hold neither holes nor undefined:
* each pass reads the element payload directly. */
readonly presentElementLoops = new Set<ts.ForOfStatement>();
/** Element keys (typeKey of the element type) of native arrays that can
* hold elements outside their element type, stored through dynamic views
* (SCR_ARR_FOREIGN); "*" for every array. Empty unless the program stores
* values through a dynamic value that can view a native array. */
foreignElementKeys: ReadonlySet<string> = new Set();
/** The fs/promises glob() call lowerForOf is lowering as a for await
* head; other glob() calls have no lowering. */
fspGlobHead: ts.Node | null = null;
@@ -6472,6 +6477,20 @@ export class Lowerer {
let dynamicViews: Set<string> | null = null;
// Element ABIs of the native arrays that an `any` or `unknown` value is
// converted to, including arrays nested in the converted value.
// Assignments, including the logical ones (`view ??= numbers`).
const assignmentOperators = new Set<ts.SyntaxKind>([
ts.SyntaxKind.EqualsToken,
ts.SyntaxKind.QuestionQuestionEqualsToken,
ts.SyntaxKind.BarBarEqualsToken,
ts.SyntaxKind.AmpersandAmpersandEqualsToken,
]);
// Initializers of class properties, parameter defaults and
// destructuring defaults, which convert to the binding's type.
const initializedBinding = (
node: ts.Node,
): node is ts.PropertyDeclaration | ts.ParameterDeclaration | ts.BindingElement =>
(ts.isPropertyDeclaration(node) || ts.isParameter(node) || ts.isBindingElement(node)) &&
node.initializer !== undefined;
const collectDynamicViews = (): Set<string> => {
const keys = new Set<string>();
const dynamicValue = (node: ts.Expression): boolean => {
@@ -6522,14 +6541,18 @@ export class Lowerer {
dynamicValue(node.initializer)
)
target(typeOrNull(node.name));
else if (initializedBinding(node) && dynamicValue(node.initializer!))
target(typeOrNull(node.name ?? node));
else if (
ts.isBinaryExpression(node) &&
node.operatorToken.kind === ts.SyntaxKind.EqualsToken &&
assignmentOperators.has(node.operatorToken.kind) &&
dynamicValue(node.right)
)
target(typeOrNull(node.left));
else if (ts.isReturnStatement(node) && node.expression && dynamicValue(node.expression))
target(contextualOrNull(node.expression));
else if (ts.isYieldExpression(node) && node.expression && dynamicValue(node.expression))
target(contextualOrNull(node.expression));
else if (ts.isCallExpression(node) || ts.isNewExpression(node)) {
for (const arg of node.arguments?.nodes ?? [])
if (!ts.isSpreadElement(arg) && dynamicValue(arg)) target(contextualOrNull(arg));
@@ -6558,6 +6581,237 @@ export class Lowerer {
});
return keys;
};
// Element ABIs of the native arrays a value converted to a dynamic type
// can reach: the conversion keeps each such array as a live view, so
// stores through the dynamic value write the native storage.
// Standard library functions that only read the arguments they take as
// `any` or `unknown`: the console methods, String, Number, Boolean,
// Array.isArray, Object.is, JSON.stringify without a replacer function,
// and util's inspect and format. Any other function can keep an
// argument (Promise.resolve, Map.prototype.set, push onto an
// `unknown[]`) or pass it to a callback (a JSON.stringify replacer, an
// emitted event), which can write through it.
const readerGlobals = new Map<string, ReadonlySet<string> | null>([
["console", null],
["JSON", new Set(["stringify"])],
["Array", new Set(["isArray"])],
["Object", new Set(["is"])],
]);
const readerFunctions = new Set(["String", "Number", "Boolean"]);
const utilReaders = new Set(["inspect", "format"]);
const standardLibraryReader = (node: ts.CallExpression | ts.NewExpression): boolean =>
panicSafe(() => {
if (!ts.isCallExpression(node)) return false;
const callee = peel(node.expression);
if (ts.isIdentifier(callee))
return [...readerFunctions].some((name) => this.isStdlibGlobal(callee, name));
if (!ts.isPropertyAccessExpression(callee)) return false;
const member = callee.name.text;
for (const [global, members] of readerGlobals) {
if (!this.isStdlibGlobal(callee.expression, global)) continue;
if (members !== null && !members.has(member)) return false;
if (global !== "JSON") return true;
const replacer = node.arguments.nodes[1];
return (
replacer === undefined ||
ts.isArrayLiteralExpression(peel(replacer)) ||
peel(replacer).kind === ts.SyntaxKind.NullKeyword
);
}
if (!utilReaders.has(member)) return false;
const declaration = this.checker.getResolvedSignature(node)?.declaration;
if (declaration === undefined) return false;
const file = ts.getSourceFileOfNode(declaration);
return this.isStdlibFile(file) && /[\\/]util\.d\.ts$/.test(file.fileName);
}, false);
const collectNativeViews = (): Set<string> => {
const keys = new Set<string>();
const seen = new Set<string>();
const collect = (type: IrType | null): void => {
if (!type) return;
const id = typeKey(type);
if (seen.has(id)) return;
seen.add(id);
if (type.kind === "array") {
if (type.elem.kind !== "dyn") keys.add(typeKey(type.elem));
collect(type.elem);
} else if (type.kind === "union") {
for (const arm of this.unions.get(type.unionId)?.arms ?? []) collect(arm);
} else if (type.kind === "record") {
for (const field of this.shapes.get(type.shapeId)?.fields ?? []) collect(field.type);
} else if (type.kind === "object") {
for (const field of this.classes.get(type.className)?.fields.values() ?? [])
collect(field);
} else if (type.kind === "map") {
collect(type.key);
collect(type.value);
} else if (type.kind === "set") collect(type.elem);
else if (type.kind === "promise") collect(type.inner);
};
const holdsDynamic = (type: IrType | null, depth: number): boolean => {
if (!type || depth > 4) return false;
if (type.kind === "dyn") return true;
if (type.kind === "array" || type.kind === "set") return holdsDynamic(type.elem, depth + 1);
if (type.kind === "map") return holdsDynamic(type.value, depth + 1);
if (type.kind === "promise") return holdsDynamic(type.inner, depth + 1);
if (type.kind === "union")
return (this.unions.get(type.unionId)?.arms ?? []).some((arm) =>
holdsDynamic(arm, depth + 1),
);
if (type.kind === "record")
return (this.shapes.get(type.shapeId)?.fields ?? []).some((field) =>
holdsDynamic(field.type, depth + 1),
);
return false;
};
const mapped = (type: ts.Type | null | undefined): IrType | null =>
type ? panicSafe(() => this.mapTypeOf(type), null) : null;
const flow = (source: ts.Expression, target: ts.Type | null | undefined): void => {
if (target === undefined) return;
const into = target === null ? null : mapped(target);
// An unanswerable target may be dynamic.
if (target !== null && into !== null && !holdsDynamic(into, 0)) return;
const from = typeOrNull(source);
if (from === null) keys.add("*");
else collect(mapped(from));
};
const visit = (node: ts.Node): void => {
if (ts.isAsExpression(node) || ts.isTypeAssertion(node))
flow(node.expression, typeOrNull(node));
else if (ts.isVariableDeclaration(node) && node.type && node.initializer)
flow(node.initializer, typeOrNull(node.name));
else if (initializedBinding(node)) flow(node.initializer!, typeOrNull(node.name ?? node));
else if (ts.isBinaryExpression(node) && assignmentOperators.has(node.operatorToken.kind))
flow(node.right, typeOrNull(node.left));
else if (ts.isReturnStatement(node) && node.expression)
flow(node.expression, contextualOrNull(node.expression));
else if (ts.isYieldExpression(node) && node.expression)
flow(node.expression, contextualOrNull(node.expression));
// A thrown value reaches a catch binding, which is dynamic.
else if (ts.isThrowStatement(node)) flow(node.expression, null);
else if (ts.isCallExpression(node) || ts.isNewExpression(node)) {
if (!standardLibraryReader(node))
for (const arg of node.arguments?.nodes ?? [])
flow(ts.isSpreadElement(arg) ? arg.expression : arg, contextualOrNull(arg));
} else if (ts.isPropertyAssignment(node))
flow(node.initializer, contextualOrNull(node.initializer));
else if (ts.isShorthandPropertyAssignment(node)) {
const name = node.name as ts.Expression;
flow(name, contextualOrNull(name));
} else if (ts.isArrayLiteralExpression(node)) {
for (const element of node.elements.nodes)
if (!ts.isSpreadElement(element) && !ts.isOmittedExpression(element))
flow(element, contextualOrNull(element));
} else if (ts.isArrowFunction(node) && !ts.isBlock(node.body))
flow(node.body as ts.Expression, contextualOrNull(node.body as ts.Expression));
};
for (const sf of sourceFiles)
ts.walkPreorder(sf, (node) => {
// A checker panic hides this conversion: any array.
if (!panicSafe(() => (visit(node), true), false)) keys.add("*");
return undefined;
});
return keys;
};
// A receiver typed `any`, `unknown` or an array of either can be a live
// view of a native array. A receiver of another type (`Worker[]`,
// `Record<string, unknown>`) is one only after a lying assertion, even
// when its values are dynamic.
const dynamicType = (type: ts.Type): boolean => {
if ((type.flags & (ts.TypeFlags.Any | ts.TypeFlags.Unknown)) !== 0) return true;
if (!this.checker.isArrayType(type) || this.checker.isTupleType(type)) return false;
const element = this.checker.getTypeArguments(type as ts.TypeReference)[0];
return (
element !== undefined && (element.flags & (ts.TypeFlags.Any | ts.TypeFlags.Unknown)) !== 0
);
};
// A binding that only ever holds the array its declaration creates (a
// const variable, or a readonly property with an initializer) is never
// a view of a native array, even when its elements are dynamic.
const freshDynamicStorage = (node: ts.Expression, depth: number): boolean => {
const e = peel(node);
if (ts.isArrayLiteralExpression(e)) return true;
if (ts.isNewExpression(e) || ts.isCallExpression(e)) {
const callee = peel(e.expression);
if (ts.isIdentifier(callee)) return this.isStdlibGlobal(callee, "Array");
return (
ts.isPropertyAccessExpression(callee) &&
(callee.name.text === "of" || callee.name.text === "from") &&
this.isStdlibGlobal(callee.expression, "Array")
);
}
if (depth > 4 || (!ts.isIdentifier(e) && !ts.isPropertyAccessExpression(e))) return false;
// A typed array along the way is the array a view shows.
const type = typeOrNull(e);
if (type === null || !dynamicType(type)) return false;
const symbol = panicSafe(
() => this.checker.getSymbolAtLocation(ts.isIdentifier(e) ? e : e.name),
undefined,
);
const declaration = symbol?.valueDeclaration;
if (!symbol || !declaration) return false;
// Variables and class properties only: parameters take any argument,
// and object literals fill interface properties.
if (
!(ts.isVariableDeclaration(declaration) && ts.isIdentifier(declaration.name)) &&
!ts.isPropertyDeclaration(declaration)
)
return false;
const assigned = assignedValues(symbol);
if (assigned === null) return false;
const values = [
...(declaration.initializer ? [declaration.initializer] : []),
...(assigned ?? []),
];
return values.length > 0 && values.every((value) => freshDynamicStorage(value, depth + 1));
};
// Every value `=` assigns to each variable or property of the program,
// by symbol; null for one also written another way (compound or
// destructuring assignment). Built on first use.
let assignments: Map<ts.Symbol, ts.Expression[] | null> | null = null;
const assignedValues = (symbol: ts.Symbol): ts.Expression[] | null | undefined => {
if (assignments === null) {
const found = new Map<ts.Symbol, ts.Expression[] | null>();
const add = (target: ts.Expression, value: ts.Expression | null): void => {
const e = peel(target);
if (ts.isArrayLiteralExpression(e) || ts.isObjectLiteralExpression(e)) {
// Destructuring: every target is written with a part of the value.
ts.forEachChild(e, (child) => {
if (ts.isIdentifier(child) || ts.isPropertyAccessExpression(child)) add(child, null);
else if (ts.isSpreadElement(child)) add(child.expression, null);
else if (ts.isPropertyAssignment(child)) add(child.initializer, null);
else if (ts.isShorthandPropertyAssignment(child))
add(child.name as ts.Expression, null);
else if (ts.isBinaryExpression(child)) add(child.left, null);
return undefined;
});
return;
}
const name = ts.isIdentifier(e) ? e : ts.isPropertyAccessExpression(e) ? e.name : null;
const written = name
? panicSafe(() => this.checker.getSymbolAtLocation(name), undefined)
: undefined;
if (!written) return;
const list = found.get(written);
if (list === null) return;
if (value === null) found.set(written, null);
else if (list) list.push(value);
else found.set(written, [value]);
};
for (const sf of sourceFiles)
ts.walkPreorder(sf, (node) => {
if (ts.isBinaryExpression(node)) {
const op = node.operatorToken.kind;
if (op === ts.SyntaxKind.EqualsToken) add(node.left, node.right);
else if (op >= ts.SyntaxKind.FirstAssignment && op <= ts.SyntaxKind.LastAssignment)
add(node.left, null);
}
return undefined;
});
assignments = found;
}
return assignments.get(symbol);
};
// `t[i]!` on a typed array, stored into an array, keeps an invalid
// index's undefined (assertedReadObservesUndefined): `!` performs no
// runtime check. A read whose consumers treat undefined like NaN stores
@@ -6575,6 +6829,11 @@ export class Lowerer {
);
};
const elementStates = new ArrayElementStates({
isDynamicArray: (node) => {
const type = typeOrNull(node);
return (type === null || dynamicType(type)) && !freshDynamicStorage(node, 0);
},
nativeViewKeys: collectNativeViews,
elementAdmitsUndefined: (key) => {
siteDependencies?.read(key);
return optionalElementKeys.has(key);
@@ -7495,6 +7754,7 @@ export class Lowerer {
}
for (const [site, present] of presentSites) if (present) this.presentElementCalls.add(site);
for (const [loop, present] of presentLoops) if (present) this.presentElementLoops.add(loop);
this.foreignElementKeys = elementStates.foreignKeys();
const globalsById = new Map(this.globalsList.map((global) => [global.id, global]));
for (const [symbol, field] of staticFieldsBySymbol) {
if (!optionalSymbols.has(symbol)) continue;
@@ -8047,6 +8307,9 @@ export class Lowerer {
...(this.usesWorkers ? { workers: true } : {}),
...(this.usesLifetimeRegions ? { lifetimeRegions: true as const } : {}),
...(this.declBoundary ? { declBoundary: true as const } : {}),
...(this.foreignElementKeys.size > 0
? { foreignElementKeys: [...this.foreignElementKeys].sort() }
: {}),
...(this.threadSlotKeys.size > 0
? { threadSlotKeys: [...this.threadSlotKeys].sort() }
: {}),
@@ -43,8 +43,17 @@ export interface ElementStateHost {
ownerOf(node: ts.Expression): ts.Symbol | null;
/** The owning slot a newly built array is stored into. */
allocationOwner(node: ts.Expression): ts.Symbol | null;
/** `node` is a dynamic value (`any`, `unknown`, `unknown[]`): when it is
* a live view of a native array, a store through it writes that array's
* storage. */
isDynamicArray(node: ts.Expression): boolean;
/** Element keys of the native arrays a value converted to a dynamic type
* can reach, as live views ("*" for any array). */
nativeViewKeys(): ReadonlySet<string>;
/** Debug provenance for a new fact. */
note: ((fact: "undefined" | "holes", key: string, site: ts.Node | undefined) => void) | null;
note:
| ((fact: "undefined" | "holes" | "foreign", key: string, site: ts.Node | undefined) => void)
| null;
/** The scan read, or added to, the facts of an element key, of an owning
* slot, or of every array (ANY_ARRAY), for dependency tracking
* (SiteDependencies). */
@@ -54,6 +63,16 @@ export interface ElementStateHost {
/** The fact name of the facts shared by every array. */
const ANY_ARRAY = "*";
/** The fact name of holes stored through dynamic views. */
const VIEW_HOLES = "%view";
/** The fact name of values stored through dynamic views. */
const VIEW_VALUES = "%view-values";
/** Element keys whose arrays store a value written through a dynamic view
* that the element type cannot hold, as a foreign element: `number[]` and
* `string[]`. Arrays of other element types reject such a write with the
* checked conversion's TypeError. */
const FOREIGN_ELEMENT_KEYS: ReadonlySet<string> = new Set(["f64", "string"]);
/** What an array value can contain besides present values. */
interface Contents {
@@ -87,6 +106,13 @@ export class ArrayElementStates {
* whose element ABI is decided per generic instantiation. */
private anyUndefined = false;
private anyHoley = false;
/** A store through a dynamic value can leave holes; they land in every
* native array such a value views. */
private viewHoles = false;
/** A store through a dynamic value can write any value, which the
* viewed native arrays then hold as undefined or foreign elements. */
private viewValues = false;
private viewKeys: ReadonlySet<string> | null = null;
/** Facts of arrays owned by one storage slot. */
private readonly undefinedOwners = new Set<ts.Symbol>();
private readonly holeyOwners = new Set<ts.Symbol>();
@@ -336,14 +362,65 @@ export class ArrayElementStates {
this.host.read?.(ANY_ARRAY);
if (keys === null) return null;
for (const key of keys) this.host.read?.(key);
this.host.read?.(VIEW_HOLES);
this.host.read?.(VIEW_VALUES);
return {
undefined:
this.anyUndefined ||
keys.some((key) => this.undefinedKeys.has(key) || this.host.elementAdmitsUndefined(key)),
holes: this.anyHoley || keys.some((key) => this.holeyKeys.has(key)),
keys.some(
(key) =>
this.undefinedKeys.has(key) ||
this.host.elementAdmitsUndefined(key) ||
this.viewedValues(key),
),
holes: this.anyHoley || keys.some((key) => this.holeyKeys.has(key) || this.viewed(key)),
};
}
/** Holes stored through a dynamic view can be in arrays of this key. */
private viewed(key: string): boolean {
return this.viewHoles && this.isViewed(key);
}
/** Values of any type stored through a dynamic view can be in arrays of
* this key. */
private viewedValues(key: string): boolean {
return this.viewValues && FOREIGN_ELEMENT_KEYS.has(key) && this.isViewed(key);
}
private isViewed(key: string): boolean {
this.viewKeys ??= this.host.nativeViewKeys();
return this.viewKeys.has(key) || this.viewKeys.has(ANY_ARRAY);
}
/** Element keys whose arrays can hold elements outside their element
* type, stored through dynamic views (of FOREIGN_ELEMENT_KEYS). Final
* once the fixed point is reached. */
foreignKeys(): ReadonlySet<string> {
if (!this.viewValues) return new Set();
return new Set([...FOREIGN_ELEMENT_KEYS].filter((key) => this.isViewed(key)));
}
/** A store through `receiver`, which has no native element key, can
* write a value of any type into a viewed native array. */
private addViewValues(receiver: ts.Expression): boolean {
if (this.viewValues || !this.host.isDynamicArray(receiver)) return false;
this.viewValues = true;
this.host.changed?.(VIEW_VALUES);
this.host.note?.("foreign", VIEW_VALUES, this.site);
return true;
}
/** A store through `receiver`, which has no native element key, can
* leave holes. Only a dynamic value can be a view of a native array. */
private addViewHoles(receiver: ts.Expression): boolean {
if (this.viewHoles || !this.host.isDynamicArray(receiver)) return false;
this.viewHoles = true;
this.host.changed?.(VIEW_HOLES);
this.host.note?.("holes", VIEW_HOLES, this.site);
return true;
}
/** Reading every index of `node` (a spread, an iteration, `Array.from`)
* can produce `undefined`. Only native arrays can: other iterables reject
* an undefined value when it is stored. */
@@ -437,7 +514,11 @@ export class ArrayElementStates {
/** A store through `receiver` can leave holes. */
private addHolesTo(receiver: ts.Expression): boolean {
const owner = this.host.ownerOf(receiver);
if (owner === null) return this.addHoles(this.host.arrayKeys(receiver));
if (owner === null) {
const keys = this.host.arrayKeys(receiver);
if (keys !== null && keys.length === 0) return this.addViewHoles(receiver);
return this.addHoles(keys);
}
if (this.holeyOwners.has(owner)) return false;
this.holeyOwners.add(owner);
this.host.changed?.(owner);
@@ -497,10 +578,18 @@ export class ArrayElementStates {
op === ts.SyntaxKind.QuestionQuestionEqualsToken ||
op === ts.SyntaxKind.BarBarEqualsToken ||
op === ts.SyntaxKind.AmpersandAmpersandEqualsToken;
const keys = this.host.arrayKeys(target.expression);
if (keys !== null && keys.length === 0) {
// Through a dynamic value: any element write can store a value the
// viewed native array's element type does not hold.
if (namesProperty(target)) return false;
let changed = false;
if (holes && this.addViewHoles(target.expression)) changed = true;
if (this.addViewValues(target.expression)) changed = true;
return changed;
}
const undefinedValue = storesValue && this.host.mayBeUndefined(node.right);
if (!holes && !undefinedValue) return false;
const keys = this.host.arrayKeys(target.expression);
if (keys !== null && keys.length === 0) return false;
let changed = false;
if (holes && this.addHolesTo(target.expression)) changed = true;
if (undefinedValue && this.addUndefinedTo(target.expression)) changed = true;
@@ -511,8 +600,6 @@ export class ArrayElementStates {
target.name.text === "length" &&
!isShrinkingLength(node, target)
) {
const keys = this.host.arrayKeys(target.expression);
if (keys !== null && keys.length === 0) return false;
return this.addHolesTo(target.expression);
}
return false;
@@ -563,7 +650,14 @@ export class ArrayElementStates {
return changed;
}
const receiver = this.host.arrayKeys(callee.expression);
if (receiver !== null && receiver.length === 0) return changed;
if (receiver !== null && receiver.length === 0) {
if (
(stored.length > 0 || callee.name.text === "fill") &&
this.addViewValues(callee.expression)
)
changed = true;
return changed;
}
for (const value of stored)
if (this.valueMayBeUndefined(value) && this.addUndefinedTo(callee.expression)) changed = true;
return changed;
@@ -596,10 +690,17 @@ export class ArrayElementStates {
));
if (!reflective) return false;
let changed = false;
for (const arg of args) {
// The written object: `Reflect.apply`'s receiver argument, otherwise
// the first argument.
const written = this.host.isGlobal(owner, "Reflect") && method === "apply" ? 1 : 0;
for (const [index, arg] of args.entries()) {
const target = ts.isSpreadElement(arg) ? arg.expression : arg;
const keys = this.host.arrayKeys(target);
if (keys !== null && keys.length === 0) continue;
if (keys !== null && keys.length === 0) {
if (index === written && this.addViewHoles(target)) changed = true;
if (index === written && this.addViewValues(target)) changed = true;
continue;
}
if (this.addUndefinedTo(target)) changed = true;
if (this.addHolesTo(target)) changed = true;
}
@@ -783,6 +884,14 @@ function isAppendIndex(target: ts.ElementAccessExpression): boolean {
);
}
/** `o["name"]`: a string literal key that is not an array index writes a
* named property, never an element. */
function namesProperty(target: ts.ElementAccessExpression): boolean {
const key = peel(target.argumentExpression);
if (!ts.isStringLiteral(key) && !ts.isNoSubstitutionTemplateLiteral(key)) return false;
return !/^(0|[1-9][0-9]*)$/.test(key.text) || Number(key.text) >= 2 ** 32 - 1;
}
/** `xs.length = 0`, `xs.length = xs.length - k` and `xs.length -= k`
* truncate; truncation never leaves holes. */
function isShrinkingLength(
@@ -4948,6 +4948,19 @@ function mapTypeInner(type: ts.Type, ctx: TypeMapperCtx): IrType | null {
const record = mapRecordType(widened, ctx);
if (record) return record;
}
// An intersection whose constituents all map to ONE representation is
// that representation. An intersection that restates an optional member
// as required (`Opts & { files: Array<{ path: string }> }` over
// `files?: Array<{ path: string }>`) gives the member the type `A[] & B[]`
// with two separately written but structurally equal element literals;
// the checker keeps both, while their arrays (tuples, maps, ...) intern
// to one IR type. Reached only after every other intersection rule
// declined, so intersections that already mapped keep their mapping. A
// dynamic result stays unmapped: two opaque types are not one type.
if (widened.isIntersectionType()) {
const same = mapEqualConstituentsIntersection(widened, ctx);
if (same) return same;
}
// Unions. `boolean` is internally `true | false`, and narrowing produces
// literal unions like `"a" | "b"` — parts that map to the SAME IR type
// collapse (deduplicated by typeKey), so a single surviving arm is just
@@ -6813,6 +6826,19 @@ export function withUnitArm(
* assigned keys, which is also Node's answer for Object.assign(fn, obj)
* (a function's own length/name are non-enumerable). Null whenever the
* shape doesn't fit — the caller falls through to the plain-func mapping. */
/** The IR type every constituent of an intersection maps to, or null when a
* constituent is unmappable or dynamic, or two constituents differ. */
function mapEqualConstituentsIntersection(widened: ts.Type, ctx: TypeMapperCtx): IrType | null {
let same: IrType | null = null;
for (const part of ts.constituentTypes(widened)) {
const mapped = mapType(part, ctx);
if (mapped === null || mapped.kind === "dyn") return null;
if (same === null) same = mapped;
else if (!typeEquals(same, mapped)) return null;
}
return same;
}
function mapHybridCallableIntersection(widened: ts.Type, ctx: TypeMapperCtx): IrType | null {
const { checker, shapes } = ctx;
if (!widened.isIntersectionType()) return null;
@@ -64,6 +64,8 @@ interface MatchRecord extends PatchMatch {
interface State {
manifest: string;
/** The directory of the project's package.json. */
projectDir: string;
manifestText: string;
parse: TypePatchParse;
byPackage: Map<string, { entries: TypePatchEntry[]; requests: Request[] }>;
@@ -112,6 +114,7 @@ export function configureTypePatches(entryPath: string, parse: TypePatchParse):
}
state = {
manifest: config.manifest,
projectDir: dirname(config.manifest),
manifestText: config.manifestText ?? "",
parse,
byPackage,
@@ -151,6 +154,22 @@ function versionAt(dir: string): string | null {
}
}
/** The version of the runtime package `pkg` an `@types` twin under
* `nodeModules` describes: the copy installed beside it, else the one the
* project resolves (pnpm keeps each package in its own directory, so a
* twin has no runtime package beside it), else null. */
function twinRuntimeVersion(s: State, nodeModules: string, pkg: string): string | null {
const beside = versionAt(nodeModules + pkg);
if (beside !== null) return beside;
for (let dir = s.projectDir; ;) {
const version = versionAt(`${dir}/node_modules/${pkg}`);
if (version !== null) return version;
const parent = dirname(dir);
if (parent === dir) return null;
dir = parent;
}
}
/** The installed package copy a file belongs to, with the requests that
* apply to it; null for files outside node_modules or of unpatched
* packages. */
@@ -172,9 +191,9 @@ function copyOf(s: State, path: string, touch = true): { copy: CopyInfo; rel: st
const pkg = twin ? unmangleTypesName(name) : name;
const patches = s.byPackage.get(pkg);
if (patches !== undefined) {
// A twin describes the runtime package installed beside it.
// A twin is matched by the runtime package's version, never its own.
const version = twin
? (versionAt(path.slice(0, at + marker.length) + pkg) ?? versionAt(root))
? twinRuntimeVersion(s, path.slice(0, at + marker.length), pkg)
: versionAt(root);
const applicable = patches.entries.filter(
(e) => version !== null && satisfies(version, e.range),
@@ -596,6 +596,26 @@ function parseJsDoc(text: string, start: number, end: number): JsDocComment {
return { start, end, tags };
}
/** The spans of a file's string, template and regular expression literal
* tokens. */
function literalSpans(sf: ts.SourceFile): { start: number; end: number }[] {
const out: { start: number; end: number }[] = [];
const kinds = new Set([
ts.SyntaxKind.StringLiteral,
ts.SyntaxKind.NoSubstitutionTemplateLiteral,
ts.SyntaxKind.TemplateHead,
ts.SyntaxKind.TemplateMiddle,
ts.SyntaxKind.TemplateTail,
ts.SyntaxKind.RegularExpressionLiteral,
]);
const visit = (n: ts.Node): void => {
if (kinds.has(n.kind)) out.push({ start: tokenStartOfNode(n), end: n.end });
ts.forEachChild(n, visit);
};
visit(sf);
return out.sort((a, b) => a.start - b.start);
}
const RETURN_TAGS = new Set(["returns", "return"]);
const PROPERTY_TAGS = new Set(["property", "prop"]);
@@ -615,15 +635,27 @@ class JsResolver {
private readonly text: string,
sf: ts.SourceFile,
) {
// JSDoc comments, not text inside string, template or regular
// expression literals that looks like one.
const literals = literalSpans(sf);
const re = /\/\*\*(?!\/)[\s\S]*?\*\//g;
for (let m = re.exec(text); m !== null; m = re.exec(text))
for (let m = re.exec(text); m !== null; m = re.exec(text)) {
const inside = literals.find((l) => l.start <= m!.index && m!.index < l.end);
if (inside !== undefined) {
re.lastIndex = inside.end;
continue;
}
this.comments.push(parseJsDoc(text, m.index, m.index + m[0].length));
}
for (const c of this.comments)
c.tags.forEach((t, i) => {
if ((t.tag === "typedef" || t.tag === "callback") && t.name !== null)
this.add(t.name, { kind: "typedef", comment: c, tagIndex: i, prefix: "" });
});
for (const s of sf.statements.nodes) this.collect(s);
// Declarations first: `exports.name = name` may come before the
// function declaration it exports, which is hoisted.
for (const s of sf.statements.nodes) if (!ts.isExpressionStatement(s)) this.collect(s);
for (const s of sf.statements.nodes) if (ts.isExpressionStatement(s)) this.collect(s);
}
private add(name: string, pos: JsPos): void {
@@ -681,9 +713,11 @@ class JsResolver {
if (named !== undefined) this.exportEquals.push(...named);
}
} else if (ts.isIdentifier(e.right) && this.roots.has(e.right.text)) {
// `exports.name = name`: the named declaration is the export.
for (const pos of this.roots.get(e.right.text)!)
if (pos.kind === "decl") this.add(target, pos);
// `exports.name = local`: the named declaration is the export (and
// already is under its own name).
if (target !== e.right.text)
for (const pos of [...this.roots.get(e.right.text)!])
if (pos.kind === "decl") this.add(target, pos);
} else this.add(target, this.declPos(s, s, e.right));
}
@@ -880,6 +880,7 @@ export const MAY_THROW_LIB_FNS: ReadonlySet<IrLibFn> = new Set([
"net.setAutoSelTimeout",
"error.argTypeThrow",
"error.propTypeThrow",
"array.foreignThrow",
"emitter.setMaxChk",
"emitter.setDefaultMaxChk",
"fs.readFileSyncBytes",
@@ -1081,6 +1082,7 @@ export const ALWAYS_THROW_LIB_FNS: ReadonlySet<IrLibFn> = new Set([
"error.nodeThrow",
"error.argTypeThrow",
"error.propTypeThrow",
"array.foreignThrow",
"fs.mkdtempChk",
"fs.readFileChk",
"fs.opendirChk",
+28 -3
View File
@@ -848,6 +848,7 @@ function buildLibFnSignatures() {
// (subject, characters, NUL-separated replacement texts): the fused
// chain of global one-character replacements.
"str.replaceChars": { argTypes: [STRING, STRING, STRING], result: STRING },
"str.codePoints": { argTypes: [STRING], result: arrayOf(STRING) },
/* ToNumber(string) — ECMA-262 7.1.4.1 StringToNumber (scr_string.c):
* trim the JS StrWhiteSpace set, empty/whitespace-only → +0, then the
* whole span must be one StrNumericLiteral — signed decimal (Infinity
@@ -1547,15 +1548,18 @@ function buildLibFnSignatures() {
* true } and a callback taking an address string or list, then dials
* the answers in order. MAY THROW (host validation, a lookup throw). */
"net.connectLookupDyn": { argTypes: [F64, STRING, DYN], result: NETSOCKET_T },
"net.sockWrite": { argTypes: [NETSOCKET_T, STRING], result: VOID },
"net.sockWriteBytes": { argTypes: [NETSOCKET_T, BYTES_U8], result: VOID },
/** socket.write: Node's Writable answer — false when the writes not yet
* completely sent, each counted in full, reach the 65536-byte
* high-water mark ('drain' follows), and on an ended or closed socket. */
"net.sockWrite": { argTypes: [NETSOCKET_T, STRING], result: BOOL },
"net.sockWriteBytes": { argTypes: [NETSOCKET_T, BYTES_U8], result: BOOL },
"net.sockEnd": { argTypes: [NETSOCKET_T], result: VOID },
"net.sockEndStr": { argTypes: [NETSOCKET_T, STRING], result: VOID },
"net.sockEndBytes": { argTypes: [NETSOCKET_T, BYTES_U8], result: VOID },
/** write/end with a CHECKED-DYNAMIC chunk (an untyped JS payload into a
* typed socket): the runtime dispatches STR/BYTES and throws Node's
* ERR_INVALID_ARG_TYPE chunk TypeError on any other kind. */
"net.sockWriteDyn": { argTypes: [NETSOCKET_T, DYN], result: VOID },
"net.sockWriteDyn": { argTypes: [NETSOCKET_T, DYN], result: BOOL },
"net.sockEndDyn": { argTypes: [NETSOCKET_T, DYN], result: VOID },
"net.sockDestroy": { argTypes: [NETSOCKET_T], result: VOID },
"net.sockPipe": { argTypes: [NETSOCKET_T, NETSOCKET_T], result: VOID },
@@ -2013,6 +2017,12 @@ function buildLibFnSignatures() {
argTypes: [NETSOCKET_T, { kind: "func", params: [], ret: VOID }, BOOL],
result: VOID,
},
/** on('drain') / once('drain'): fires after a write answered false, once
* the write buffer empties. */
"net.sockOnDrain": {
argTypes: [NETSOCKET_T, { kind: "func", params: [], ret: VOID }, BOOL],
result: VOID,
},
"net.sockRemoteAddress": { argTypes: [NETSOCKET_T], result: VOID },
/** socket.encrypted — `boolean | undefined`: the true arm iff the
* socket carries a TLS transport (Node types `encrypted: true` on
@@ -2139,6 +2149,12 @@ function buildLibFnSignatures() {
/** socket.address(): { address, family, port } or {} as a dyn. */
"net.sockAddress": { argTypes: [NETSOCKET_T], result: DYN },
"net.sockBytesWritten": { argTypes: [NETSOCKET_T], result: F64 },
/** socket.writableLength: Node's state.length, the length of every
* write not yet completely sent. */
"net.sockWritableLength": { argTypes: [NETSOCKET_T], result: F64 },
/** socket.writableNeedDrain: a write answered false and 'drain' has not
* fired, with the write half open. */
"net.sockWritableNeedDrain": { argTypes: [NETSOCKET_T], result: BOOL },
/** remotePort (0), remoteFamily (1), localAddress (2), localPort (3),
* localFamily (4): dyn values, undefined when unknown. */
"net.sockAddrField": { argTypes: [NETSOCKET_T, F64], result: DYN },
@@ -3974,6 +3990,15 @@ function buildLibFnSignatures() {
// The string keys `for...in` visits on an array (args[0], any element
// type) after its indices. Borrowed; never throws.
"array.namedKeys": { argTypes: [null], result: arrayOf(STRING) },
// The value of the foreign element (SCR_ARR_FOREIGN) at args[1] of an
// array (args[0], any element type); callers test the element state
// first. Borrowed; never throws.
"array.foreign": { argTypes: [null, F64], result: DYN },
/** The checked conversion's TypeError for the foreign element at args[1]
* of an array (args[0]) read where a value described by args[2] is
* expected. ALWAYS THROWS; polymorphic result (the error.nodeThrow
* pattern). May-throw seed. */
"array.foreignThrow": { argTypes: [null, F64, STRING], result: VOID },
// Whether the array (args[0]) still has the key (args[1]) from its
// namedKeys list. Borrowed; never throws.
"array.hasNamedKey": { argTypes: [null, STRING], result: BOOL },
@@ -221,6 +221,7 @@ const MODULE_FIELDS: Record<StringFields<Omit<IrModule, "functions">>, Role> = {
"wasmUntranslated[].sha256": "other",
"wasmUntranslated[].reason": "other",
"threadSlotKeys[]": "other",
"foreignElementKeys[]": "other",
"freezableArrays[]": "other",
};
@@ -867,6 +867,8 @@ export function analyzeIntegerRangeFacts(
);
if (e.type.kind === "f64") {
if (e.method === "length") range = { min: 0, max: STRING_MAX_UNITS };
// A string holds at most SCR_STR_MAX_CAP (0x7fffffff) bytes.
else if (e.method === "byteLength") range = { min: 0, max: STRING_MAX_UNITS };
// An empty needle is found at the clamped position, up to the length.
else if (e.method === "indexOf") range = { min: -1, max: STRING_MAX_UNITS };
}
@@ -31,7 +31,8 @@ export const STR_INTRINSIC_SIGS: Record<
normalize: { argTypes: [STRING], minArgs: 1, result: STRING },
isWellFormed: { argTypes: [], minArgs: 0, result: BOOL },
toWellFormed: { argTypes: [], minArgs: 0, result: STRING },
cpAt: { argTypes: [F64], minArgs: 1, result: STRING },
byteLength: { argTypes: [], minArgs: 0, result: F64 },
cpAtByte: { argTypes: [F64], minArgs: 1, result: STRING },
};
/** Per-method signature for regexIntrinsic. test/source/flags take the
+11 -4
View File
@@ -1067,6 +1067,10 @@ export interface IrModule {
* ProgramLifetime instance lives in a counted region (the region.*
* intrinsics; backend/llvm/regions.ts). */
lifetimeRegions?: true;
/** Element keys (typeKey of the element type) of native arrays that can
* hold elements outside their element type (SCR_ARR_FOREIGN), stored
* through dynamic views; "*" for every array. Absent when none can. */
foreignElementKeys?: string[];
/** Classes used as ThreadSlots keys (@scriptc/experimental): the root of
* each one's hierarchy carries a slot word (the slots.* intrinsics;
* backend/llvm/thread-slots.ts). */
@@ -2447,10 +2451,13 @@ export type IrStrIntrinsicMethod =
// surrogate, and the copy with each one replaced by U+FFFD.
| "isWellFormed"
| "toWellFormed"
// The string iterator's step: the full code-POINT character at a UTF-16
// index (astral chars come back whole where charAt truncates) — for-of
// over strings desugars onto it, advancing by the result's length.
| "cpAt";
// The string iterator over the WTF-8 bytes: the byte count of a string,
// and the code-point character that starts at a byte offset (a lone
// surrogate is a one-unit character). for-of, spread and Array.from of a
// string walk a byte cursor with them, advancing by the character's
// byte count; the cursor is always a character start below the count.
| "byteLength"
| "cpAtByte";
/** The typed-array/Buffer method surface (bytesIntrinsic). Receiver/arg
* conventions (validated): `length`/`byteLength` are property reads → f64;
+3 -1
View File
@@ -4313,7 +4313,9 @@ function validateFunction(
if (
e.fn === "array.namedKeys" ||
e.fn === "array.hasNamedKey" ||
e.fn === "array.hasProperties"
e.fn === "array.hasProperties" ||
e.fn === "array.foreign" ||
e.fn === "array.foreignThrow"
) {
if (e.args[0]?.type.kind !== "array") {
err(`libCall ${e.fn} arg 0: expected an array, got ${e.args[0]?.type.kind}`, e.loc);
+1 -1
View File
@@ -1,6 +1,6 @@
{
"schemaVersion": 2,
"compilerVersion": "0.4.4",
"compilerVersion": "0.4.5",
"coverage": [
"Entries are projected mechanically from the compiler's own decision tables: the diagnostics registry, the unsupported-syntax dispatch tables, the stdlib and node-builtin lowering tables, and the supported-builtin-module list. Nothing is hand-maintained; the manifest regenerates byte-identically from the source tree at this version.",
"Absence from this manifest means 'not projected', never 'unsupported'. Surfaces lowered through dedicated code paths rather than tables are not yet projected: console, JSON, Promise/async and the timer surface, the net/http/tls/https/dgram/dns/assert/test/stream/readline module member surfaces, template literals, the regex slice, global functions (parseInt, parseFloat, isNaN, isFinite), and the process surface outside its ambient slice.",
@@ -201,7 +201,7 @@ console.log(lib.total([1, 2, 3]));
expect(fn).toContain("@scr_js_keyed_probe(");
// The length read answers from the native array's length.
expect(fn).toContain("icg.rest");
expect(fn).toContain("@scr_js_get_probe(");
expect(fn).toContain("@scr_js_get_probe_native(");
} finally {
await rm(dir, { recursive: true, force: true });
}
@@ -378,7 +378,7 @@ done:
expect(await symbols(helperObject, ["-u"])).toEqual(await symbols(clangObject, ["-u"]));
expect(await symbols(helperObject, ["-gU"])).toEqual(await symbols(clangObject, ["-gU"]));
expect(await symbols(helperObject, ["-gU"])).toEqual(["0000000000000000 T _main"]);
expect(await symbols(helperObject, ["-u"])).toContain("_scr_runtime_abi_v44");
expect(await symbols(helperObject, ["-u"])).toContain("_scr_runtime_abi_v45");
const clangExe = join(dir, "clang-program");
const linkDriver = join(dir, "clang-driver.c");
@@ -712,7 +712,7 @@ done:
.split("\n");
expect(undefinedSymbols).toContain("_sf_scale");
expect(undefinedSymbols).toContain("_sf_callback_mix");
expect(undefinedSymbols).toContain("_scr_runtime_abi_v44");
expect(undefinedSymbols).toContain("_scr_runtime_abi_v45");
});
test("helper and clang object paths retain the same outbound FFI C ABI", async () => {
@@ -0,0 +1,42 @@
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { expect, test } from "vitest";
import { compile } from "../src/index.js";
async function emitLlvm(source: string): Promise<string> {
const dir = await mkdtemp(join(tmpdir(), "scriptc-regex-replacers-"));
try {
const entry = join(dir, "main.ts");
const outPath = join(dir, "main.ll");
await writeFile(entry, source);
const result = await compile(entry, { outDir: dir, outPath, outputKind: "llvm" });
expect(result.ok, JSON.stringify(result.diagnostics)).toBe(true);
return await readFile(outPath, "utf8");
} finally {
await rm(dir, { recursive: true, force: true });
}
}
test("global substitutions of regex literals with literal replacements run a replacer", async () => {
const llvm = await emitLlvm(`
const s = process.argv[2] ?? "a<b>c";
const rep = process.argv[3] ?? "-";
console.log(
s.replace(/<[^>]*>/g, "|"),
s.replaceAll(/a[bc]/g, "<2>"),
s.replace(/<[^>]*>/g, "|"),
s.replace(/<[^>]*>/g, "$&$&"),
s.replace(/<[^>]*>/, "|"),
s.replace(/<[^>]*>/gy, "|"),
s.replace(/<[^>]*>/g, rep),
);
`);
const replacers = llvm.match(/^define internal i32 @sc_rr_\d+\(/gm) ?? [];
// One replacer per (pattern, flags, replacement): the repeated site shares it.
expect(replacers).toHaveLength(2);
expect(llvm.match(/call ptr @scr_regex_replace_with\(/g)).toHaveLength(3);
// $ patterns, non-global and sticky patterns, and computed replacements
// keep the runtime's loop.
expect(llvm.match(/call ptr @scr_regex_replace\(/g)!.length).toBeGreaterThanOrEqual(4);
});
@@ -102,6 +102,82 @@ console.log(copy.map((item) => weigh(item)).join(","));
expect(signature(spread, "weigh")).toBe("weigh(Item|undefined): f64");
});
test("holes stored through a dynamic view widen callbacks over the arrays it views", async () => {
const viewed = await lower(`${ITEM}
const items: Item[] = [new Item(1)];
const view: unknown[] = items;
view[3] = new Item(2);
console.log(items.findIndex((item) => weigh(item) > 1));
`);
expect(signature(viewed, "weigh")).toBe("weigh(Item|undefined): f64");
// A dynamic store cannot reach an element type no dynamic value views.
const other = await lower(`${ITEM}
const items: Item[] = [new Item(1)];
const values: unknown[] = [1];
values[3] = 2;
console.log(items.findIndex((item) => weigh(item) > 1));
`);
expect(signature(other, "weigh")).toBe("weigh(Item): f64");
// A view without stores that leave holes keeps the parameters.
const read = await lower(`${ITEM}
const items: Item[] = [new Item(1)];
const view: unknown[] = items;
console.log(view.length, items.findIndex((item) => weigh(item) > 1));
`);
expect(signature(read, "weigh")).toBe("weigh(Item): f64");
// Standard library functions that take arrays as `any` only read them.
const readers = await lower(`${ITEM}
const items: Item[] = [new Item(1)];
console.log(items.length, JSON.stringify(items));
const values: unknown[] = [1];
values[3] = 2;
console.log(items.findIndex((item) => weigh(item) > 1));
`);
expect(signature(readers, "weigh")).toBe("weigh(Item): f64");
// A store through a Record type is not a store through a view.
const record = await lower(`${ITEM}
const items: Item[] = [new Item(1)];
const view: unknown = items;
const bag: Record<string, unknown> = {};
bag["3"] = view;
console.log(items.findIndex((item) => weigh(item) > 1));
`);
expect(signature(record, "weigh")).toBe("weigh(Item): f64");
// An unknown[] that only holds the array its declaration creates is never
// a view, so stores past its end leave the viewed arrays alone.
const fresh = await lower(`${ITEM}
class Slots {
readonly states: unknown[] = [];
}
const slots = new Slots();
const items: Item[] = [new Item(1)];
const view: unknown = items;
const states = slots.states;
states[3] = view;
console.log(items.findIndex((item) => weigh(item) > 1));
`);
expect(signature(fresh, "weigh")).toBe("weigh(Item): f64");
// A mutable property every assignment of which is a fresh array is fresh
// storage too.
const reassigned = await lower(`${ITEM}
class Facts {
private facts: unknown[] = [];
begin(): void {
this.facts = [];
}
add(fact: unknown): void {
this.facts[3] = fact;
}
}
const items: Item[] = [new Item(1)];
const facts = new Facts();
facts.begin();
facts.add(items);
console.log(items.findIndex((item) => weigh(item) > 1));
`);
expect(signature(reassigned, "weigh")).toBe("weigh(Item): f64");
});
test("holes that only their owning slot can observe stay out of callbacks", async () => {
const module = await lower(`${ITEM}
class Cache {
@@ -0,0 +1,47 @@
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, expect, test } from "vitest";
import { compile } from "../src/index.js";
const directories: string[] = [];
afterEach(async () => {
await Promise.all(directories.splice(0).map((dir) => rm(dir, { recursive: true, force: true })));
});
async function emit(source: string): Promise<string> {
const outDir = await mkdtemp(join(tmpdir(), "scriptc-string-iteration-"));
directories.push(outDir);
const entry = join(outDir, "main.ts");
await writeFile(entry, source);
const outPath = join(outDir, "program.ll");
const result = await compile(entry, { entry, outDir, outPath, outputKind: "llvm" });
expect(result.ok, JSON.stringify(result)).toBe(true);
return readFile(outPath, "utf8");
}
const calls = (llvm: string, symbol: string): string[] =>
llvm.split("\n").filter((line) => line.includes("call ") && line.includes(`@${symbol}(`));
test("for-of over a string walks a byte cursor", async () => {
const llvm = await emit(`
function count(s: string): number {
let n = 0;
for (const c of s) n += c.length;
return n;
}
console.log(count(process.argv[1] ?? ""));
`);
expect(calls(llvm, "sc_str_cp_at_byte")).toHaveLength(1);
expect(calls(llvm, "scr_str_cp_at")).toHaveLength(0);
// The helper answers ASCII bytes from the one-character table.
expect(llvm).toContain("define internal ptr @sc_str_cp_at_byte(ptr %s, i64 %b)");
});
test("spread and Array.from of a string build the array in one call", async () => {
const llvm = await emit(`
const s = process.argv[1] ?? "";
console.log([...s].length, Array.from(s).length);
`);
expect(calls(llvm, "scr_str_code_points")).toHaveLength(2);
});
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/6067-view-holes-in-typed-arrays.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/6068-foreign-elements-through-views.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/6069-foreign-element-writers.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7515-net-socket-backpressure.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/7519-net-socket-drain-listener-close.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/8010-http-header-raw-bytes.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/8011-foreign-element-tojson.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/8347-string-iteration-code-points.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/8348-string-iteration-worker-characters.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/8351-regex-literal-replacers.ts"
],
"diags": []
}
@@ -0,0 +1,6 @@
{
"order": [
"<repo>/tests/corpus/intersection-restated-members.ts"
],
"diags": []
}
@@ -0,0 +1,7 @@
{
"order": [
"<repo>/tests/corpus/native-array-dynamic-reads/reads.js",
"<repo>/tests/corpus/native-array-dynamic-reads/main.ts"
],
"diags": []
}
+96 -1
View File
@@ -2,7 +2,7 @@
* packages: commander's HelpContext.helpWidth @typedef property and ms's
* @return tags, patched from the fixtures' package.json; installed-version
* selection; and the warnings for patches that do not apply as written. */
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, expect, test } from "vitest";
@@ -125,6 +125,101 @@ test("an @types twin is matched by the runtime package's installed version", ()
}
});
test("an @types twin in pnpm's layout is matched by the runtime package the project resolves", () => {
// pnpm keeps each package in its own directory under node_modules/.pnpm
// and links the project's dependencies from node_modules: the twin has no
// runtime package beside it.
const root = mkdtempSync(join(tmpdir(), "scriptc-type-patches-pnpm-"));
try {
const store = join(root, "node_modules/.pnpm");
const runtime = join(store, "tallyjs@0.5.0/node_modules/tallyjs");
const types = join(store, "@types+tallyjs@0.4.0/node_modules/@types/tallyjs");
mkdirSync(runtime, { recursive: true });
mkdirSync(types, { recursive: true });
writeFileSync(join(runtime, "package.json"), '{ "name": "tallyjs", "version": "0.5.0" }');
writeFileSync(join(runtime, "index.js"), "exports.totals = () => ({});\n");
writeFileSync(join(types, "package.json"), '{ "name": "@types/tallyjs", "version": "0.4.0" }');
writeFileSync(
join(types, "index.d.ts"),
"export type Totals = {\n max: number;\n};\nexport declare function totals(): Totals;\n",
);
mkdirSync(join(root, "node_modules/@types"), { recursive: true });
symlinkSync(runtime, join(root, "node_modules/tallyjs"));
symlinkSync(types, join(root, "node_modules/@types/tallyjs"));
const patchFor = (range: string) =>
writeFileSync(
join(root, "package.json"),
JSON.stringify({
scriptc: {
typePatches: {
[`tallyjs@${range}`]: { declarations: { "Totals.max": "| undefined" } },
},
},
}),
);
// A range the runtime (0.5.0) satisfies applies, though the twin's own
// version (0.4.0) does not satisfy it.
patchFor("0.5.0");
configureTypePatches(join(root, "main.ts"), parse);
expect(typePatchedText(join(types, "index.d.ts"))).toContain("max: number | undefined;");
// A range only the twin's own version satisfies does not.
patchFor("0.4.0");
configureTypePatches(join(root, "main.ts"), parse);
expect(typePatchedText(join(types, "index.d.ts"))).toBeUndefined();
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("an export before its hoisted function is patched, and string text that looks like JSDoc is not", () => {
const root = mkdtempSync(join(tmpdir(), "scriptc-type-patches-js-"));
try {
const pkg = join(root, "node_modules/hoistkit");
mkdirSync(pkg, { recursive: true });
writeFileSync(join(pkg, "package.json"), '{ "name": "hoistkit", "version": "1.0.0" }');
const source = [
'"use strict";',
"exports.measure = size;",
'exports.doc = "/** @typedef {object} Opts\\n * @property {number} width */";',
"",
"/**",
" * @returns {number}",
" */",
"function size(s) {",
" return s.length || undefined;",
"}",
"",
].join("\n");
writeFileSync(join(pkg, "index.js"), source);
writeFileSync(
join(root, "package.json"),
JSON.stringify({
scriptc: {
typePatches: {
"hoistkit@^1.0.0": {
declarations: { "measure()": "| undefined", "Opts.width": "| undefined" },
},
},
},
}),
);
configureTypePatches(join(root, "main.ts"), parse);
// The export, under another name, comes before the function it exports:
// the function's @returns tag widens.
expect(typePatchedText(join(pkg, "index.js"))).toBe(
source.replace(" * @returns {number}\n", " * @returns {number | undefined}\n"),
);
// The string's text is a value, not a typedef: the patch finds nothing.
expect(
typePatchDiagnostics()!
.diagnostics.map((d) => d.message)
.join("\n"),
).toContain("Opts.width");
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("files replaces a declaration file and declarations still apply on top", () => {
const root = join(fixtures, "npm/cases/type-patch-files");
configureTypePatches(join(root, "main.ts"), parse);
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/experimental",
"version": "0.4.4",
"version": "0.4.5",
"description": "Experimental scriptc APIs, with Node.js implementations: lifetime regions for ProgramLifetime objects, intern tables shared between threads, and per-thread slots on shared objects",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/llvm-darwin-arm64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Pinned LLVM code-generation helper for scriptc on macOS arm64",
"license": "Apache-2.0 WITH LLVM-exception",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/llvm-darwin-x64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Pinned LLVM code-generation helper for scriptc on macOS x64",
"license": "Apache-2.0 WITH LLVM-exception",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/llvm-linux-arm64-gnu",
"version": "0.4.4",
"version": "0.4.5",
"description": "Pinned LLVM code-generation helper for scriptc on Linux arm64",
"license": "Apache-2.0 WITH LLVM-exception",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/llvm-linux-arm64-musl",
"version": "0.4.4",
"version": "0.4.5",
"description": "Statically linked LLVM code-generation helper for scriptc on Linux arm64 musl",
"license": "Apache-2.0 WITH LLVM-exception",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/llvm-linux-x64-gnu",
"version": "0.4.4",
"version": "0.4.5",
"description": "Pinned LLVM code-generation helper for scriptc on Linux x64",
"license": "Apache-2.0 WITH LLVM-exception",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/llvm-linux-x64-musl",
"version": "0.4.4",
"version": "0.4.5",
"description": "Statically linked LLVM code-generation helper for scriptc on Linux x64 musl",
"license": "Apache-2.0 WITH LLVM-exception",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/llvm-win32-x64-msvc",
"version": "0.4.4",
"version": "0.4.5",
"description": "Pinned LLVM code-generation helper for scriptc on Windows x64",
"license": "Apache-2.0 WITH LLVM-exception",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/runtime-android-arm64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Precompiled scriptc library runtime pack for Android arm64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/runtime-darwin-arm64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Precompiled scriptc runtime pack for macOS arm64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
@@ -366,7 +366,7 @@ async function build() {
package: packageManifest.name,
version: packageManifest.version,
target: RUNTIME_PACK_MATRIX.target,
runtime_abi: { version: 44, marker: "scr_runtime_abi_v44" },
runtime_abi: { version: 45, marker: "scr_runtime_abi_v45" },
compiler: {
command: compiler,
identity: compilerVersion,
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/runtime-darwin-x64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Precompiled scriptc runtime pack for macOS x64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@scriptc/runtime-ios-arm64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Precompiled scriptc library runtime pack for iOS arm64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",
@@ -1,6 +1,6 @@
{
"name": "@scriptc/runtime-ios-simulator-arm64",
"version": "0.4.4",
"version": "0.4.5",
"description": "Precompiled scriptc library runtime pack for iOS simulator arm64",
"license": "Apache-2.0",
"homepage": "https://scriptc.dev",

Some files were not shown because too many files have changed in this diff Show More