mirror of
https://github.com/vercel-labs/scriptc.git
synced 2026-11-08 08:44:37 +08:00
Malte Ubl
b275f47fb3
readFileSync(path | fd, "utf8") (any ASCII-case utf8/utf-8 spelling and the empty default) used to read into a Buffer, scan it for UTF-8 validity, then copy it into a new string while scanning it again for ASCII. It now reads directly into the final string allocation, scans each read chunk for ASCII while it is still in cache, and validates the remainder in place (scr_str_adopt_utf8_lossy). Only malformed input is repaired into a new string, with the same WHATWG maximal-subpart U+FFFD replacement as Buffer#toString; a BOM is kept. The raw-string readers behind fs.promises.readFile(path, "utf8") and readFileSync(fd, "utf8") now decode the same way, so malformed bytes and encoded surrogates become U+FFFD as in Node instead of passing through. The ASCII prefix scan tests 32 bytes per step (scr_bytes_ascii_span), which also speeds up Buffer and TextDecoder UTF-8 decoding of long ASCII runs. The corpus program covers empty files, BOMs, every malformed-subpart class, truncation at EOF, chunk-boundary offsets, large ASCII and multibyte files, /proc pseudo-files, fd and promise reads; the runtime unit test checks the in-place decode under ASan and the RC audit.