mirror of
https://github.com/vercel-labs/scriptc.git
synced 2026-10-28 06:53:42 +08:00
Chris Tate
db2cb5c99e
readline split redirected input byte by byte, twice per chunk (once to find whether the chunk ends a line, once to split it). Before splitting it copied each chunk out of its Buffer and into a scratch buffer, decoded it into a new string, and copied the unfinished line and the whole chunk into a joined buffer. Now: - one scan per chunk: memchr for a CR once, then LF to LF with memchr when the chunk has none, and the LF, CR and CRLF split otherwise; - the unfinished line joins only the first line of the chunk; - an ASCII chunk with no partial UTF-8 sequence held from the previous one is split where it is, and a chunk that is not shared memory is read from its own storage. Lines are still copied into their strings before the first is delivered. Buffer.from(string, 'latin1' | 'ascii') encoded character by character; an ASCII prefix, all of an ASCII string, is now copied with memcpy. Line events, their order and the CRLF delay are unchanged. Corpus 8344 covers lines joined across chunks, a line longer than a chunk, empty lines, mixed line endings and UTF-8 split across chunks. The new line-reader workload reads a 67 MB log through readline over a file stream.