String storage can hold bytes that are not well-formed UTF-8: an HTTP
header value keeps the raw bytes a client sent. Encoding such a string
read past its end.
- scr_bytes_next_cp decoded any high byte as the lead of a sequence and
read up to three bytes after it without a bound. It now takes the
storage length, decodes only complete, well-formed sequences, and
returns any other byte as U+0000..U+00FF, advancing by one. That is
each such byte's value in Node, which decodes header values as latin1.
This covers latin1, ascii, binary and utf16le encoding, hex and base64
of non-ASCII text (which narrow through latin1), Buffer#write and
TextEncoder#encodeInto.
- base64 decoding read one byte past its input when the eight-character
groups ended exactly at the end.
The runtime's bytes unit test (built with ASan) encodes ill-formed
storage at every offset within a word and random high bytes, and
compares latin1, ascii and utf16le with a scalar reference. A corpus
program sends such header bytes to a server and encodes them as Node
does.
Chris Tate
e24ed12933