Chris Tate c5f423b464 Decode malformed UTF-8 to U+FFFD in the ICU converter and at directory and argument boundaries ...

The ICU unit's WTF-8 to UTF-16 conversion counted units from lead bytes
alone. A 4-byte-shaped encoding of a BMP value counted two units and wrote
one, so ICU received a length past the units written, and word segments
built from those ranges could return prior heap contents. The conversion now
decodes each maximal ill-formed subsequence to U+FFFD, as the WHATWG UTF-8
decoder does, keeps WTF-8's 3-byte lone surrogates as their unit, and
reports the units it wrote. The decoder lives in scr_intl_utf16.h.

Directory entry names (readdir, opendir) and process.argv entries now decode
as UTF-8 with U+FFFD for malformed bytes, as Node decodes them, instead of
entering strings as raw bytes.

test_intl_utf16.c checks the decoder against Node's TextDecoder over
malformed sequences. intl-malformed-utf8.test.ts compares a program that
segments malformed directory names and an argument against Node.

2026-10-29 08:07:53 -05:00

..

src

Decode malformed UTF-8 to U+FFFD in the ICU converter and at directory and argument boundaries

2026-10-29 08:07:53 -05:00

test

Decode malformed UTF-8 to U+FFFD in the ICU converter and at directory and argument boundaries

2026-10-29 08:07:53 -05:00

vendor

Match V8's default locale and calendar in the ICU formatters, and build Intl programs without Zig from the runtime pack

2026-10-29 02:42:35 -05:00

LICENSE

init

2026-07-22 18:08:04 -05:00

package.json

Prepare v0.4.5 release

2026-10-25 11:26:13 -05:00