Martin Vogel 1b79558975 fix(grammar): accept a lone '&' in JSX strings and text (#1736) ...

A plain '&' inside JSX -- a URL query in an attribute
(href="https://fonts.googleapis.com/css2?family=Inter&display=swap"),
or text such as <p>Tom & Jerry</p> -- produced a tree-sitter ERROR node
in .js, .jsx and .tsx files. The file was flagged parse_partial in the
coverage view, and the error recovery turned words from the URL
(create-next-app -> create/next/app) into identifiers that resolved to
unrelated symbols as false USAGE edges.

Root cause is upstream tree-sitter-javascript, which the tsx dialect
builds on: unescaped_{double,single}_jsx_string_fragment is
/([^"&]|&[^#A-Za-z])+/, so '&' is only accepted before a space/digit
or as a complete html_character_reference (&amp; / &#38;). jsx_text is
scanned by the external scanner, which stops at every '&', and nothing
in _jsx_child accepted a '&' that is not a reference. So '&' + letter in
a string, and any lone '&' in JSX text, had no valid token.

Fix: javascript and tsx become self-maintained forks of the pins we
already vendored (javascript @ 58404d8cf191; typescript tsx @
75b3874edb2d on its javascript base @ 3a837b6f3658), following the
arkts model. One patch (tools/tree-sitter-javascript/patches/
jsx-lone-ampersand.patch) is applied to both javascript bases:
- a one-character '&' token at lexical precedence -1, aliased to the
  existing string_fragment (in strings) and jsx_text (in text); the
  lower precedence keeps &amp; / &#38; lexing as html_character_reference
- the string-fragment regexes exclude a '&' directly before the closing
  quote, so "a&" no longer runs past the end of the string
Node types are unchanged (same type/named set). The external scanners
stay upstream bytes. Parsers regenerated with tree-sitter-cli 0.25.10
(ABI 15; tsx moves from ABI 14, and its tree_sitter/ headers now match
arkts). Without the patch both fork dirs regenerate upstream exactly.
The upstream corpus passes unchanged: javascript 116/116; tsx gives the
same results as unpatched upstream.

Proof on vercel/next.js examples/ (canary 725705bd, 1,722 js/jsx/tsx
files): parse_partial 92 -> 47 (tsx 33 -> 0, jsx 2 -> 0, js 12 -> 2; the
two left are Babel do-expressions and Flow types), no newly flagged
file, code-graph nodes 14,739 -> 14,739, edges 24,773 -> 24,657: the
115 USAGE edges removed were all false cross-example edges from URL
fragments, plus a 4 SIMILAR_TO / +3 SEMANTICALLY_RELATED reshuffle
from the corrected ASTs.

Registered like arkts: MANIFEST.md fork rows + notes (ABI tally
recounted), audit FORKS map, THIRD_PARTY.md, vendored checksums.
LICENSEs stay upstream's MIT byte-identical.

Tests: parse_coverage gains a .js/.jsx/.tsx matrix (double/single-quoted
attribute, '&' before the quote, '&' + letter/space/digit in text, plus
&amp;/&#38; controls) and a guard that broken JSX is still flagged.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

2026-09-29 03:02:51 +02:00

..

corpus

fix(grammar): accept a lone '&' in JSX strings and text (#1736)

2026-09-29 03:02:51 +02:00