Files
Charlie Marsh 301b7ed4d6 [ty] Share heap-backed parser names with CharStr (#26594)
## Summary

This PR changes `ruff_python_ast::Name` from a 24-byte `CompactString`
to a 16-byte immutable `CharStr` from
[`astral-sh/char_str`](https://github.com/astral-sh/char_str).

`char_str` is an Astral-maintained fork of
[`lean_string`](https://github.com/ryota2357/lean_string) that provides
compact owned strings with mutable and immutable variants: growable
`CharString` for construction and immutable, exactly allocated `CharStr`
for storage. (It also provides Salsa and get-size features.)

Names up to 16 UTF-8 bytes remain inline, static names remain
allocation-free, and longer names use an exactly sized,
reference-counted allocation with no unused capacity.

Removing eight bytes from every `Name` also shrinks common AST nodes.
For example, `Identifier` and `ExprName` drop from 40 to 32 bytes, while
`ExprAttribute`, `StmtFunctionDef`, `StmtClassDef`, and several other
variants each drop by eight bytes.

The parser interns names that exceed the inline capacity, so repeated
longer names share one backing allocation within a parser invocation.

## Why 16 bytes?

Across the full `mypy_primer` corpus, there 20.8 million name
occurrences:

| UTF-8 length | Occurrences | Share | Unique name/file pairs | Share |
|---|---:|---:|---:|---:|
| 1–16 bytes | 18,955,757 | 91.02% | 3,192,424 | 79.42% |
| 17–24 bytes | 1,348,894 | 6.48% | 529,826 | 13.18% |
| 25+ bytes | 522,376 | 2.51% | 297,575 | 7.40% |
| Total | 20,827,027 | 100% | 4,019,825 | 100% |

The 16-byte cutoff keeps 91% of occurrences inline while saving eight
bytes for every occurrence. The 1–16-byte bucket alone removes 151.6 MB
of handle storage across the corpus.

The tradeoff is the 17–24-byte bucket: `CompactString` stored those
names inline, while `CharStr` allocates once per unique name within a
file. But the interning amortizes that cost when a name repeats. Names
longer than 24 bytes already allocated on `main`; for those names, this
change both reduces the handle size and deduplicates repeated backing
allocations.

## Memory impact

Accounting for the observed per-file multiplicities and structural
sharing, modeled `Name` storage across the corpus changes as follows:

| | `main` | This PR | Difference |
|---|---:|---:|---:|
| `Name` handles and backing storage | 526.18 MB | 366.27 MB | -159.91
MB (-30.4%) |

The current [instrumented memory
report](https://github.com/astral-sh/ruff/pull/26594#issuecomment-4906725175)
shows a consistent reduction in total retained memory:

| Project | `main` | This PR | Difference |
|---|---:|---:|---:|
| flake8 | 29.21 MB | 28.71 MB | -1.69% |
| trio | 71.01 MB | 70.01 MB | -1.41% |
| sphinx | 168.81 MB | 166.82 MB | -1.18% |
| prefect | 454.75 MB | 449.06 MB | -1.25% |

Within those totals, `parsed_module` decreases by roughly 2.1% and
`semantic_index` by 2.4–2.7%. Several name-heavy collections decrease by
considerably more.
2026-07-16 14:53:48 -04:00
..