Files
ruff/crates/ruff_python_parser
Charlie Marsh b0290a1f04 [ty] Decode consecutive escapes in bytes literals without repeated searches (#30729)
## Summary

When parsing `b"\x00\xff\x12"`, we search for the next backslash after
every escape and copy the empty slice before it. A generated Qt resource
can contain millions of adjacent escapes.

We now decode consecutive escapes in a loop and search again when we
reach literal text. Raw literals and literals with no escapes still go
straight into a boxed slice. Ruff uses the same parser.

Cold `ty check` runs used 18.10–18.40% fewer instructions on the
[Receita Federal
dashboard](https://github.com/MrCl0wnLab/Python_PyQt5_DashReceitaFederal/tree/45f503e696f53b7a19c094c93650136746f75276)
and 10.52–10.68% fewer on [pyqt5-fugueicons
3.5.6.2](https://pypi.org/project/pyqt5-fugueicons/3.5.6.2/). Both
checks covered the full project with PyQt5 installed. Output and exit
statuses matched across two Cachegrind pairs run in opposite orders.

<!-- ty-pinned-project-instructions:start -->
| Project | [Base
instructions](https://github.com/astral-sh/ruff/commit/943f623f55c9aa625b0b1b21af731e451819accc)
| [PR
instructions](https://github.com/astral-sh/ruff/commit/73c182411d36f0ca15ecdf7a5ce19602fccfba5f)
| Change | Paired range |
| --- | ---: | ---: | ---: | ---: |
|
[Black](https://github.com/psf/black/tree/55996d1fb8342725c8a19b6023cd9439c3c1d6f7)
(inconclusive) | 3,109,688,394 | 3,112,370,720 | +0.09% | -0.06% to
+0.23% |
|
[discord.py](https://github.com/Rapptz/discord.py/tree/65232c38702be5844cf2ce865a4777eb1928b5d0)
(inconclusive) | 7,988,415,246 | 7,986,092,835.5 | -0.03% | -0.14% to
+0.08% |
| [Home
Assistant](https://github.com/home-assistant/core/tree/c19ae285ede74cd64c3045a3a4aa43290c224f1b)
(inconclusive) | 153,601,199,195.5 | 153,570,946,275 | -0.02% | -0.03%
to -0.01% |
|
[isort](https://github.com/pycqa/isort/tree/a514c71ed86c2216d0040c7d60e302ed76a4f7ab)
(inconclusive) | 1,815,648,783 | 1,814,312,772 | -0.07% | -0.23% to
+0.09% |
|
[Jinja](https://github.com/pallets/jinja/tree/5ef70112a1ff19c05324ff889dd30405b1002044)
(inconclusive) | 1,901,015,497.5 | 1,901,438,268 | +0.02% | -0.02% to
+0.06% |
|
[pandas](https://github.com/pandas-dev/pandas/tree/a0225b65720018f4654390bb16d254daa6ee5e54)
(inconclusive) | 27,633,120,794 | 27,635,985,419 | +0.01% | -0.02% to
+0.04% |
|
[pandas-stubs](https://github.com/pandas-dev/pandas-stubs/tree/8dd6d91b1c05b26e30f7ddbe3dc3d4042a92bdd6)
(inconclusive) | 2,876,252,823 | 2,871,593,500 | -0.16% | -0.20% to
-0.12% |
|
[Prefect](https://github.com/PrefectHQ/prefect/tree/a9452ac21852755311913cb27887b083b5007c03)
(inconclusive) | 4,865,787,363 | 4,865,661,476 | 0.00% | -0.01% to
+0.01% |
|
[PyTorch](https://github.com/pytorch/pytorch/tree/ab92945cea42e47ae398aead69af0872d340ea0f)
(inconclusive) | 93,392,592,427.5 | 93,389,192,089 | 0.00% | -0.04% to
+0.03% |
|
[HTTPX](https://github.com/encode/httpx/tree/b5addb64f0161ff6bfe94c124ef76f6a1fba5254)
(inconclusive) | 1,651,258,325 | 1,652,028,954.5 | +0.05% | -0.02% to
+0.12% |
|
[NumPy](https://github.com/numpy/numpy/tree/f9f1772ce010001208475f751d19253469788410)
(inconclusive) | 8,264,291,000.5 | 8,280,480,492 | +0.20% | -0.43% to
+0.83% |
|
[FastAPI](https://github.com/fastapi/fastapi/tree/7a0a3f0b286c6c865cf4398ea0e957a87b8bffa4)
(inconclusive) | 2,946,172,205 | 2,945,431,231.5 | -0.03% | -0.05% to
0.00% |
|
[SQLAlchemy](https://github.com/sqlalchemy/sqlalchemy/tree/10f4d1e2d706a7722ca78e79ce05b00333a7aa5b)
(inconclusive) | 20,737,932,044 | 20,729,916,281.5 | -0.04% | -0.30% to
+0.22% |
|
[Rich](https://github.com/Textualize/rich/tree/9d8f9a372cc5916fd4781fec207ced7ddac2f08f)
(inconclusive) | 2,677,955,779.5 | 2,675,681,234.5 | -0.09% | -0.22% to
+0.05% |
|
[Django](https://github.com/django/django/tree/b501b8493f31cf8cba31211d59b3b78642b8cb02)
| 13,406,151,960.5 | 13,391,818,551.5 | -0.11% | -0.19% to -0.02% |
|
[SymPy](https://github.com/sympy/sympy/tree/85d3998a4517ffbc8a9709b8efa769bb417c174a)
(inconclusive) | 44,777,601,144 | 44,757,487,322.5 | -0.04% | -0.07% to
-0.02% |
| [OpenAI Python
SDK](https://github.com/openai/openai-python/tree/f6e8480f9a5b2a4d02b17a1da54a1e57964bc312)
| 15,443,651,644.5 | 15,428,984,317 | -0.09% | -0.11% to -0.08% |
|
[pytest](https://github.com/pytest-dev/pytest/tree/8b005ed7f972987033ec34dc1e68b0635fb8142b)
(inconclusive) | 8,598,245,018 | 8,602,942,340 | +0.05% | +0.02% to
+0.09% |
|
[aiohttp](https://github.com/aio-libs/aiohttp/tree/fe099491d275ef2c21bf683e024e34d51ac05d43)
| 4,068,369,032 | 4,064,286,198 | -0.10% | -0.15% to -0.05% |
|
[Ansible](https://github.com/ansible/ansible/tree/b4b2b1e82a535d27f902a71fc8396c3af138fca2)
(inconclusive) | 14,020,459,851 | 14,020,252,364.5 | 0.00% | -0.07% to
+0.06% |
|
[botocore](https://github.com/boto/botocore/tree/e250abbe956da791c16086575e6ddb1f2da81a78)
(inconclusive) | 2,934,192,207 | 2,933,585,829 | -0.02% | -0.03% to
-0.01% |
|
[Celery](https://github.com/celery/celery/tree/9c2f238334331db43f15956f131235fd3bb70fd1)
| 4,654,505,699 | 4,649,437,556.5 | -0.11% | -0.11% to -0.11% |
|
[Dask](https://github.com/dask/dask/tree/249a51a1db4261791b93b4ce04e67341729939c3)
(inconclusive) | 32,508,993,735.5 | 32,499,096,750.5 | -0.03% | -0.07%
to +0.01% |
|
[NetworkX](https://github.com/networkx/networkx/tree/6855cd586982cc2e4710b8eb87251ccf56674bf1)
(inconclusive) | 15,213,659,333.5 | 15,215,990,067.5 | +0.02% | 0.00% to
+0.03% |
|
[Pydantic](https://github.com/pydantic/pydantic/tree/560f0f96a37d6913afdfddb2de22030ef08e68c3)
(inconclusive) | 12,888,809,333 | 12,881,892,725 | -0.05% | -0.10% to
0.00% |
|
[scikit-learn](https://github.com/scikit-learn/scikit-learn/tree/ee56c8b5d809dc7b970a140ed501269b63c69cf8)
(inconclusive) | 58,204,434,489.5 | 58,209,929,274 | +0.01% | 0.00% to
+0.02% |
|
[Scrapy](https://github.com/scrapy/scrapy/tree/e63a574c767d7a9438c21db158f605f1c44c6e6a)
(inconclusive) | 8,740,085,360 | 8,739,502,452.5 | -0.01% | -0.06% to
+0.04% |
|
[Transformers](https://github.com/huggingface/transformers/tree/3a134bc37f7c22c801b618e289b0255425f22647)
(inconclusive) | 96,487,907,813.5 | 96,512,722,255.5 | +0.03% | -0.07%
to +0.13% |
<!-- ty-pinned-project-instructions:end -->
2026-11-21 22:29:36 -05:00
..
2026-11-19 10:32:47 -05:00
2026-11-19 10:32:47 -05:00

Ruff Python Parser

Ruff's Python parser is a hand-written recursive descent parser which can parse Python source code into an Abstract Syntax Tree (AST). It also utilizes the Pratt parsing technique to parse expressions with different precedence.

Try out the parser in the playground.

Contributing

Refer to the contributing guidelines to get started and GitHub issues with the parser label for issues that need help.

Versioning

This crate is an internal component of Ruff. The Rust API exposed here is unstable and will have frequent breaking changes.

This version (0.0.23) is a component of Ruff 0.17.6. The source can be found here.

See Ruff's crate versioning policy for details on versioning.