Files
ruff/crates/ruff_python_parser
Charlie Marsh e3304857ec Specialize ASCII identifier lexing (#29503)
## Summary

We give ASCII identifiers a dedicated path, avoiding Unicode bookkeeping
and checking string prefixes only when a one- or two-letter name reaches
a quote. Both paths share the existing batched ASCII scanner, preserving
fast scanning for Unicode names with long ASCII suffixes.

Stacked on #29463 to measure the incremental improvement. [CodSpeed
reports](https://github.com/astral-sh/ruff/pull/29503/checks?check_run_id=118300735426)
6.6–9.8% greater efficiency across all five lexer benchmarks, with no
reported regressions. We preserve the existing scanner and add no
inlining attributes or unsafe code.
2026-10-23 13:58:34 -04:00
..
2026-10-22 13:54:24 -04:00
2026-10-22 13:54:24 -04:00

Ruff Python Parser

Ruff's Python parser is a hand-written recursive descent parser which can parse Python source code into an Abstract Syntax Tree (AST). It also utilizes the Pratt parsing technique to parse expressions with different precedence.

Try out the parser in the playground.

Contributing

Refer to the contributing guidelines to get started and GitHub issues with the parser label for issues that need help.

Versioning

This crate is an internal component of Ruff. The Rust API exposed here is unstable and will have frequent breaking changes.

This version (0.0.19) is a component of Ruff 0.17.2. The source can be found here.

See Ruff's crate versioning policy for details on versioning.