mirror of
https://github.com/astral-sh/ruff.git
synced 2026-11-22 17:43:09 +08:00
## Summary On main, parsing `b"\x41"` calls `parse_unicode_literal(2)`. The x86-64 profiling build uses one function for widths 2, 4, and 8, so the loop bound is a runtime argument. For each digit (`4`, then `1`), the loop saves its source position and calls `next_char()` to decode and consume a UTF-8 character before checking whether it is a hex digit. This PR calls `parse_unicode_literal::<2>()` for `\x`, which lets the compiler specialize the loop for two digits. This is the largest measured improvement. The new loop also peeks at the next byte and advances by one when it sees a valid ASCII hex digit. Only if that check fails does it save the source position and call `next_char()`. For an invalid escape such as `"\x4é"`, both versions consume `é` and report its two-byte range. Callgrind instruction counts for a whole uncached `ruff check` on [JAX](https://github.com/jax-ml/jax/tree/55029b8f7e65adb79764103871ec0357fa440227/jax/_src/internal_test_util/export_back_compat_test_data). On the compatibility fixtures, changing only the width to `const N` saves 8.36 million of the 13.19 million instructions saved by the full change. | Workload | Main | `const N` only | `const N` + deferred error position | Full change | Total reduction | | --- | ---: | ---: | ---: | ---: | ---: | | JAX export compatibility fixtures | 131,434,592 | 123,077,887 | 120,580,490 | 118,244,820 | 10.04% | | JAX export implementation, compatibility fixtures, and export tests | 226,412,243 | 218,062,626 | 215,565,059 | 213,219,152 | 5.83% | ## Test Plan The parser tests passed, and diagnostics matched main on valid, malformed, and non-ASCII escapes.
Ruff Python Parser
Ruff's Python parser is a hand-written recursive descent parser which can parse Python source code into an Abstract Syntax Tree (AST). It also utilizes the Pratt parsing technique to parse expressions with different precedence.
Try out the parser in the playground.
Contributing
Refer to the contributing guidelines to get started and GitHub issues with the parser label for issues that need help.
Versioning
This crate is an internal component of Ruff. The Rust API exposed here is unstable and will have frequent breaking changes.
This version (0.0.21) is a component of Ruff 0.17.4. The source can be found here.
See Ruff's crate versioning policy for details on versioning.