Files
ruff/crates/ruff_python_parser
Micha Reiser 9a5220ecdb Reuse parser scratch buffers (#26798)
## Summary

Use scratch buffers instead of allocating new `Vec`s and shrinking them
when constructing the AST node. While this doesn't remove the need for
copying the elements, it reduces the allocations within our parser
(`shrink_to_fit` often needs to allocate a new buffer and move all
elements). One shared scratch buffer per node type is sufficient,
because our parser is recursive (more nested parse rule always push to
the end, and parse function drain from the end as they complete).


## Performance

Very consistent perf improvement

## Codspeed memory regression

This can cause the parser's peak memory to increase, because we retain
the scratch buffer allocation after parsing a clause completed. However,
that memory is only temporary and released as soon as the parser
completes. Most project-level memory reports show a memory reduction.
It's only specific parser benchmarks that show a memory increase
2026-07-15 17:09:07 -07:00
..
2026-07-15 17:09:07 -07:00
2026-07-09 15:42:27 -04:00

Ruff Python Parser

Ruff's Python parser is a hand-written recursive descent parser which can parse Python source code into an Abstract Syntax Tree (AST). It also utilizes the Pratt parsing technique to parse expressions with different precedence.

Try out the parser in the playground.

Contributing

Refer to the contributing guidelines to get started and GitHub issues with the parser label for issues that need help.

Versioning

This crate is an internal component of Ruff. The Rust API exposed here is unstable and will have frequent breaking changes.

This version (0.0.4) is a component of Ruff 0.15.21. The source can be found here.

See Ruff's crate versioning policy for details on versioning.