A visitor that checks for a yield and then walks the expression can
dispatch on its kind twice. Inlining walk_expr lets the compiler combine
the checks and removes the stack copy of the classified expression on
recursive calls.
Leave inlining to the compiler with debug assertions enabled. At
opt-level=0, forcing the walker into the visitor enlarges its stack
frame enough to overflow the ty server thread on 2,000 nested additions.
Resolving a child ID loads the row pointer from NodeStorage. Preserve
the field order of Ast to keep NodeStorage at offset zero and the row
pointer close to the start of the header.
In the x86-64 profiling build, the load in
Arguments::as_single_positional moved from offset 168 to offset zero.
That shrank the instruction from seven bytes to three. The same load
occurs throughout the generated child accessors.
Every variant of a classified expression carries the same row and arena
pointers. Expose the generated conversions across crate boundaries so
the compiler can return those pointers directly in ExprKind::view.
Start with a bounded reservation based on source length. When the row
buffer fills, use the observed node density to estimate how much space
the rest of the source will need.
Store child IDs in the arena. Expose borrowed typed nodes, fields and
source-order lists through generated accessors. The walkers accept an
already classified expression or statement. Resolve child IDs with
checked indexing.
Store positional and keyword arguments in one list for calls and class headers. Source-order visitors can walk the list directly, and evaluation-order visitors visit starred arguments before keywords.
Box keyword values so a positional argument does not carry the full size of a keyword entry.
Traverse source order when each parameter is checked independently,
and inspect roles directly when locating the positional-only slash.
Expression searches can check annotations and defaults in the same
written order, including for recovered signatures.
## Summary
Today, `ruff_python_ast` has two implementations of
`parenthesized_range`, but only tests use the source-tokenizing one.
This PR removes the unused implementation and moves the remaining
distinct cases into the token-based implementation's tests.
Testing: All AST integration tests pass, and retired instruction counts
show no regression in Ruff.
## Summary
Extracting PEP 723 metadata can scan the same suffix for every unclosed
opening delimiter, leading to quadratic work. Resume at the end of a
failed scan and accept already decoded Python source to avoid redundant
UTF-8 validation.
Testing: Parser tests pass and Ruff and ty compile.
## Summary
Use a shared ty override for Ruff’s standalone scripts now that PEP 723
scripts can inherit project settings. Fix the diagnostics in the two PR
automation scripts so they can use the same checks.
## Notes to reviewers
All 28 standalone scripts passed ty with locked dependencies, and the
changed-file hooks and mocked workflow checks passed.
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Summary
--
There were several new cargo and clippy lints here, so I tried to break
up the changes into several
commits:
- deleting some unused workspace dependencies, these looked kind of
intentional as they also had
some shear ignores, but it seems okay to delete those too?
- allowing the [redundant_homepage] lint. We could delete a lot of
these, but the ones in the `ruff`
and `ty` crates seem necessary for maturin, as I noted in a comment.
- fixing the [missing_lints_inheritance] lint. It seems that several of
our crates don't inherit our
workspace lints. For now I just included the `allow` for
`redundant_homepage`, but we could follow
up by inheriting the full table and fixing all the additional lints that
produces. (This commit is
actually combined with the previous one)
- allowing cargo's new [unused_dependencies] lint, which seems redundant
with `shear` and had some
false positives on our benchmark dependencies. Codex tried to work
around this with some
`cfg`-gated imports in the benchmark crate, but ignoring this seemed
better for now, at least
while we're still using `shear`.
- running `clippy --fix` on the remaining lint errors, with one manual
intervention where Clippy
dropped some of our `cfg` handling. These new diagnostics were mostly
around returning boolean
expressions.
- renaming `NodeIndex::_NONE` to avoid [used_underscore_items]
There are also some new APIs in 1.98 that we can use now. I'll follow up
with those separately.
[redundant_homepage]:
https://doc.rust-lang.org/cargo/reference/lints.html#redundant_homepage
[missing_lints_inheritance]:
https://doc.rust-lang.org/cargo/reference/lints.html#missing_lints_inheritance
[used_underscore_items]:
https://clippy.rust-lang.org/master/#used_underscore_items
[unused_dependencies]:
https://doc.rust-lang.org/cargo/reference/lints.html#unused_dependencies
## Summary
Previously, Ruff parsed the leading words of each docstring line as a
possible section name. Both `Returns:` and `Returns: the new value.`
matched `Returns`, so Ruff extracted the name again and checked the
remaining text to reject the second line.
Ruff now skips the indentation already measured by `leading_space`,
trims trailing whitespace, and strips one optional trailing colon. It
passes the rest of the line to the existing section-name parser.
`Returns:` yields `Returns`. The inline description in `Returns: the new
value.` makes the match fail immediately. On a match,
`suspected_as_section` returns both the section kind and the borrowed
name so the caller can record its source range.
Each measurement counts instructions for one uncached lint of the NumPy
`globals.py` benchmark after initialization.
| Rules | Before | After | Reduction |
| --- | ---: | ---: | ---: |
| All stable | 819,973 | 767,658 | 6.4% |
| All with preview | 951,580 | 899,315 | 5.5% |
## Test Plan
Ran the linter and Python AST tests and compared diagnostics and edits
across the benchmark files and docstring conventions.
## Summary
Ruff uses `ComparableExpr` to compare Python syntax while ignoring
details such as source ranges and expression contexts. Previously,
`ComparableExpr::from(&expr)` recursively built a second tree containing
the fields used for comparison. For `factory(item, *items, limit=2,
**options)`, it boxed the comparable callee and the operand of `*items`,
and collected the positional arguments and keywords into two vectors.
Equality and hashing then walked that tree. Statements, match patterns,
and modules built similar trees.
`ComparableExpr<'a>` now holds a single `ast::ExprRef<'a>`. The
statement, pattern, and module keys likewise borrow their AST nodes.
Constructing a key takes constant time and allocates nothing. When
comparing two calls, equality checks the callees first. If one is
`factory` and the other is `other`, it can return before visiting any
arguments. If the callees match, it walks the original argument slices,
comparing each child as needed. Hashing walks the same structural
fields, and `Debug` displays them. Borrowing the expression also removes
the need to run `drop` after comparing.
The field helpers exhaustively destructure each node so that adding an
AST field requires deciding whether it participates in comparison.
| Workload | `main` | This PR |
| --- | ---: | ---: |
| Size of a `ComparableExpr` on x86-64 | 128 bytes | 16 bytes |
| Construct the call key | 4 allocations, 800 bytes | 0 allocations |
| Construct the key for an `if` statement containing the call | 8
allocations, 1,864 bytes | 0 allocations |
| Construct two equal call keys and compare them | 2,745 instructions |
731 instructions |
| Construct one call key and hash it | 1,621 instructions | 589
instructions |
Codspeed reports a 3% improvement on most linter benchmarks.
## Considerations
Repeatedly comparing or hashing the same key now becomes slightly more
expensive because the implementation repeats the normalization in each
hash and equal call. However, the most common case is that we compare
nodes that are unequal, where the early return outweights building the
full normalized tree. If hashing becomes a problem, we should cache the
hash key (ideally only at call sites where it matters because this
increases instruction counts again).
## Test Plan
Structural comparison and formatter tests passed, and all five lint
fixtures produced identical diagnostics on both builds.
## Summary
Eleven lint checks collect `return` statements from the current
function. Their visitor now skips expressions as well as nested
functions and classes. The annotation fixer keeps its existing visitor
because it also needs to detect `yield` and `yield from` before
suggesting a return type.
For PLR0911 alone, 196 calls with all stable rules enabled, this cuts
instructions from 323,004 to 64,650 (−79.98%)
## Test Plan
The existing Ruff linter unit suite passes with no snapshot changes.
Enable `require-build-hashes` for the Python projects and inline scripts
so isolated builds require declared hashes for their build dependencies.
Pin the build backends and their dependencies with hashes.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
Summary
--
This PR updates our default Python version to 3.11, in line with 3.10
reaching [EOL] next month, and
updates our latest Python version to 3.15. I followed last year's PR
(#20725) to find everywhere
that needed to change. Fortunately there were not as many test changes
needed as last year's
deferred annotation issues.
I also updated the language classifier in our pyproject.toml.
[EOL]: https://devguide.python.org/versions/#supported-versions
Test Plan
--
Existing tests with updated snapshots. Most of the updates were to
snapshots for TC001, TC002, and
TC003, which now default to the lazy-import fix.
After updating the snapshots, I tried to make sure we still had coverage
for whatever changed. That
meant adding a few more mdtests for the TC rules, and one small mdtest
for RUF017 to restore
coverage for the `functools.reduce` fix.
## Summary
- Use the integer bit APIs stabilized in Rust 1.97 for packed node
indices, token capacity, rule iteration, and type-variable support sets.
- Use Cargo's warning policy in our native and WASM Clippy checks,
keeping warnings fatal without changing compiler flags and invalidating
the build cache. Update the contributing guides to match.
Preserves the Rust 1.97 MSRV. The warning policy change follows
https://github.com/astral-sh/uv/pull/22122.