## Summary
Today, `ruff_python_ast` has two implementations of
`parenthesized_range`, but only tests use the source-tokenizing one.
This PR removes the unused implementation and moves the remaining
distinct cases into the token-based implementation's tests.
Testing: All AST integration tests pass, and retired instruction counts
show no regression in Ruff.
## Summary
Extracting PEP 723 metadata can scan the same suffix for every unclosed
opening delimiter, leading to quadratic work. Resume at the end of a
failed scan and accept already decoded Python source to avoid redundant
UTF-8 validation.
Testing: Parser tests pass and Ruff and ty compile.
## Summary
Use a shared ty override for Ruff’s standalone scripts now that PEP 723
scripts can inherit project settings. Fix the diagnostics in the two PR
automation scripts so they can use the same checks.
## Notes to reviewers
All 28 standalone scripts passed ty with locked dependencies, and the
changed-file hooks and mocked workflow checks passed.
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Summary
--
There were several new cargo and clippy lints here, so I tried to break
up the changes into several
commits:
- deleting some unused workspace dependencies, these looked kind of
intentional as they also had
some shear ignores, but it seems okay to delete those too?
- allowing the [redundant_homepage] lint. We could delete a lot of
these, but the ones in the `ruff`
and `ty` crates seem necessary for maturin, as I noted in a comment.
- fixing the [missing_lints_inheritance] lint. It seems that several of
our crates don't inherit our
workspace lints. For now I just included the `allow` for
`redundant_homepage`, but we could follow
up by inheriting the full table and fixing all the additional lints that
produces. (This commit is
actually combined with the previous one)
- allowing cargo's new [unused_dependencies] lint, which seems redundant
with `shear` and had some
false positives on our benchmark dependencies. Codex tried to work
around this with some
`cfg`-gated imports in the benchmark crate, but ignoring this seemed
better for now, at least
while we're still using `shear`.
- running `clippy --fix` on the remaining lint errors, with one manual
intervention where Clippy
dropped some of our `cfg` handling. These new diagnostics were mostly
around returning boolean
expressions.
- renaming `NodeIndex::_NONE` to avoid [used_underscore_items]
There are also some new APIs in 1.98 that we can use now. I'll follow up
with those separately.
[redundant_homepage]:
https://doc.rust-lang.org/cargo/reference/lints.html#redundant_homepage
[missing_lints_inheritance]:
https://doc.rust-lang.org/cargo/reference/lints.html#missing_lints_inheritance
[used_underscore_items]:
https://clippy.rust-lang.org/master/#used_underscore_items
[unused_dependencies]:
https://doc.rust-lang.org/cargo/reference/lints.html#unused_dependencies
## Summary
Previously, Ruff parsed the leading words of each docstring line as a
possible section name. Both `Returns:` and `Returns: the new value.`
matched `Returns`, so Ruff extracted the name again and checked the
remaining text to reject the second line.
Ruff now skips the indentation already measured by `leading_space`,
trims trailing whitespace, and strips one optional trailing colon. It
passes the rest of the line to the existing section-name parser.
`Returns:` yields `Returns`. The inline description in `Returns: the new
value.` makes the match fail immediately. On a match,
`suspected_as_section` returns both the section kind and the borrowed
name so the caller can record its source range.
Each measurement counts instructions for one uncached lint of the NumPy
`globals.py` benchmark after initialization.
| Rules | Before | After | Reduction |
| --- | ---: | ---: | ---: |
| All stable | 819,973 | 767,658 | 6.4% |
| All with preview | 951,580 | 899,315 | 5.5% |
## Test Plan
Ran the linter and Python AST tests and compared diagnostics and edits
across the benchmark files and docstring conventions.
## Summary
Ruff uses `ComparableExpr` to compare Python syntax while ignoring
details such as source ranges and expression contexts. Previously,
`ComparableExpr::from(&expr)` recursively built a second tree containing
the fields used for comparison. For `factory(item, *items, limit=2,
**options)`, it boxed the comparable callee and the operand of `*items`,
and collected the positional arguments and keywords into two vectors.
Equality and hashing then walked that tree. Statements, match patterns,
and modules built similar trees.
`ComparableExpr<'a>` now holds a single `ast::ExprRef<'a>`. The
statement, pattern, and module keys likewise borrow their AST nodes.
Constructing a key takes constant time and allocates nothing. When
comparing two calls, equality checks the callees first. If one is
`factory` and the other is `other`, it can return before visiting any
arguments. If the callees match, it walks the original argument slices,
comparing each child as needed. Hashing walks the same structural
fields, and `Debug` displays them. Borrowing the expression also removes
the need to run `drop` after comparing.
The field helpers exhaustively destructure each node so that adding an
AST field requires deciding whether it participates in comparison.
| Workload | `main` | This PR |
| --- | ---: | ---: |
| Size of a `ComparableExpr` on x86-64 | 128 bytes | 16 bytes |
| Construct the call key | 4 allocations, 800 bytes | 0 allocations |
| Construct the key for an `if` statement containing the call | 8
allocations, 1,864 bytes | 0 allocations |
| Construct two equal call keys and compare them | 2,745 instructions |
731 instructions |
| Construct one call key and hash it | 1,621 instructions | 589
instructions |
Codspeed reports a 3% improvement on most linter benchmarks.
## Considerations
Repeatedly comparing or hashing the same key now becomes slightly more
expensive because the implementation repeats the normalization in each
hash and equal call. However, the most common case is that we compare
nodes that are unequal, where the early return outweights building the
full normalized tree. If hashing becomes a problem, we should cache the
hash key (ideally only at call sites where it matters because this
increases instruction counts again).
## Test Plan
Structural comparison and formatter tests passed, and all five lint
fixtures produced identical diagnostics on both builds.
## Summary
Eleven lint checks collect `return` statements from the current
function. Their visitor now skips expressions as well as nested
functions and classes. The annotation fixer keeps its existing visitor
because it also needs to detect `yield` and `yield from` before
suggesting a return type.
For PLR0911 alone, 196 calls with all stable rules enabled, this cuts
instructions from 323,004 to 64,650 (−79.98%)
## Test Plan
The existing Ruff linter unit suite passes with no snapshot changes.
Enable `require-build-hashes` for the Python projects and inline scripts
so isolated builds require declared hashes for their build dependencies.
Pin the build backends and their dependencies with hashes.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
Summary
--
This PR updates our default Python version to 3.11, in line with 3.10
reaching [EOL] next month, and
updates our latest Python version to 3.15. I followed last year's PR
(#20725) to find everywhere
that needed to change. Fortunately there were not as many test changes
needed as last year's
deferred annotation issues.
I also updated the language classifier in our pyproject.toml.
[EOL]: https://devguide.python.org/versions/#supported-versions
Test Plan
--
Existing tests with updated snapshots. Most of the updates were to
snapshots for TC001, TC002, and
TC003, which now default to the lazy-import fix.
After updating the snapshots, I tried to make sure we still had coverage
for whatever changed. That
meant adding a few more mdtests for the TC rules, and one small mdtest
for RUF017 to restore
coverage for the `functools.reduce` fix.
## Summary
- Use the integer bit APIs stabilized in Rust 1.97 for packed node
indices, token capacity, rule iteration, and type-variable support sets.
- Use Cargo's warning policy in our native and WASM Clippy checks,
keeping warnings fatal without changing compiler flags and invalidating
the build cache. Update the contributing guides to match.
Preserves the Rust 1.97 MSRV. The warning policy change follows
https://github.com/astral-sh/uv/pull/22122.
## Summary
We add accessors and iteration methods to `ExprCompare` and use them
throughout AST visitors, formatting, lint rules, and type inference. For
`a < b <= c`, `iter` yields `(a, Lt, b)` and `(b, LtE, c)`. `as_single`
returns a triple only when there is exactly one operator and two
operands, so callers can match an operator and both operands together
while rejecting comparison chains. Rules that only inspect operands use
the existing operand slice directly.
The `first_operand`, `first_operator`, and `second_operand` accessors
assert the parser's existing guarantee that a comparison has at least
one operator and two operands. Error recovery synthesizes missing
operands; manually constructed ASTs must uphold the same invariant.
## Summary
We reduce `Expr` from 64 to 56 bytes on 64-bit targets. The largest
variants determine the enum's size, so shrinking those variants saves
eight bytes in every `Expr` value, including expressions in the parsed
modules retained by ty.
The changes are:
- Store positional call arguments in `ThinVec`, reducing `Arguments` and
`ExprCall` by eight bytes.
- Store all comparison operands in one boxed slice. This removes the
separate left-operand allocation and reduces `ExprCompare` by eight
bytes while keeping operators in a boxed slice.
- Store dictionary-comprehension generators in a boxed slice, reducing
`ExprDictComp` by eight bytes.
- Store f-string and t-string elements in boxed slices, and represent a
single f-string directly as `FString`. This keeps single parts inline
and reduces `ExprFString` and `ExprTString` by eight bytes.
The parser, AST visitors, formatter, lint rules, and type inference use
the updated layouts. Size assertions cover the 56-byte enum and its
affected variants.
Delegate `Name` equality with borrowed strings to `CharStr`, including
Salsa lookups, so these callers use the compact string's comparison
implementation.
This connects Ruff's name comparisons to the short-string fast paths in
https://github.com/astral-sh/char_str/pull/40. Those fast paths require
a later `char_str` release; this change is compatible with the current
dependency.
In #28092 I fixed a bug where ty's redundant-cast removal fix could
create broken code if the original `cast()` call was multi-line. Doug
commented that this could perhaps be a general-purpose utility -- and it
turned out there were two existing similar Ruff rules with pre-existing
multi-line support that could benefit from such a shared utility.
This PR introduces a shared utility in `ruff_python_edits` for removing
a function call in favor of one of its arguments. This is the common
shape between redundant-cast removal, redundant `int()` removal, and
redundant `round()` removal.
The new utility is more robust than the previous code for any of the
three cases. Previously removing redundant `int`, `round`, and
`typing.cast` calls could discard parentheses needed for line
continuation or for token separation. For example, `int(1).to_bytes()`
could become `1.to_bytes()`, which is invalid.
The shared helper preserves optional argument parentheses and the
comments inside them, adds grouping where required, and avoids redundant
parentheses when the surrounding syntax already allows the expression.
Ruff and ty retain their own diagnostic eligibility and fix-safety
decisions.
## Test plan
- Extend RUF046 and RUF057 fixtures for split callees, assignment
expressions, integer attributes, adjacent keywords, and comments inside
optional parentheses. Existing snapshot cases retain their fix text and
safety classifications.
- Add compact helper tests in `ruff_python_edits` for conditional
callees, comprehension iterables, and dictionary expressions inside
f-strings.
- Existing ty cast mdtests cover positional and keyword arguments,
operator precedence, multiline values, and comment preservation.
Summary
--
This PR updates our ruff-pre-commit dependency to 0.16.5, which supports
the new category selectors,
and then enables `lint.preview` and the new default categories so that
we can dogfood them on our
code. There were 66 diagnostics initially, which I also fixed or
ignored:
```
28 FURB113 [ ] repeated-append
12 ASYNC240 [ ] blocking-path-method-in-async-function
6 RUF105 [*] noqa-comments
5 PLW1510 [ ] subprocess-run-without-check
3 PLW0108 [ ] unnecessary-lambda
2 EXE001 [ ] shebang-not-executable
2 RUF039 [*] unraw-re-pattern
1 B024 [ ] abstract-base-class-without-abstract-method
1 FURB142 [*] for-loop-set-mutations
1 PERF102 [ ] incorrect-dict-iterator
1 PLC0207 [*] missing-maxsplit-arg
1 PLW0127 [ ] self-assigning-variable
1 TRY004 [ ] type-check-without-type-error
1 PLW0120 [*] useless-else-on-loop
1 TRY201 [ ] verbose-raise
```
Of these, I ignored `repeated-append` and `blocking-path-method-in-async-function`. I think they are
still reasonable defaults, just less useful/important in our uses.
0.16.5 also includes https://github.com/astral-sh/ruff-pre-commit/pull/159, so I removed `markdown` from our main `ruff-format` hook, leaving it for the later mdtest formatting stage.
I also had to make a few changes related to config discovery in tests:
- move our cached ecosystem checkouts to `$RUNNER_TEMP` instead of inside of our checkout
- convert some CLI tests to use the `CliTest` helpers
- add `configurationPreference: editorOnly` to some notebook tests
In these cases, our own Ruff settings were being inherited by test code. This likely affects many other tests, but I only updated the obviously failing ones for now. I'm least confident about the notebook tests, but as far as I can tell from the history (https://github.com/astral-sh/ruff/pull/11944, https://github.com/astral-sh/ruff/pull/22810), the specific diagnostics are not the important parts being asserted.
Test Plan
--
CI on this PR
Removing a redundant `typing.cast` could discard parentheses needed for
multiline expressions, introducing syntax errors or silently changing
arithmetic results. Preserve the argument's existing parentheses and
enclosed comments, and add grouping when a multiline argument relies on
the call's parentheses for line continuation.
The shared parentheses lookup now checks for an actual closing token so
that constructing a fix for an unfinished call does not panic.
Fixes https://github.com/astral-sh/ty/issues/4399.
## Test plan
- Added mdtests for multiline conditional and arithmetic expressions,
including a line break before an operator that must not become a
separate statement.
- Covered reordered keyword arguments and comments inside argument
parentheses.
- Added unit coverage for unterminated casts with no final newline, with
and without a trailing comment.
## Summary
We now run Hawk 0.1.13 with Rust 1.98.0 in CI, following the pattern
used by uv. The configuration preserves intentional public APIs and
generated interfaces while checking all configured production binaries
and workspace non-production targets with `-D warnings`.
The accompanying cleanup narrows internal visibility, removes unused
benchmark classification state and accessors, and records intentional
public interfaces as audited Hawk expectations. The resulting
configuration reports zero Hawk findings across the configured targets.
## Summary
Skip checking scripts with invalid PEP 723 metadata blocks, that are,
blocks that ty can't parse successfully or configurations that don't
resolve successfully. Instead, emit one diagnostic that points to the
invalid metadata.
ty still initializes a `Program` for each of those scripts so that LSP
operations continue to work. But we skip those scripts in all code paths
where we perform semantic checks.
Fixes https://github.com/astral-sh/ty/issues/4181.
## Test plan
Added tests
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
Adds explicit 7d cooldowns to all of our PEP 723 scripts, as well as
similar cooldowns via `.npmrc`.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
NFC.
<!-- How was it tested? -->
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
## Summary
Preserve source ranges for PEP 723 script metadata by building a compact
source map during extraction and applying it when deserializing ranged
values.
Closes https://github.com/astral-sh/ty/issues/4180
## Summary
This PR reduces the size of `Expr` from 72 to 64 bytes by storing only
the start offset on `ExprCall` (64 to 56 bytes) and boxing the uncommon
implicitly concatenated string representation in `ExprStringLiteral` (64
to 48 bytes). Both changes are required to reduce the size of the
containing `Expr` enum.
`ExprCall` derives its end offset from its argument list; parser and
error-recovery paths assert that their ranges share the same end.
---------
Co-authored-by: Charlie Marsh <charliemarsh@openai.com>