Files
Brent Westbrook d55ed14faa Handle renamed rules in ecosystem reports (#30468)
Summary
--

This PR updates our ecosystem report to run unknown rule names through
[renamed-rule (RUF202)](https://docs.astral.sh/ruff/rules/renamed-rule/)
before normalizing them back to codes. In my first rule rename PR, the
stable ecosystem report failed on typeshed because its configuration
uses the human-readable name of one of the renamed rules. Our current
normalization process relies on `ruff rule --all --output-format=json`,
which no longer contains the old rule name, causing the human-readable
name not to be mapped back to a code, which fails on stable:

```
ruff failed
  Cause: Invalid selector `blank-line-after-function` in `ignore` from `/home/runner/work/_temp/ruff-ecosystem-checkouts/python:typeshed/pyproject.toml`. Selecting rules by name requires preview mode
```


https://github.com/astral-sh/ruff/actions/runs/43917728573/job/133476621427

Instead of running `RUF202` on every config file as I initially
pictured, we synthesize a config file containing only any unknown rule
names and send that to Ruff via stdin. This both avoids the performance
cost of creating a subprocess for every project/config and retains some
ecosystem coverage for redirected prefixes and codes.

Test Plan
--

CI on #30465, which I'll stack on this PR
2026-11-18 09:53:38 -05:00
..

ruff-ecosystem

Compare lint and format results for two different ruff versions (e.g. main and a PR) on real world projects.

Installation

From the Ruff project root, install with uv:

uv tool install -e ./python/ruff-ecosystem

Usage

ruff-ecosystem <check | format> <baseline executable> <comparison executable>

Note executable paths may be absolute, relative to the current working directory, or will be looked up in the current Python environment and PATH.

Run ruff check ecosystem checks comparing your debug build to your system Ruff:

ruff-ecosystem check ruff "./target/debug/ruff"

Run ruff format ecosystem checks comparing your debug build to your system Ruff:

ruff-ecosystem format ruff "./target/debug/ruff"

Run ruff format ecosystem checks comparing with changes to code that is already formatted:

ruff-ecosystem format ruff "./target/debug/ruff" --format-comparison ruff-then-ruff

Run ruff format ecosystem checks comparing with the Black formatter:

ruff-ecosystem format black ruff -v --cache python/checkouts --format-comparison black-and-ruff

The default output format is markdown, which includes nice summaries of the changes. You can use --output-format json to display the raw data — this is particularly useful when making changes to the ecosystem checks.

Add --html-output report.html to also write a searchable HTML report containing all changes.

Development

When developing, it can be useful to set the --pdb flag to drop into a debugger on failure:

ruff-ecosystem check ruff "./target/debug/ruff" --pdb

You can also provide a path to cache checkouts to speed up repeated runs:

ruff-ecosystem check ruff "./target/debug/ruff" --cache ./repos