## Summary
This PR splits `mismatched-type-name` into eight independently
configurable warnings for `NewType`, functional enums, `TypeVar`,
`ParamSpec`, `TypeVarTuple`, `namedtuple`/`NamedTuple`, `TypedDict`, and
`TypeAliasType`.
~~This PR is actually quite a bit less breaking for users than other
rule splits we've done over the last few days. Because
`mismatched-type-name` is no longer the canonical name for _any_ rule on
this branch, we're able to make it a deprecated alias for _all_ the new
rules, meaning that it will continue to work on the CLI, in ty
configuration files, and ty suppression comments (we'll just emit a
warning diagnostic telling users to upgrade their configuration
file/suppression comment).~~
~~A more specific rule takes precedence over the deprecated name if both
are given within the same configuration table. On the command line, the
last matching flag wins (e.g. `--error=mismatched-enum-name
--ignore=mismatched-type-name` will suppress diagnostics for mismatched
enum names).~~
(After review from @MichaReiser, I backed out these changes. It's now a
somewhat breaking change, because the rule is now marked as "removed"
rather than "deprecated".)
As well as splitting the rule into eight separate rules, it also cleans
up our error recovery for several of these type factories. For example,
in the following snippet, we fail to recognize `T` as a `TypeVar` due to
bad error recovery from the non-literal name. But we don't really need a
nonliteral name in this case to determine that it's a validly
constructed TypeVar. It's probably a mistake, but there's no _soundness_
issue here:
```py
from typing import TypeVar
def f(x: str):
# main emits an error: `invalid-typevar`
# this PR emits a warning: `mismatched-typevar-name`
T = TypeVar(x)
def g(x: T) -> T:
return x
reveal_type(g(42)) # main: Todo. This PR: `Literal[42]`.
```
## Motivation
The main motivation here is that having a namedtuple or enum with the
"wrong name", for example, just seems much less "obviously wrong" than
having a typevar or paramspec with the "wrong name". The [ecosystem
report](https://github.com/astral-sh/ruff/pull/29957#issuecomment-6603964612)
is evidence of this. namedtuples and enums define runtime classes that
can be used in runtime contexts and instantiated; these constructs
aren't _just_ for type checkers.
TypeVars/ParamSpecs/TypeVarTuples/NewTypes/TypedDicts are all _only_ for
type checkers and there's no plausible reason why you'd want to give the
type a different name to the variable it's assigned to.
`mismatched-namedtuple-name` and `mismatched-enum-name` are therefore
now disabled by default, matching the default severity we already
settled on for `mistmatched-sentinel-name` in
https://github.com/astral-sh/ruff/pull/27206. `mismatched-typevar-name`,
`mismatched-paramspec-name`, `mismatched-typevartuple-name`,
`mismatched-enum-name`, `mismatched-namedtuple-name` and
`mismatched-newtype-name` all remain enabled by default with `warning`
severity.
Summary
--
This is the first in a series of rule rename PRs. I started here because
the renames are fairly straightforward and are within a single linter
group. Most of these rules were missing some mention of docstrings in
their names or were missing or misusing something like `missing` or
`incorrect`. For example, many of the rules that start with `missing-`
actually enforce a single line somewhere, so they flag 2+ lines as well
as zero.
The first commit contains the changes in `codes.rs`, which I thought was
probably easiest to review
and comment on if any of the names need further refinement. The second
commit contains the rest of
the downstream changes.
| Code | Old name | New name |
|---------------------------------------------------|-----------------------------------------|---------------------------------------------------|
| [`D201`](https://docs.astral.sh/ruff/rules/D201/) |
`blank-line-before-function` | `blank-line-before-function-docstring` |
| [`D202`](https://docs.astral.sh/ruff/rules/D202/) |
`blank-line-after-function` | `blank-line-after-function-docstring` |
| [`D203`](https://docs.astral.sh/ruff/rules/D203/) |
`incorrect-blank-line-before-class` |
`incorrect-blank-line-before-class-docstring` |
| [`D204`](https://docs.astral.sh/ruff/rules/D204/) |
`incorrect-blank-line-after-class` |
`incorrect-blank-line-after-class-docstring` |
| [`D205`](https://docs.astral.sh/ruff/rules/D205/) |
`missing-blank-line-after-summary` |
`incorrect-blank-line-after-docstring-summary` |
| [`D207`](https://docs.astral.sh/ruff/rules/D207/) |
`under-indentation` | `under-indented-docstring-line` |
| [`D208`](https://docs.astral.sh/ruff/rules/D208/) | `over-indentation`
| `over-indented-docstring-line` |
| [`D209`](https://docs.astral.sh/ruff/rules/D209/) |
`new-line-after-last-paragraph` |
`missing-new-line-after-last-docstring-paragraph` |
| [`D210`](https://docs.astral.sh/ruff/rules/D210/) |
`surrounding-whitespace` | `whitespace-around-docstring-text` |
| [`D211`](https://docs.astral.sh/ruff/rules/D211/) |
`blank-line-before-class` | `blank-line-before-class-docstring` |
| [`D212`](https://docs.astral.sh/ruff/rules/D212/) |
`multi-line-summary-first-line` |
`multiline-docstring-summary-on-separate-line` |
| [`D213`](https://docs.astral.sh/ruff/rules/D213/) |
`multi-line-summary-second-line` |
`multiline-docstring-summary-on-first-line` |
| [`D214`](https://docs.astral.sh/ruff/rules/D214/) |
`overindented-section` | `over-indented-docstring-section` |
| [`D215`](https://docs.astral.sh/ruff/rules/D215/) |
`overindented-section-underline` |
`over-indented-docstring-section-underline` |
| [`D300`](https://docs.astral.sh/ruff/rules/D300/) |
`triple-single-quotes` | `unconventional-docstring-quotes` |
| [`D408`](https://docs.astral.sh/ruff/rules/D408/) |
`missing-section-underline-after-name` |
`blank-line-before-docstring-section-underline` |
| [`D410`](https://docs.astral.sh/ruff/rules/D410/) |
`no-blank-line-after-section` |
`missing-blank-line-after-docstring-section` |
| [`D411`](https://docs.astral.sh/ruff/rules/D411/) |
`no-blank-line-before-section` |
`missing-blank-line-before-docstring-section` |
| [`D413`](https://docs.astral.sh/ruff/rules/D413/) |
`missing-blank-line-after-last-section` |
`missing-blank-line-after-last-docstring-section` |
| [`D417`](https://docs.astral.sh/ruff/rules/D417/) |
`undocumented-param` | `undocumented-parameter` |
## Summary
Use a shared ty override for Ruff’s standalone scripts now that PEP 723
scripts can inherit project settings. Fix the diagnostics in the two PR
automation scripts so they can use the same checks.
## Notes to reviewers
All 28 standalone scripts passed ty with locked dependencies, and the
changed-file hooks and mocked workflow checks passed.
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
## Summary
Today, `single-overload` defaults to severity error. I suggest we
downgrade the rule to warning. While this is most likely unintentional
and something the typing spec explicitly calls out type checker should
report. I don't think it should be an error:
* There's no runtime error, it does not lead to downstream errors (e.g.
because a value stores now a string instead of an int).
* It can be used to expose a narrower API than what the implementation
supports. TypeScript's compiler does not warn about a single-overlaod.
And the following is a valid use case in TypeScript, that should also
work in Python.
The use case is from VueJS:
```ts
// This component declares only the prop "foo".
useModel(props, "foo") // OK
useModel(props, "bar") // Type error: there is no "bar" prop
```
The `useModel` overload requires name (the second argument) to match
`Literal["foo"]`. So the second call is disallowed.
However, the `name` argument of the `useModel` implementation is
intentionally annotated as `name: string`, so that the implementation
can show a debug-only warning that the prop `bar` is disallowed, without
TypeScript raising any narrowing or dead code warnings.
This is certainly an advanced feature. But it suggests that there are
valid use cases for having a single overload, and that disqualifies it
from being a correctness rule.
Changing the severity doesn't change whether ty fails on CI, unless the
user explicitly opted in to existing with 0 when there are only warning
diagnostics.
## Test Plan
Verified the default severity in the CLI and ran the overload and typing
conformance tests.
Summary
--
This PR extends our infrastructure for redirecting rule codes to
redirect rule names too, in
preparation for the many renames we have planned before stabilizing
human-readable names. As a proof
of concept, I also renamed
[`missing-return-type-undocumented-public-function` (`ANN201`)], which
seemed like a very clear rename target, having been accidentally renamed
in #3283.
The first commit contains the redirect changes, and the second commit
contains the `ANN201` rename,
along with the tests it allows us to exercise.
[`missing-return-type-undocumented-public-function` (`ANN201`)]:
https://docs.astral.sh/ruff/rules/missing-return-type-undocumented-public-function/
Test Plan
--
New CLI tests
## Summary
Users can now suppress an attribute error that affects only some union
members and still catch names that are missing from every member. For
example, both lines below previously reported `unresolved-attribute`:
```python
def clean(value: str | int) -> str:
value.stirp() # missing-attribute: neither str nor int has stirp
return value.strip() # missing-union-attribute: str has strip, int does not
```
The motivation is that a project adopting ty may want `value.strip()` on
`str | None` to only warn while a typo remains an error-severity
diagnostic.
Both rules default to errors. The same distinction applies to optional
values such as `str | None` and to attribute deletion. Attributes
defined only conditionally on a class continue to use
`possibly-missing-attribute`.
### Comparison to other type checkers
Pyright distinguishes between `reportOptionalMemberAccess` (which is
specifically for unions with `None`) and `reportAttributeAccessIssue`,
which is for all other unions. mypy, meanwhile, distinguishes between
`union-attr` (which is an error code for all attribute-access errors on
union types) and `attr-defined` (which is an error code for all
attribute-access errors on non-union types). So this new split doesn't
_exactly_ align with either precedent -- but I _do_ think it's probably
the most useful split we can do, and it's the one that makes the most
sense _a priori_ in my opinion.
### Compatibility
`unresolved-attribute` remains an alias for `missing-attribute` in
configuration and inline ignores. To preserve a setting such as
`unresolved-attribute = "warn"` for both cases, also set
`missing-union-attribute = "warn"`. On an access that fails for only
some union members, replace `# ty: ignore[unresolved-attribute]` with `#
ty: ignore[missing-union-attribute]`. Tools that consume diagnostic
codes should recognize both new names.
Fixesastral-sh/ty#2839 and fixesastral-sh/ty#2471.
## Test Plan
A comparison with ty 0.0.96 on typeshed-stats, docstring-adder,
sphinx-lint, and typing_extensions found the same messages, locations,
and severities. Eight diagnostics received `missing-union-attribute`
across those four projects (two in sphinx-lint and six in
typing_extensions).
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
Update Ruff's locked `ecosystem-analyzer` revision to pick up the
`schema_salad` setup fix. Align the setup script's `mypy-primer` lock
with the analyzer. [ecosystem-analyzer
#246](https://github.com/astral-sh/ecosystem-analyzer/pull/246) ·
[mypy_primer #275](https://github.com/hauntsaninja/mypy_primer/pull/275)
The lock checks passed, and the setup script installed `schema_salad` at
`0fd6c25fb0` with the dependency cutoff from the failing ecosystem run.
`uv pip check` passed.
When a ty fuzz run passes after an earlier failure on the same PR,
#30047 will update the failure comment to show that the latest run
passed. A PR with no earlier report should stay quiet. CI on #30047
calls the reusable report publisher from `main`, so the new input must
land on `main` before that PR can pass it.
Add an optional `update-existing-only` input to the publisher. When
enabled, the publisher updates the bot’s existing comment for that
report. If there is no stored comment ID or the comment has been
deleted, it skips publication. Callers that omit the input can still
create a comment.
## Summary
The CLI benchmarks now let each checker choose how many workers to use.
On the 16-core benchmark Mac, mypy goes from 16 external workers to
zero, and Pyright checks in one process. ty, Pyrefly, and Zuban still
select 16 threads.
`--single-threaded` still limits checker parallelism. The editor
benchmarks keep their existing settings.
The [18-project default-concurrency
results](https://github.com/astral-sh/ty/pull/4930) were collected with
the file-selection audit.
## Summary
We added Django, SymPy, the OpenAI Python SDK, and pytest to the ty
benchmarks, bringing the suite from 14 projects to 18. Each project has
a CLI benchmark and an editor benchmark that changes a return type. For
Django, we change `force_str` to return `int` and wait for a diagnostic
in `django.contrib.auth.hashers`.
## Summary
On Home Assistant, giving Pyright a list of every input file made a cold
check 4.4× slower than giving it the project directory. Pyright checked
the same files and reported the same diagnostics in both runs. Each
checker now discovers files from the project’s configured directories.
We also enable checking inside unannotated functions, point imports at
the project checkout, and give each CLI the available CPU count. On the
benchmark Mac, mypy gets 16 workers in place of the two configured by
Home Assistant. The language servers use the same physical path for the
workspace and its files, including when `/tmp` is a symlink.
The [published measurements](https://github.com/astral-sh/ty/pull/4889)
used the [file-selection
audit](https://github.com/astral-sh/ruff/blob/44595abd2280f6f9cfd6408179baea436c52d792/scripts/ty_benchmark/src/benchmark/targets.py).
On NumPy the audit selected 205 files. With native discovery, mypy finds
207 and Pyright reports 359.
---------
Co-authored-by: Charlie Marsh <charliemarsh@openai.com>
## Summary
We added Zuban 0.10.2 to the CLI and editor benchmarks. It finished 13
of the 14 CLI benchmarks. We stopped the first NumPy warmup after ten
minutes. Its language server also crashed on the edits to discord.py,
PyTorch, and NumPy.
Zuban runs by default in both suites. Its language server reads settings
from `pyproject.toml`, so the runner writes the CLI settings there for
each editor benchmark and restores the original file afterward.
## Summary
We updated eight of the 14 project revisions in the ty benchmarks. The
checker versions and dependency cutoff stay fixed so we can measure how
checking a newer version of each project affects performance.
(pandas removed its top-level `typings/` directory, so its benchmark now
targets `pandas/`.)
## Summary
The ty benchmarks now cover 14 projects. We added HTTPX, NumPy, FastAPI,
SQLAlchemy, and Rich. Each has an editor benchmark that changes a return
type and measures how long it takes a diagnostic to appear in a file
that imports it.
We also updated the runner to Python 3.15, Pyrefly to 1.4.0, and mypy to
2.4.0.
## Summary
Changing the lexer's keyword set now takes one edit to the Rust list.
The compiler finds a collision-free multiplier and builds the 64-slot
table, so we can remove the Python generator and its lockfile.
A two-byte keyword like `if` packs to `0x6669_0000_0000_0000`. Only the
low two bytes of the multiplier affect its slot. We place keywords from
shortest to longest, choosing each new byte of the multiplier without
disturbing the shorter keywords. The lexer still selects a slot with one
multiply and shift, then compares the full word.
## Summary
We scan ASCII identifier continuations in sixteen-byte batches using a
byte lookup table, avoiding per-character decoding. Keyword lookup
filters by length and initial byte, then uses a compact hash table with
a full-word comparison to reject collisions. Non-ASCII suffixes retain
the existing Unicode path.
Report comments include a redundant ZIP download link after the hosted
HTML links. Show the ZIP link only when the HTML report is unavailable,
including when its upload fails.
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
ty ecosystem comments currently require downloading a ZIP to view the
full reports. Add an optional `upload-html` mode to the shared report
publisher and enable it for ty, linking directly to the diagnostic diff
and timing results while retaining the downloadable artifact.
The publisher runs from `main`, uses STS for GitHub credentials, and
obtains temporary AWS upload credentials through GitHub OIDC when
hosting is enabled. Each caller supplies filenames and content types,
with optional link labels. The publisher reads only those files from a
bounded archive without extracting or executing them during publication,
and stores reports under separate paths for each PR, report, run, and
attempt. If an upload fails, the publisher still attempts to post the
summary and, when available, the artifact link.
The `automations` environment requires `REPORTS_BUCKET`, `REPORTS_URL`,
`REPORTS_ROLE_ARN`, and `REPORTS_REGION` from the report hosting
infrastructure.
---------
Co-authored-by: Zanie Blue <2586601+zanieb@users.noreply.github.com>
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
The report footer renders the commit SHA as code, so it is not
clickable. Link the commit, label it as the revision the results
describe, and separate the footer from the report.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
Add `publish-pull-request-report.yml` as a shared publisher for
automation reports, using STS for GitHub writes and the KV actions for
comment IDs. Store IDs in `pull-request` scope so later runs can update
the same report.
Look up each stored ID directly and verify the bot, PR, marker, and run
metadata. When a report is published, missing or expired keys create a
new comment without scanning existing comments; deleted comments are
replaced. Malformed IDs, IDs belonging to another comment, and KV errors
fail the job. The publisher verifies the source run and current PR head
before writing; report workflows enable it separately.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
Move reviewer selection from `astral-bot` into a serialized Actions
worker triggered by Ruff's dispatch policy. The worker uses the
repository's reviewer pools and requests reviews as
`astral-automations-bot[bot]` using STS. Store round-robin credits in
`workflow` scope, with read and write access restricted to
`pull-request-reviewers.yml` on `main` in the `automations` environment.
When no reviewer state is stored, credits start at zero.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
Enable `require-build-hashes` for the Python projects and inline scripts
so isolated builds require declared hashes for their build dependencies.
Pin the build backends and their dependencies with hashes.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
## Summary
Use thin LTO for the instrumented release executable in the PGO build,
while keeping fat LTO for the final executable. Pass the override
through `cargo rustc` so dependency compilation and the workspace's
release profile are unchanged. Debug builds are unchanged.
This reduces the estimated PGO pipeline time by 54% in a Linux x86-64
comparison, with essentially unchanged final binary size and no
consistent runtime regression across the holdout projects used for the
original PGO work in #27570.
## Build results
| Measurement | Fat training | Thin training |
| --- | ---: | ---: |
| Instrumented build | 487.86 s | 85.50 s |
| Final fat-LTO build, median | 247.75 s | 250.43 s |
| Estimated complete PGO pipeline | 738.02 s | 338.26 s |
| Final stripped executable, median | 22.719 MiB | 22.724 MiB |
Instrumented builds were measured once per mode. Final build times and
sizes are medians of three independently collected profiles per mode.
Pipeline estimates add the instrumented build, median training and
profile merge, and median final build; corpus downloads are excluded.
These are clean local build measurements, not CI timing predictions.
## Runtime results
We trained on the canonical 618-file corpus and evaluated linting and
formatting on Django, pandas, scikit-learn, SciPy, and SymPy, whose
repositories are disjoint from the training corpus. The holdout pins
come from the contemporaneous Windows follow-up in #27573, since the
initial Linux PR did not record exact holdout revisions.
Each case used three independent profile pairs and 12 interleaved
fat/thin timing pairs per profile, for 720 timed runs in total. Runs
used eight pinned CPUs and eight threads, with balanced execution order
and warm caches. Diagnostics and formatting diffs were checked
separately from the quiet timed commands and matched across all six
final binaries.
Positive changes below mean thin training was slower. Intervals are
per-workload hierarchical bootstrap 95% confidence intervals, resampling
profile pairs and then paired timing observations; they are not adjusted
for multiple comparisons.
| Holdout | CPU change (95% CI) | Wall change (95% CI) |
| --- | ---: | ---: |
| Django lint | -0.21% [-0.77%, +0.49%] | +0.10% [-1.41%, +1.29%] |
| Django format | -0.10% [-1.18%, +0.72%] | -0.27% [-1.37%, +1.45%] |
| pandas lint | -0.01% [-0.45%, +0.39%] | -0.67% [-2.49%, +1.26%] |
| pandas format | -0.26% [-1.10%, +0.67%] | -0.84% [-2.81%, +2.12%] |
| scikit-learn lint | +0.02% [-0.62%, +1.12%] | -0.02% [-1.11%, +1.86%]
|
| scikit-learn format | -0.52% [-2.14%, +0.62%] | -0.79% [-2.08%,
+1.03%] |
| SciPy lint | -0.21% [-0.70%, +0.43%] | -0.64% [-4.59%, +1.52%] |
| SciPy format | -0.52% [-1.03%, +0.13%] | -2.47% [-6.20%, +1.61%] |
| SymPy lint | +0.01% [-0.30%, +0.27%] | -0.77% [-3.23%, +0.88%] |
| SymPy format | -0.18% [-1.89%, +1.84%] | -0.58% [-3.58%, +3.30%] |
A longer SymPy formatting repeat with the same three profile pairs and
48 timing pairs per profile measured -0.18% CPU [-1.96%, +1.66%] and
-0.28% wall time [-2.19%, +1.58%]. Individual profiles varied in both
directions, so the near-flat average should not be interpreted as
identical performance for every trained binary. The pandas formatting
wall-time interval remains inconclusive at a 2% slowdown threshold.
All six profiles had matching function/CFG identities and counter
layouts, with no missing-profile or profile-mismatch compiler
diagnostics. Their counter values and `.profdata` bytes were not
identical.
Measurements used the Rust/Cargo source at
`7c6f8fc8fc5655dfa47e125b05453007b9e7a6c7`, the standard release
codegen-unit settings, and Ohm Rust 1.99.0-dev (`08bbd755a`, LLVM
23.1.1) with experimental defaults disabled. The benchmark was not rerun
on this PR's newer base revision. This is an exploratory Linux x86-64
study with three profiles per mode, not a guarantee of zero runtime cost
or equivalent results on macOS, Windows, or ARM.
Follow-up to #29151: upgrade the benchmark dependency from hyperfine
2.0.0-alpha.2 to 2.0.0.
Use `--env RAYON_NUM_THREADS=1` instead of an `env` command prefix for
the single-threaded formatter benchmarks. This avoids measuring the
overhead of `env` itself (~5 ms).
## Summary
Upgrade hyperfine to 2.0.0-alpha.2, which now includes support for
memory measurements (see output below).
Tested:
<details>
<summary>Formatter benchmark output</summary>
```text
Benchmark 1: ./target/release/ruff format ./fixture (2 runs)
mean ± σ min … max
Wall Time 12.8 ms ± 0.2 ms 12.7 ms … 12.9 ms
Memory 16.6 MiB ± 0.1 MiB 16.5 MiB … 16.6 MiB
Benchmark 2: black --no-cache ./fixture --safe (2 runs)
mean ± σ min … max
Wall Time 207.1 ms ± 4.8 ms 203.7 ms … 210.5 ms +1517.9% (16.2x slower)
Memory 37.3 MiB ± 0.1 MiB 37.2 MiB … 37.4 MiB +125.2% (2.3x more)
Benchmark 3: black --no-cache ./fixture --fast (2 runs)
mean ± σ min … max
Wall Time 226.6 ms ± 7.4 ms 221.4 ms … 231.9 ms +1670.7% (17.7x slower)
Memory 37.4 MiB ± 0.1 MiB 37.3 MiB … 37.5 MiB +126.2% (2.3x more)
Benchmark 4: black --no-cache ./fixture --safe --preview (2 runs)
mean ± σ min … max
Wall Time 250.1 ms ± 41.8 ms 220.6 ms … 279.6 ms +1854.3% (19.5x slower)
Memory 37.4 MiB ± 0.3 MiB 37.2 MiB … 37.6 MiB +126.0% (2.3x more)
Benchmark 5: black --no-cache ./fixture --fast --preview (2 runs)
mean ± σ min … max
Wall Time 186.2 ms ± 2.5 ms 184.5 ms … 188.0 ms +1355.1% (14.6x slower)
Memory 37.3 MiB ± 0.0 MiB 37.3 MiB … 37.3 MiB +125.2% (2.3x more)
Benchmark 6: autopep8 ./fixture --recursive --in-place (2 runs)
mean ± σ min … max
Wall Time 41.6 ms ± 0.7 ms 41.1 ms … 42.1 ms +225.2% (3.3x slower)
Memory 26.3 MiB ± 0.0 MiB 26.3 MiB … 26.3 MiB +58.8% (1.6x more)
Benchmark 7: yapf ./fixture --parallel --recursive --in-place (2 runs)
mean ± σ min … max
Wall Time 142.1 ms ± 4.8 ms 138.7 ms … 145.5 ms +1010.6% (11.1x slower)
Memory 29.1 MiB ± 0.3 MiB 28.9 MiB … 29.4 MiB +76.0% (1.8x more)
Benchmark 1: env RAYON_NUM_THREADS=1 ./target/release/ruff format ./fixture (2 runs)
mean ± σ min … max
Wall Time 12.7 ms ± 2.5 ms 10.9 ms … 14.4 ms
Memory 14.6 MiB ± 0.1 MiB 14.6 MiB … 14.7 MiB
Benchmark 2: black --no-cache ./fixture --workers=1 --safe (2 runs)
mean ± σ min … max
Wall Time 95.2 ms ± 3.2 ms 92.9 ms … 97.4 ms +650.6% (7.5x slower)
Memory 38.4 MiB ± 0.0 MiB 38.4 MiB … 38.4 MiB +162.6% (2.6x more)
Benchmark 3: black --no-cache ./fixture --workers=1 --fast (2 runs)
mean ± σ min … max
Wall Time 89.6 ms ± 2.8 ms 87.6 ms … 91.6 ms +606.5% (7.1x slower)
Memory 38.4 MiB ± 0.0 MiB 38.4 MiB … 38.4 MiB +162.7% (2.6x more)
Benchmark 4: black --no-cache ./fixture --workers=1 --safe --preview (2 runs)
mean ± σ min … max
Wall Time 94.7 ms ± 0.6 ms 94.3 ms … 95.2 ms +646.9% (7.5x slower)
Memory 38.5 MiB ± 0.1 MiB 38.5 MiB … 38.6 MiB +163.5% (2.6x more)
Benchmark 5: black --no-cache ./fixture --workers=1 --fast --preview (2 runs)
mean ± σ min … max
Wall Time 92.7 ms ± 0.6 ms 92.3 ms … 93.2 ms +631.4% (7.3x slower)
Memory 38.4 MiB ± 0.1 MiB 38.3 MiB … 38.4 MiB +162.5% (2.6x more)
Benchmark 6: autopep8 ./fixture --in-place --recursive --jobs=1 (2 runs)
mean ± σ min … max
Wall Time 43.5 ms ± 3.7 ms 40.9 ms … 46.1 ms +242.7% (3.4x slower)
Memory 26.5 MiB ± 0.1 MiB 26.4 MiB … 26.6 MiB +81.2% (1.8x more)
Benchmark 7: yapf ./fixture --recursive --in-place (2 runs)
mean ± σ min … max
Wall Time 56.8 ms ± 1.2 ms 55.9 ms … 57.6 ms +347.8% (4.5x slower)
Memory 27.3 MiB ± 0.1 MiB 27.2 MiB … 27.4 MiB +86.5% (1.9x more)
Benchmark 1: ./target/release/ruff format ./fixture --check (2 runs)
mean ± σ min … max
Wall Time 10.2 ms ± 0.3 ms 9.9 ms … 10.4 ms
Memory 16.9 MiB ± 0.1 MiB 16.8 MiB … 17.0 MiB
Benchmark 2: black --no-cache ./fixture --check --safe (2 runs)
mean ± σ min … max
Wall Time 193.8 ms ± 6.2 ms 189.5 ms … 198.2 ms +1806.9% (19.1x slower)
Memory 37.3 MiB ± 0.1 MiB 37.2 MiB … 37.4 MiB +120.8% (2.2x more)
Benchmark 3: black --no-cache ./fixture --check --fast (2 runs)
mean ± σ min … max
Wall Time 188.2 ms ± 1.9 ms 186.8 ms … 189.6 ms +1751.4% (18.5x slower)
Memory 37.3 MiB ± 0.1 MiB 37.3 MiB … 37.4 MiB +121.1% (2.2x more)
Benchmark 4: black --no-cache ./fixture --check --safe --preview (2 runs)
mean ± σ min … max
Wall Time 202.1 ms ± 6.3 ms 197.7 ms … 206.6 ms +1888.5% (19.9x slower)
Memory 37.2 MiB ± 0.0 MiB 37.2 MiB … 37.2 MiB +120.5% (2.2x more)
Benchmark 5: black --no-cache ./fixture --check --fast --preview (2 runs)
mean ± σ min … max
Wall Time 197.0 ms ± 20.1 ms 182.8 ms … 211.2 ms +1837.8% (19.4x slower)
Memory 37.2 MiB ± 0.0 MiB 37.2 MiB … 37.2 MiB +120.5% (2.2x more)
Benchmark 6: autopep8 ./fixture --recursive --diff (2 runs)
mean ± σ min … max
Wall Time 38.5 ms ± 0.1 ms 38.4 ms … 38.6 ms +278.4% (3.8x slower)
Memory 26.3 MiB ± 0.0 MiB 26.3 MiB … 26.4 MiB +56.0% (1.6x more)
Benchmark 7: yapf ./fixture --parallel --recursive --quiet (2 runs)
mean ± σ min … max
Wall Time 144.8 ms ± 7.4 ms 139.6 ms … 150.1 ms +1324.8% (14.2x slower)
Memory 29.0 MiB ± 0.2 MiB 28.9 MiB … 29.1 MiB +71.6% (1.7x more)
```
</details>
<details>
<summary>ty_benchmark (multi-threaded, then single-threaded)</summary>
```text
jinja
-----
Benchmark 1: ty (2 runs)
mean ± σ min … max
Wall Time 73.0 ms ± 5.6 ms 69.0 ms … 76.9 ms
Memory 100.1 MiB ± 0.1 MiB 100.0 MiB … 100.2 MiB
Warning: Ignoring non-zero exit code.
Benchmark 2: Pyrefly (2 runs)
mean ± σ min … max
Wall Time 118.3 ms ± 6.0 ms 114.0 ms … 122.5 ms +62.1% (1.6x slower)
Memory 130.9 MiB ± 0.2 MiB 130.8 MiB … 131.1 MiB +30.8% (1.3x more)
Warning: Ignoring non-zero exit code.
Benchmark 3: mypy (2 runs)
mean ± σ min … max
Wall Time 802.9 ms ± 8.0 ms 797.2 ms … 808.6 ms +1000.2% (11.0x slower)
Memory 143.4 MiB ± 0.1 MiB 143.3 MiB … 143.5 MiB +43.2% (1.4x more)
Warning: Ignoring non-zero exit code.
Benchmark 4: mypy (warm) (2 runs)
mean ± σ min … max
Wall Time 208.8 ms ± 5.6 ms 204.9 ms … 212.7 ms +186.1% (2.9x slower)
Memory 51.9 MiB ± 0.2 MiB 51.8 MiB … 52.0 MiB -48.2% (1.9x less)
Warning: Ignoring non-zero exit code.
Benchmark 5: Pyright (2 runs)
mean ± σ min … max
Wall Time 1198.8 ms ± 12.0 ms 1190.3 ms … 1207.3 ms +1542.8% (16.4x slower)
Memory 251.6 MiB ± 1.3 MiB 250.7 MiB … 252.5 MiB +151.3% (2.5x more)
Warning: Ignoring non-zero exit code.
jinja
-----
Benchmark 1: ty (2 runs)
mean ± σ min … max
Wall Time 226.1 ms ± 14.0 ms 216.2 ms … 236.0 ms
Memory 84.9 MiB ± 0.2 MiB 84.8 MiB … 85.0 MiB
Warning: Ignoring non-zero exit code.
Benchmark 2: Pyrefly (2 runs)
mean ± σ min … max
Wall Time 210.4 ms ± 6.9 ms 205.5 ms … 215.3 ms -7.0%
Memory 92.3 MiB ± 0.1 MiB 92.2 MiB … 92.4 MiB +8.8%
Warning: Ignoring non-zero exit code.
Benchmark 3: mypy (2 runs)
mean ± σ min … max
Wall Time 896.4 ms ± 88.8 ms 833.7 ms … 959.2 ms +296.4% (4.0x slower)
Memory 144.3 MiB ± 0.6 MiB 143.9 MiB … 144.7 MiB +70.0% (1.7x more)
Warning: Ignoring non-zero exit code.
Benchmark 4: mypy (warm) (2 runs)
mean ± σ min … max
Wall Time 213.0 ms ± 3.7 ms 210.4 ms … 215.6 ms -5.8%
Memory 51.7 MiB ± 0.1 MiB 51.6 MiB … 51.8 MiB -39.1% (1.6x less)
Warning: Ignoring non-zero exit code.
Benchmark 5: Pyright (2 runs)
mean ± σ min … max
Wall Time 981.9 ms ± 13.0 ms 972.6 ms … 991.1 ms +334.2% (4.3x slower)
Memory 315.4 MiB ± 5.9 MiB 311.2 MiB … 319.5 MiB +271.6% (3.7x more)
Warning: Ignoring non-zero exit code.
```
</details>
I also tested most of the other hyperfine invocations mentioned
somewhere in this repo.
## Summary
Enable `truthiness-test-of-none-union` in the standalone scripts' inline
configuration and in the `py-fuzzer` and `ty_benchmark` projects,
following #28889.