Commit Graph
655 Commits
Author SHA1 Message Date
Alex Waygood d8c6e237e7 [ty] Split mismatched-type-name by type constructor (#29957)
## Summary

This PR splits `mismatched-type-name` into eight independently
configurable warnings for `NewType`, functional enums, `TypeVar`,
`ParamSpec`, `TypeVarTuple`, `namedtuple`/`NamedTuple`, `TypedDict`, and
`TypeAliasType`.

~~This PR is actually quite a bit less breaking for users than other
rule splits we've done over the last few days. Because
`mismatched-type-name` is no longer the canonical name for _any_ rule on
this branch, we're able to make it a deprecated alias for _all_ the new
rules, meaning that it will continue to work on the CLI, in ty
configuration files, and ty suppression comments (we'll just emit a
warning diagnostic telling users to upgrade their configuration
file/suppression comment).~~

~~A more specific rule takes precedence over the deprecated name if both
are given within the same configuration table. On the command line, the
last matching flag wins (e.g. `--error=mismatched-enum-name
--ignore=mismatched-type-name` will suppress diagnostics for mismatched
enum names).~~

(After review from @MichaReiser, I backed out these changes. It's now a
somewhat breaking change, because the rule is now marked as "removed"
rather than "deprecated".)

As well as splitting the rule into eight separate rules, it also cleans
up our error recovery for several of these type factories. For example,
in the following snippet, we fail to recognize `T` as a `TypeVar` due to
bad error recovery from the non-literal name. But we don't really need a
nonliteral name in this case to determine that it's a validly
constructed TypeVar. It's probably a mistake, but there's no _soundness_
issue here:

```py
from typing import TypeVar

def f(x: str):
    # main emits an error: `invalid-typevar`
    # this PR emits a warning: `mismatched-typevar-name`
    T = TypeVar(x)

    def g(x: T) -> T:
        return x
    
    reveal_type(g(42))  # main: Todo. This PR: `Literal[42]`.
```

## Motivation

The main motivation here is that having a namedtuple or enum with the
"wrong name", for example, just seems much less "obviously wrong" than
having a typevar or paramspec with the "wrong name". The [ecosystem
report](https://github.com/astral-sh/ruff/pull/29957#issuecomment-6603964612)
is evidence of this. namedtuples and enums define runtime classes that
can be used in runtime contexts and instantiated; these constructs
aren't _just_ for type checkers.
TypeVars/ParamSpecs/TypeVarTuples/NewTypes/TypedDicts are all _only_ for
type checkers and there's no plausible reason why you'd want to give the
type a different name to the variable it's assigned to.

`mismatched-namedtuple-name` and `mismatched-enum-name` are therefore
now disabled by default, matching the default severity we already
settled on for `mistmatched-sentinel-name` in
https://github.com/astral-sh/ruff/pull/27206. `mismatched-typevar-name`,
`mismatched-paramspec-name`, `mismatched-typevartuple-name`,
`mismatched-enum-name`, `mismatched-namedtuple-name` and
`mismatched-newtype-name` all remain enabled by default with `warning`
severity.
2026-11-20 12:41:39 +00:00
Brent Westbrook d936a5566e Rename 17 rules for source layout, imports, and directives (#30477)
| Code | Old name | New name |
|---|---|---|
| [`COM819`](https://docs.astral.sh/ruff/rules/COM819/) |
`prohibited-trailing-comma` | `unnecessary-trailing-comma` |
| [`E301`](https://docs.astral.sh/ruff/rules/E301/) |
`blank-line-between-methods` | `missing-blank-line-before-method` |
| [`E302`](https://docs.astral.sh/ruff/rules/E302/) |
`blank-lines-top-level` |
`missing-blank-lines-before-top-level-definition` |
| [`E305`](https://docs.astral.sh/ruff/rules/E305/) |
`blank-lines-after-function-or-class` |
`missing-blank-lines-after-top-level-definition` |
| [`E306`](https://docs.astral.sh/ruff/rules/E306/) |
`blank-lines-before-nested-definition` |
`missing-blank-line-before-nested-definition` |
| [`EXE001`](https://docs.astral.sh/ruff/rules/EXE001/) |
`shebang-not-executable` | `non-executable-file-with-shebang` |
| [`EXE002`](https://docs.astral.sh/ruff/rules/EXE002/) |
`shebang-missing-executable-file` | `executable-file-without-shebang` |
| [`F403`](https://docs.astral.sh/ruff/rules/F403/) |
`undefined-local-with-import-star` | `star-import` |
| [`F405`](https://docs.astral.sh/ruff/rules/F405/) |
`undefined-local-with-import-star-usage` |
`possibly-undefined-name-with-star-import` |
| [`F406`](https://docs.astral.sh/ruff/rules/F406/) |
`undefined-local-with-nested-import-star-usage` |
`non-module-level-star-import` |
| [`PLE2510`](https://docs.astral.sh/ruff/rules/PLE2510/) |
`invalid-character-backspace` | `unescaped-backspace-character` |
| [`PLE2512`](https://docs.astral.sh/ruff/rules/PLE2512/) |
`invalid-character-sub` | `unescaped-substitute-character` |
| [`PLE2513`](https://docs.astral.sh/ruff/rules/PLE2513/) |
`invalid-character-esc` | `unescaped-escape-character` |
| [`PLE2514`](https://docs.astral.sh/ruff/rules/PLE2514/) |
`invalid-character-nul` | `unescaped-null-character` |
| [`PLE2515`](https://docs.astral.sh/ruff/rules/PLE2515/) |
`invalid-character-zero-width-space` |
`unescaped-zero-width-space-character` |
| [`RUF100`](https://docs.astral.sh/ruff/rules/RUF100/) | `unused-noqa`
| `unused-suppression-comment` |
| [`RUF101`](https://docs.astral.sh/ruff/rules/RUF101/) |
`redirected-noqa` | `renamed-rule-in-suppression-comment` |
2026-11-19 19:12:57 -05:00
Brent Westbrook af3adfd58b Rename 25 rules for definitions, naming, and typing (#30476)
| Code | Old name | New name |

|---------------------------------------------------------|------------------------------------------------|-----------------------------------------------------|
| [`FBT001`](https://docs.astral.sh/ruff/rules/FBT001/) |
`boolean-type-hint-positional-argument` |
`boolean-annotated-positional-parameter` |
| [`FBT002`](https://docs.astral.sh/ruff/rules/FBT002/) |
`boolean-default-value-positional-argument` |
`boolean-default-value-positional-parameter` |
| [`N801`](https://docs.astral.sh/ruff/rules/N801/) |
`invalid-class-name` | `non-camel-case-class-name` |
| [`N802`](https://docs.astral.sh/ruff/rules/N802/) |
`invalid-function-name` | `non-lowercase-function-name` |
| [`N803`](https://docs.astral.sh/ruff/rules/N803/) |
`invalid-argument-name` | `non-lowercase-parameter-name` |
| [`N804`](https://docs.astral.sh/ruff/rules/N804/) |
`invalid-first-argument-name-for-class-method` |
`class-method-receiver-not-named-cls` |
| [`N805`](https://docs.astral.sh/ruff/rules/N805/) |
`invalid-first-argument-name-for-method` |
`instance-method-receiver-not-named-self` |
| [`N813`](https://docs.astral.sh/ruff/rules/N813/) |
`camelcase-imported-as-lowercase` | `camel-case-imported-as-lowercase` |
| [`N814`](https://docs.astral.sh/ruff/rules/N814/) |
`camelcase-imported-as-constant` | `camel-case-imported-as-constant` |
| [`N817`](https://docs.astral.sh/ruff/rules/N817/) |
`camelcase-imported-as-acronym` | `camel-case-imported-as-acronym` |
| [`N818`](https://docs.astral.sh/ruff/rules/N818/) |
`error-suffix-on-exception-name` |
`exception-class-without-error-suffix` |
| [`PLR0202`](https://docs.astral.sh/ruff/rules/PLR0202/) |
`no-classmethod-decorator` | `non-decorator-classmethod` |
| [`PLR0203`](https://docs.astral.sh/ruff/rules/PLR0203/) |
`no-staticmethod-decorator` | `non-decorator-staticmethod` |
| [`PLR6301`](https://docs.astral.sh/ruff/rules/PLR6301/) |
`no-self-use` | `unused-self` |
| [`PYI011`](https://docs.astral.sh/ruff/rules/PYI011/) |
`typed-argument-default-in-stub` |
`complex-default-for-annotated-parameter-in-stub` |
| [`PYI014`](https://docs.astral.sh/ruff/rules/PYI014/) |
`argument-default-in-stub` |
`complex-default-for-unannotated-parameter-in-stub` |
| [`PYI026`](https://docs.astral.sh/ruff/rules/PYI026/) |
`type-alias-without-annotation` | `unannotated-type-alias-in-stub` |
| [`PYI050`](https://docs.astral.sh/ruff/rules/PYI050/) |
`no-return-argument-annotation-in-stub` |
`parameter-annotated-with-no-return` |
| [`PYI051`](https://docs.astral.sh/ruff/rules/PYI051/) |
`redundant-literal-union` | `literal-union-with-supertype` |
| [`PYI057`](https://docs.astral.sh/ruff/rules/PYI057/) |
`byte-string-usage` | `deprecated-byte-string-type` |
| [`PYI058`](https://docs.astral.sh/ruff/rules/PYI058/) |
`generator-return-from-iter-method` |
`iter-method-with-generator-return-annotation` |
| [`TC006`](https://docs.astral.sh/ruff/rules/TC006/) |
`runtime-cast-value` | `cast-with-unquoted-type` |
| [`UP013`](https://docs.astral.sh/ruff/rules/UP013/) |
`convert-typed-dict-functional-to-class` | `functional-typed-dict` |
| [`UP014`](https://docs.astral.sh/ruff/rules/UP014/) |
`convert-named-tuple-functional-to-class` | `functional-named-tuple` |
| [`UP042`](https://docs.astral.sh/ruff/rules/UP042/) |
`replace-str-enum` | `manual-str-enum` |
2026-11-19 15:07:50 -05:00
Brent Westbrook b444480b79 Rename 20 pydocstyle rules (#30465)
Summary
--

This is the first in a series of rule rename PRs. I started here because
the renames are fairly straightforward and are within a single linter
group. Most of these rules were missing some mention of docstrings in
their names or were missing or misusing something like `missing` or
`incorrect`. For example, many of the rules that start with `missing-`
actually enforce a single line somewhere, so they flag 2+ lines as well
as zero.

The first commit contains the changes in `codes.rs`, which I thought was
probably easiest to review
and comment on if any of the names need further refinement. The second
commit contains the rest of
the downstream changes.

| Code | Old name | New name |

|---------------------------------------------------|-----------------------------------------|---------------------------------------------------|
| [`D201`](https://docs.astral.sh/ruff/rules/D201/) |
`blank-line-before-function` | `blank-line-before-function-docstring` |
| [`D202`](https://docs.astral.sh/ruff/rules/D202/) |
`blank-line-after-function` | `blank-line-after-function-docstring` |
| [`D203`](https://docs.astral.sh/ruff/rules/D203/) |
`incorrect-blank-line-before-class` |
`incorrect-blank-line-before-class-docstring` |
| [`D204`](https://docs.astral.sh/ruff/rules/D204/) |
`incorrect-blank-line-after-class` |
`incorrect-blank-line-after-class-docstring` |
| [`D205`](https://docs.astral.sh/ruff/rules/D205/) |
`missing-blank-line-after-summary` |
`incorrect-blank-line-after-docstring-summary` |
| [`D207`](https://docs.astral.sh/ruff/rules/D207/) |
`under-indentation` | `under-indented-docstring-line` |
| [`D208`](https://docs.astral.sh/ruff/rules/D208/) | `over-indentation`
| `over-indented-docstring-line` |
| [`D209`](https://docs.astral.sh/ruff/rules/D209/) |
`new-line-after-last-paragraph` |
`missing-new-line-after-last-docstring-paragraph` |
| [`D210`](https://docs.astral.sh/ruff/rules/D210/) |
`surrounding-whitespace` | `whitespace-around-docstring-text` |
| [`D211`](https://docs.astral.sh/ruff/rules/D211/) |
`blank-line-before-class` | `blank-line-before-class-docstring` |
| [`D212`](https://docs.astral.sh/ruff/rules/D212/) |
`multi-line-summary-first-line` |
`multiline-docstring-summary-on-separate-line` |
| [`D213`](https://docs.astral.sh/ruff/rules/D213/) |
`multi-line-summary-second-line` |
`multiline-docstring-summary-on-first-line` |
| [`D214`](https://docs.astral.sh/ruff/rules/D214/) |
`overindented-section` | `over-indented-docstring-section` |
| [`D215`](https://docs.astral.sh/ruff/rules/D215/) |
`overindented-section-underline` |
`over-indented-docstring-section-underline` |
| [`D300`](https://docs.astral.sh/ruff/rules/D300/) |
`triple-single-quotes` | `unconventional-docstring-quotes` |
| [`D408`](https://docs.astral.sh/ruff/rules/D408/) |
`missing-section-underline-after-name` |
`blank-line-before-docstring-section-underline` |
| [`D410`](https://docs.astral.sh/ruff/rules/D410/) |
`no-blank-line-after-section` |
`missing-blank-line-after-docstring-section` |
| [`D411`](https://docs.astral.sh/ruff/rules/D411/) |
`no-blank-line-before-section` |
`missing-blank-line-before-docstring-section` |
| [`D413`](https://docs.astral.sh/ruff/rules/D413/) |
`missing-blank-line-after-last-section` |
`missing-blank-line-after-last-docstring-section` |
| [`D417`](https://docs.astral.sh/ruff/rules/D417/) |
`undocumented-param` | `undocumented-parameter` |
2026-11-19 11:59:53 -05:00
astral-automations-bot[bot]andBrent Westbrook a7ba68b491 Bump version to 0.17.6 (#30578)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2026-11-19 10:32:47 -05:00
Alex Waygood 1b93171700 [ty] Add NumPy to ecosystem runs (#30572) 2026-11-19 13:18:13 +00:00
Micha ReiserandAlex Waygood 0222c19dd8 Share ty configuration for Ruff’s standalone scripts (#30463)
## Summary

Use a shared ty override for Ruff’s standalone scripts now that PEP 723
scripts can inherit project settings. Fix the diagnostics in the two PR
automation scripts so they can use the same checks.

## Notes to reviewers

All 28 standalone scripts passed ty with locked dependencies, and the
changed-file hooks and mocked workflow checks passed.

---------

Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2026-11-16 17:35:08 +00:00
0ebb77d35c Bump version to 0.17.5 (#30380)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2026-11-12 17:26:23 +00:00
Alex WaygoodandMicha Reiser 22e77392fb [ty] Split invalid-return-type into 3 focussed rules; rename empty-body and unsound-return (#29947)
Co-authored-by: Micha Reiser <micha@reiser.io>
2026-11-11 16:59:57 +00:00
Micha Reiser eeba58f5c2 [ty] Warn by default for a single overload (#30214)
## Summary

Today, `single-overload` defaults to severity error. I suggest we
downgrade the rule to warning. While this is most likely unintentional
and something the typing spec explicitly calls out type checker should
report. I don't think it should be an error:

* There's no runtime error, it does not lead to downstream errors (e.g.
because a value stores now a string instead of an int).
* It can be used to expose a narrower API than what the implementation
supports. TypeScript's compiler does not warn about a single-overlaod.
And the following is a valid use case in TypeScript, that should also
work in Python.

The use case is from VueJS:

```ts
// This component declares only the prop "foo".
useModel(props, "foo") // OK
useModel(props, "bar") // Type error: there is no "bar" prop
```

The `useModel` overload requires name (the second argument) to match
`Literal["foo"]`. So the second call is disallowed.


However, the `name` argument of the `useModel` implementation is
intentionally annotated as `name: string`, so that the implementation
can show a debug-only warning that the prop `bar` is disallowed, without
TypeScript raising any narrowing or dead code warnings.


This is certainly an advanced feature. But it suggests that there are
valid use cases for having a single overload, and that disqualifies it
from being a correctness rule.

Changing the severity doesn't change whether ty fails on CI, unless the
user explicitly opted in to existing with 0 when there are only warning
diagnostics.

## Test Plan

Verified the default severity in the CLI and ran the overload and typing
conformance tests.
2026-11-11 16:24:49 +01:00
Alex Waygood a0358433d2 Restore automatic PR labels and the ty title prefix (#30275) 2026-11-10 18:01:21 +00:00
zaniebot f94677a7de Assign reviewers to pull requests from forks (#30272) 2026-11-10 16:52:08 +00:00
Brent Westbrook 259397be2a Redirect renamed rules (#30081)
Summary
--

This PR extends our infrastructure for redirecting rule codes to
redirect rule names too, in
preparation for the many renames we have planned before stabilizing
human-readable names. As a proof
of concept, I also renamed
[`missing-return-type-undocumented-public-function` (`ANN201`)], which
seemed like a very clear rename target, having been accidentally renamed
in #3283.

The first commit contains the redirect changes, and the second commit
contains the `ANN201` rename,
along with the tests it allows us to exercise.

[`missing-return-type-undocumented-public-function` (`ANN201`)]:
https://docs.astral.sh/ruff/rules/missing-return-type-undocumented-public-function/

Test Plan
--

New CLI tests
2026-11-10 10:02:17 -05:00
Alex WaygoodandBrent Westbrook ce0077c3e5 [ty] Distinguish attributes missing from some union members (#29944)
## Summary

Users can now suppress an attribute error that affects only some union
members and still catch names that are missing from every member. For
example, both lines below previously reported `unresolved-attribute`:

```python
def clean(value: str | int) -> str:
    value.stirp()          # missing-attribute: neither str nor int has stirp
    return value.strip()   # missing-union-attribute: str has strip, int does not
```

The motivation is that a project adopting ty may want `value.strip()` on
`str | None` to only warn while a typo remains an error-severity
diagnostic.

Both rules default to errors. The same distinction applies to optional
values such as `str | None` and to attribute deletion. Attributes
defined only conditionally on a class continue to use
`possibly-missing-attribute`.

### Comparison to other type checkers

Pyright distinguishes between `reportOptionalMemberAccess` (which is
specifically for unions with `None`) and `reportAttributeAccessIssue`,
which is for all other unions. mypy, meanwhile, distinguishes between
`union-attr` (which is an error code for all attribute-access errors on
union types) and `attr-defined` (which is an error code for all
attribute-access errors on non-union types). So this new split doesn't
_exactly_ align with either precedent -- but I _do_ think it's probably
the most useful split we can do, and it's the one that makes the most
sense _a priori_ in my opinion.

### Compatibility

`unresolved-attribute` remains an alias for `missing-attribute` in
configuration and inline ignores. To preserve a setting such as
`unresolved-attribute = "warn"` for both cases, also set
`missing-union-attribute = "warn"`. On an access that fails for only
some union members, replace `# ty: ignore[unresolved-attribute]` with `#
ty: ignore[missing-union-attribute]`. Tools that consume diagnostic
codes should recognize both new names.

Fixes astral-sh/ty#2839 and fixes astral-sh/ty#2471.

## Test Plan

A comparison with ty 0.0.96 on typeshed-stats, docstring-adder,
sphinx-lint, and typing_extensions found the same messages, locations,
and severities. Eight diagnostics received `missing-union-attribute`
across those four projects (two in sphinx-lint and six in
typing_extensions).

---------

Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2026-11-06 14:46:50 -05:00
David Peter fca4b6eb67 [ty] Pick up the schema_salad ecosystem setup fix (#30136)
Update Ruff's locked `ecosystem-analyzer` revision to pick up the
`schema_salad` setup fix. Align the setup script's `mypy-primer` lock
with the analyzer. [ecosystem-analyzer
#246](https://github.com/astral-sh/ecosystem-analyzer/pull/246) ·
[mypy_primer #275](https://github.com/hauntsaninja/mypy_primer/pull/275)

The lock checks passed, and the setup script installed `schema_salad` at
`0fd6c25fb0` with the dependency cutoff from the failing ecosystem run.
`uv pip check` passed.
2026-11-06 15:24:47 +00:00
astral-automations-bot[bot]andBrent Westbrook 05cdc36a45 Bump version to 0.17.4 (#30073)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2026-11-05 13:22:15 -05:00
Alex Waygood 0696c1efd3 Add an update-only mode to the PR report publisher (#30063)
When a ty fuzz run passes after an earlier failure on the same PR,
#30047 will update the failure comment to show that the latest run
passed. A PR with no earlier report should stay quiet. CI on #30047
calls the reusable report publisher from `main`, so the new input must
land on `main` before that PR can pass it.

Add an optional `update-existing-only` input to the publisher. When
enabled, the publisher updates the bot’s existing comment for that
report. If there is no stored comment ID or the comment has been
deleted, it skips publication. Callers that omit the input can still
create a comment.
2026-11-05 11:40:37 +00:00
Charlie Marsh 75793d7b70 [ty] Benchmark CLI checkers with default concurrency (#30044)
## Summary

The CLI benchmarks now let each checker choose how many workers to use.
On the 16-core benchmark Mac, mypy goes from 16 external workers to
zero, and Pyright checks in one process. ty, Pyrefly, and Zuban still
select 16 threads.

`--single-threaded` still limits checker parallelism. The editor
benchmarks keep their existing settings.

The [18-project default-concurrency
results](https://github.com/astral-sh/ty/pull/4930) were collected with
the file-selection audit.
2026-11-04 18:14:16 -05:00
Charlie Marsh 79403d5133 [ty] Benchmark Django, SymPy, the OpenAI Python SDK, and pytest (#30050)
## Summary

We added Django, SymPy, the OpenAI Python SDK, and pytest to the ty
benchmarks, bringing the suite from 14 projects to 18. Each project has
a CLI benchmark and an editor benchmark that changes a return type. For
Django, we change `force_str` to return `int` and wait for a diagnostic
in `django.contrib.auth.hashers`.
2026-11-04 18:14:16 -05:00
Charlie MarshandCharlie Marsh db38fceeee [ty] Align checker settings in the benchmark runner (#29921)
## Summary

On Home Assistant, giving Pyright a list of every input file made a cold
check 4.4× slower than giving it the project directory. Pyright checked
the same files and reported the same diagnostics in both runs. Each
checker now discovers files from the project’s configured directories.

We also enable checking inside unannotated functions, point imports at
the project checkout, and give each CLI the available CPU count. On the
benchmark Mac, mypy gets 16 workers in place of the two configured by
Home Assistant. The language servers use the same physical path for the
workspace and its files, including when `/tmp` is a symlink.

The [published measurements](https://github.com/astral-sh/ty/pull/4889)
used the [file-selection
audit](https://github.com/astral-sh/ruff/blob/44595abd2280f6f9cfd6408179baea436c52d792/scripts/ty_benchmark/src/benchmark/targets.py).
On NumPy the audit selected 205 files. With native discovery, mypy finds
207 and Pyright reports 359.

---------

Co-authored-by: Charlie Marsh <charliemarsh@openai.com>
2026-11-04 18:14:16 -05:00
Charlie Marsh 62852de1c0 [ty] Add Zuban to CLI and LSP benchmarks (#29920)
## Summary

We added Zuban 0.10.2 to the CLI and editor benchmarks. It finished 13
of the 14 CLI benchmarks. We stopped the first NumPy warmup after ten
minutes. Its language server also crashed on the edits to discord.py,
PyTorch, and NumPy.

Zuban runs by default in both suites. Its language server reads settings
from `pyproject.toml`, so the runner writes the CLI settings there for
each editor benchmark and restores the original file afterward.
2026-11-04 21:47:50 +00:00
Charlie Marsh 6cf046770c [ty] Refresh benchmark project revisions (#29696)
## Summary

We updated eight of the 14 project revisions in the ty benchmarks. The
checker versions and dependency cutoff stay fixed so we can measure how
checking a newer version of each project affects performance.

(pandas removed its top-level `typings/` directory, so its benchmark now
targets `pandas/`.)
2026-11-04 21:47:50 +00:00
Charlie Marsh e6f74eed8c [ty] Update benchmark tools and expand project coverage (#29695)
## Summary

The ty benchmarks now cover 14 projects. We added HTTPX, NumPy, FastAPI,
SQLAlchemy, and Rich. Each has an editor benchmark that changes a return
type and measures how long it takes a diagnostic to appear in a file
that imports it.

We also updated the runner to Python 3.15, Pyrefly to 1.4.0, and mypy to
2.4.0.
2026-11-04 21:47:49 +00:00
renovate[bot] 1d62d967ae Update dependency pyrefly to v1.3.2 (#30000) 2026-11-04 00:49:37 -08:00
Alex Waygood a71ae066c6 Update Pylint to 4.0.9 in the benchmark environment (#29833) 2026-10-29 22:17:09 +00:00
astral-automations-bot[bot]andBrent Westbrook ba65abf8ba Bump version to 0.17.3 (#29745)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2026-10-26 20:14:27 -04:00
Charlie Marsh 5e90790d11 Compute the lexer keyword hash at compile time (#29677)
## Summary

Changing the lexer's keyword set now takes one edit to the Rust list.
The compiler finds a collision-free multiplier and builds the 64-slot
table, so we can remove the Python generator and its lockfile.

A two-byte keyword like `if` packs to `0x6669_0000_0000_0000`. Only the
low two bytes of the multiplier affect its slot. We place keywords from
shortest to longest, choosing each new byte of the multiplier without
disturbing the shorter keywords. The lexer still selects a slot with one
multiply and shift, then compares the full word.
2026-10-23 15:29:49 -04:00
Charlie Marsh de9d86e245 Speed up identifier and keyword lexing (#29463)
## Summary

We scan ASCII identifier continuations in sixteen-byte batches using a
byte lookup table, avoiding per-character decoding. Keyword lookup
filters by length and initial byte, then uses a compact hash table with
a full-word comparison to reject collisions. Non-ASCII suffixes retain
the existing Unicode path.
2026-10-23 13:58:33 -04:00
Zanie Blueandzaniebot b5968c8a8d Omit the report ZIP link when HTML is available (#29661)
Report comments include a redundant ZIP download link after the hosted
HTML links. Show the ZIP link only when the HTML report is unavailable,
including when its upload fails.

Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
2026-10-22 17:00:29 -07:00
dependabot[bot] 0e190851d3 Bump black from 26.3.1 to 26.10.1 in /scripts/benchmarks (#29657)
Signed-off-by: dependabot[bot] <support@github.com>
2026-10-22 23:25:25 +00:00
7fab688f61 [ty] Host ecosystem reports from pull requests (#29625)
ty ecosystem comments currently require downloading a ZIP to view the
full reports. Add an optional `upload-html` mode to the shared report
publisher and enable it for ty, linking directly to the diagnostic diff
and timing results while retaining the downloadable artifact.

The publisher runs from `main`, uses STS for GitHub credentials, and
obtains temporary AWS upload credentials through GitHub OIDC when
hosting is enabled. Each caller supplies filenames and content types,
with optional link labels. The publisher reads only those files from a
bounded archive without extracting or executing them during publication,
and stores reports under separate paths for each PR, report, run, and
attempt. If an upload fails, the publisher still attempts to post the
summary and, when available, the artifact link.

The `automations` environment requires `REPORTS_BUCKET`, `REPORTS_URL`,
`REPORTS_ROLE_ARN`, and `REPORTS_REGION` from the report hosting
infrastructure.

---------

Co-authored-by: Zanie Blue <2586601+zanieb@users.noreply.github.com>
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
2026-10-22 22:44:25 +00:00
astral-automations-bot[bot]andBrent Westbrook 30516b42ab Bump version to 0.17.2 (#29637)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2026-10-22 13:54:24 -04:00
Zanie Blueandzaniebot ec891bab3b Link the commit in pull request report footers (#29621)
The report footer renders the commit SHA as code, so it is not
clickable. Link the commit, label it as the revision the results
describe, and separate the footer from the report.

---------

Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
2026-10-21 23:22:03 -07:00
Zanie Blueandzaniebot ad83e52a22 Add a shared pull request report publisher (#29603)
Add `publish-pull-request-report.yml` as a shared publisher for
automation reports, using STS for GitHub writes and the KV actions for
comment IDs. Store IDs in `pull-request` scope so later runs can update
the same report.

Look up each stored ID directly and verify the bot, PR, marker, and run
metadata. When a report is published, missing or expired keys create a
new comment without scanning existing comments; deleted comments are
replaced. Malformed IDs, IDs belonging to another comment, and KV errors
fail the job. The publisher verifies the source run and current PR head
before writing; report workflows enable it separately.

---------

Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
2026-10-21 22:49:24 -07:00
Zanie Blueandzaniebot 332cacc6a3 Move pull request reviewer selection to GitHub Actions (#29613)
Move reviewer selection from `astral-bot` into a serialized Actions
worker triggered by Ruff's dispatch policy. The worker uses the
repository's reviewer pools and requests reviews as
`astral-automations-bot[bot]` using STS. Store round-robin credits in
`workflow` scope, with read and write access restricted to
`pull-request-reviewers.yml` on `main` in the `automations` environment.
When no reviewer state is stored, credits start at zero.

---------

Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
2026-10-21 22:32:14 -07:00
renovate[bot] 5bf63e4c10 Update dependency pyrefly to v1.3.1 (#29564)
This PR contains the following updates:

| Package | Change |
[Age](https://docs.renovatebot.com/merge-confidence/) |
[Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| pyrefly | `==1.3.0` → `==1.3.1` |
![age](https://developer.mend.io/api/mc/badges/age/pypi/pyrefly/1.3.1?slim=true)
|
![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/pyrefly/1.3.0/1.3.1?slim=true)
|

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - "before 4am on Wednesday"
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNzEuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE3MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-20 19:43:15 -07:00
David Peter c64734bcea [ty] Enable return type inference in scripts and tools (#29439)
## Summary

Enable return type inference for all 24 standalone Python scripts
checked by ty in CI, along with `py-fuzzer` and `ty_benchmark`.
2026-10-16 12:41:30 +00:00
Zanie Blueandzaniebot c79af46103 Require hashes for build dependencies (#29336)
Enable `require-build-hashes` for the Python projects and inline scripts
so isolated builds require declared hashes for their build dependencies.
Pin the build backends and their dependencies with hashes.

---------

Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
2026-10-15 20:15:23 +01:00
astral-automations-bot[bot] 4683270c69 Bump version to 0.17.1 (#29420)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
2026-10-15 10:56:41 -04:00
renovate[bot] ab433b1e2b Update dependency pyright to v1.1.414 (#29349)
This PR contains the following updates:

| Package | Change |
[Age](https://docs.renovatebot.com/merge-confidence/) |
[Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [pyright](https://redirect.github.com/Microsoft/pyright)
([source](https://redirect.github.com/Microsoft/pyright/tree/HEAD/packages/pyright))
| [`1.1.413` →
`1.1.414`](https://renovatebot.com/diffs/npm/pyright/1.1.413/1.1.414) |
![age](https://developer.mend.io/api/mc/badges/age/npm/pyright/1.1.414?slim=true)
|
![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/pyright/1.1.413/1.1.414?slim=true)
|

---

### Release Notes

<details>
<summary>Microsoft/pyright (pyright)</summary>

###
[`v1.1.414`](https://redirect.github.com/microsoft/pyright/releases/tag/1.1.414):
Published 1.1.414

[Compare
Source](https://redirect.github.com/Microsoft/pyright/compare/1.1.413...1.1.414)

#### Changes:

-
[`a28cce4`](https://redirect.github.com/Microsoft/pyright/commit/a28cce4077d87b80bead12de809f79845e39c8ba)
Bump version to 1.1.414
([#&#8203;11724](https://redirect.github.com/Microsoft/pyright/issues/11724))
-
[`10f9e77`](https://redirect.github.com/Microsoft/pyright/commit/10f9e773c012d3ae15fd3232915a86ffda64d371)
Bump the github-actions group across 1 directory with 5 updates
([#&#8203;11720](https://redirect.github.com/Microsoft/pyright/issues/11720))
-
[`d72e2fe`](https://redirect.github.com/Microsoft/pyright/commit/d72e2fe89e4ad4227d6c5be9bb6e57b082768e23)
Fix pnpm setup in benchmark workflow
([#&#8203;11719](https://redirect.github.com/Microsoft/pyright/issues/11719))
-
[`854df36`](https://redirect.github.com/Microsoft/pyright/commit/854df36c470a636ca44fbf222ea7d4f328f5f471)
Allow assigning signature metadata to functions
([#&#8203;11714](https://redirect.github.com/Microsoft/pyright/issues/11714))
-
[`c773932`](https://redirect.github.com/Microsoft/pyright/commit/c77393240247e017db6538d35876e723c518e06c)
Publish pyright-typeserver to npm
([#&#8203;11713](https://redirect.github.com/Microsoft/pyright/issues/11713))
-
[`fa9b200`](https://redirect.github.com/Microsoft/pyright/commit/fa9b200906fadc093e862ce62ec1afa985c7233d)
Pull Pylance with Pyright 1.1.413
([#&#8203;11697](https://redirect.github.com/Microsoft/pyright/issues/11697))
-
[`e690634`](https://redirect.github.com/Microsoft/pyright/commit/e690634a94ab05cf7dd003b63b1ac67cd6d6878e)
Fix printExpression output for slice expressions and set displays
([#&#8203;11598](https://redirect.github.com/Microsoft/pyright/issues/11598))
-
[`685de39`](https://redirect.github.com/Microsoft/pyright/commit/685de39b5d3f32092fb576c9de54e5472a28447d)
Report a specific diagnostic for walrus in a comprehension iterable
([#&#8203;11606](https://redirect.github.com/Microsoft/pyright/issues/11606))
\[
[#&#8203;11514](https://redirect.github.com/Microsoft/pyright/issues/11514)
]
-
[`5a958ca`](https://redirect.github.com/Microsoft/pyright/commit/5a958ca8f5119affaca3c6cfe2157cdc87480f0a)
Add regression tests for sentinel conditional type guards
([#&#8203;11686](https://redirect.github.com/Microsoft/pyright/issues/11686))
-
[`6a7d491`](https://redirect.github.com/Microsoft/pyright/commit/6a7d491300be1fe1c2328fb64b511f7e2a4b356a)
Preserve benchmark result comments
([#&#8203;11682](https://redirect.github.com/Microsoft/pyright/issues/11682))

<details><summary><b>See More</b></summary>

-
[`8575d83`](https://redirect.github.com/Microsoft/pyright/commit/8575d8332f73f96721987e373f9844a666446b94)
Fix Sentinel base type assignability
([#&#8203;11676](https://redirect.github.com/Microsoft/pyright/issues/11676))
-
[`37c0386`](https://redirect.github.com/Microsoft/pyright/commit/37c03866d7929e4745e802c37e7e9bcabeda080b)
Optimize type equality fast paths
([#&#8203;11420](https://redirect.github.com/Microsoft/pyright/issues/11420))
-
[`b8dd159`](https://redirect.github.com/Microsoft/pyright/commit/b8dd159d12ff7e99ad8e8854737ce21e9ecb7042)
Fix keyword-form assignability for standard dest parameters
([#&#8203;11670](https://redirect.github.com/Microsoft/pyright/issues/11670))
-
[`bf6ec03`](https://redirect.github.com/Microsoft/pyright/commit/bf6ec03650710f7dc34fcf42442e6590d960f5cb)
Fix benchmark comparison and reporting
([#&#8203;11680](https://redirect.github.com/Microsoft/pyright/issues/11680))
-
[`390f0aa`](https://redirect.github.com/Microsoft/pyright/commit/390f0aaa54fb0d6f17725a7f999e2457079b3bcb)
Fix TypeGuard and TypeIs narrowing for functions with union return types
([#&#8203;11608](https://redirect.github.com/Microsoft/pyright/issues/11608))
-
[`fabb9f9`](https://redirect.github.com/Microsoft/pyright/commit/fabb9f9f9005c1180a108cac1bb7daf9d1bb0721)
Fix type narrowing for equality comparisons with IntEnum and StrEnum
values
([#&#8203;11622](https://redirect.github.com/Microsoft/pyright/issues/11622))
-
[`20380f2`](https://redirect.github.com/Microsoft/pyright/commit/20380f21f4afe7db79dd4dd7d3e31ab7a6ca4c24)
Push Pylance changes to Pyright
([#&#8203;11678](https://redirect.github.com/Microsoft/pyright/issues/11678))
-
[`3a1e14d`](https://redirect.github.com/Microsoft/pyright/commit/3a1e14df628d4e7454be3bc46ce0f8be672c08fd)
Benchmark pull request merge commits
([#&#8203;11677](https://redirect.github.com/Microsoft/pyright/issues/11677))
-
[`d5162f9`](https://redirect.github.com/Microsoft/pyright/commit/d5162f944f9a565549939eb4dc87934c2f170689)
Cover two remaining dataclass field() override branches
([#&#8203;11679](https://redirect.github.com/Microsoft/pyright/issues/11679))
\[
[#&#8203;11665](https://redirect.github.com/Microsoft/pyright/issues/11665)
]
-
[`0b8c227`](https://redirect.github.com/Microsoft/pyright/commit/0b8c227baf95bae353ce9268336ea09d912cd728)
Fix crashes after clearing analyzer caches
([#&#8203;11675](https://redirect.github.com/Microsoft/pyright/issues/11675))
-
[`b895058`](https://redirect.github.com/Microsoft/pyright/commit/b89505828a17ce2a75df68419eaf10ab9707104d)
Do not inherit dataclass defaults when a subclass uses field() without
one.
([#&#8203;11665](https://redirect.github.com/Microsoft/pyright/issues/11665))
\[
[#&#8203;11660](https://redirect.github.com/Microsoft/pyright/issues/11660)
]
-
[`99854b7`](https://redirect.github.com/Microsoft/pyright/commit/99854b716017afd28e79e61e3904834e0fcbe854)
Dispatch typecheck benchmark workflow directly
([#&#8203;11674](https://redirect.github.com/Microsoft/pyright/issues/11674))
-
[`1be1899`](https://redirect.github.com/Microsoft/pyright/commit/1be1899ebec8591346b64a388ec195298e78f44e)
Do not treat objects with explicit **iter** as iterable via **getitem**.
([#&#8203;11645](https://redirect.github.com/Microsoft/pyright/issues/11645))
-
[`beaac8f`](https://redirect.github.com/Microsoft/pyright/commit/beaac8f2f965a4b27bfb1f958392254fcf86b11e)
Fix functools.partial type evaluation for callable class instances
([#&#8203;11672](https://redirect.github.com/Microsoft/pyright/issues/11672))
-
[`5b448bb`](https://redirect.github.com/Microsoft/pyright/commit/5b448bb99ddee5084ad51e1f7c4eb48455cc492c)
Do not eliminate a TypedDict with extra items on a negative key check
([#&#8203;11663](https://redirect.github.com/Microsoft/pyright/issues/11663))
-
[`bacb814`](https://redirect.github.com/Microsoft/pyright/commit/bacb8141b4e93fe2583db032a5318f42aa3a75e9)
Fix benchmark comment trigger
([#&#8203;11673](https://redirect.github.com/Microsoft/pyright/issues/11673))
-
[`93a6550`](https://redirect.github.com/Microsoft/pyright/commit/93a6550b2acc61d930edf54ad4621638cdc70777)
Add cross-type-checker benchmark regression tracking
([#&#8203;11574](https://redirect.github.com/Microsoft/pyright/issues/11574))
-
[`85837ef`](https://redirect.github.com/Microsoft/pyright/commit/85837efea752fbb1cd04c261a51cded566e21e57)
Add non-publishing registry validation mode
([#&#8203;11667](https://redirect.github.com/Microsoft/pyright/issues/11667))
-
[`3f87071`](https://redirect.github.com/Microsoft/pyright/commit/3f870712c2a2076ccfbfe685c3dbda7a453a590a)
Install pinned pnpm on Azure DevOps build agents
([#&#8203;11661](https://redirect.github.com/Microsoft/pyright/issues/11661))
\[
[#&#8203;11652](https://redirect.github.com/Microsoft/pyright/issues/11652)
]
-
[`d5e77a7`](https://redirect.github.com/Microsoft/pyright/commit/d5e77a7f4856f8e7caee1fa788b25bf67901a3c9)
Fix protocol cache for specialized self overloads
([#&#8203;11649](https://redirect.github.com/Microsoft/pyright/issues/11649))
-
[`7ddd3b1`](https://redirect.github.com/Microsoft/pyright/commit/7ddd3b121bf3421fd3e98ec4f3b38209e24c08d0)
Push Pylance changes to Pyright
([#&#8203;11658](https://redirect.github.com/Microsoft/pyright/issues/11658))
-
[`5cf4afa`](https://redirect.github.com/Microsoft/pyright/commit/5cf4afa88b491795af0a2bfd1683ec24c60dcf03)
Migrate CI package installation to pnpm
([#&#8203;11652](https://redirect.github.com/Microsoft/pyright/issues/11652))
-
[`334e3fb`](https://redirect.github.com/Microsoft/pyright/commit/334e3fb792f7b912a44715e74528fab22b73bf1e)
Pin GitHub Actions to full-length commit SHAs
([#&#8203;11628](https://redirect.github.com/Microsoft/pyright/issues/11628))

This list of changes was [auto
generated](https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=15264113\&view=logs).</details>

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - "before 4am on Wednesday"
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNTUuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE1NS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-14 08:05:39 +01:00
renovate[bot] 518d03e379 Update dependency pyrefly to v1.3.0 (#29357)
This PR contains the following updates:

| Package | Change |
[Age](https://docs.renovatebot.com/merge-confidence/) |
[Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| pyrefly | `==1.2.0` → `==1.3.0` |
![age](https://developer.mend.io/api/mc/badges/age/pypi/pyrefly/1.3.0?slim=true)
|
![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/pyrefly/1.2.0/1.3.0?slim=true)
|

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - "before 4am on Wednesday"
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNTUuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE1NS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-14 08:04:29 +01:00
Charlie Marsh a5aee680b1 Use thin LTO for PGO training (#29271)
## Summary

Use thin LTO for the instrumented release executable in the PGO build,
while keeping fat LTO for the final executable. Pass the override
through `cargo rustc` so dependency compilation and the workspace's
release profile are unchanged. Debug builds are unchanged.

This reduces the estimated PGO pipeline time by 54% in a Linux x86-64
comparison, with essentially unchanged final binary size and no
consistent runtime regression across the holdout projects used for the
original PGO work in #27570.

## Build results

| Measurement | Fat training | Thin training |
| --- | ---: | ---: |
| Instrumented build | 487.86 s | 85.50 s |
| Final fat-LTO build, median | 247.75 s | 250.43 s |
| Estimated complete PGO pipeline | 738.02 s | 338.26 s |
| Final stripped executable, median | 22.719 MiB | 22.724 MiB |

Instrumented builds were measured once per mode. Final build times and
sizes are medians of three independently collected profiles per mode.
Pipeline estimates add the instrumented build, median training and
profile merge, and median final build; corpus downloads are excluded.
These are clean local build measurements, not CI timing predictions.

## Runtime results

We trained on the canonical 618-file corpus and evaluated linting and
formatting on Django, pandas, scikit-learn, SciPy, and SymPy, whose
repositories are disjoint from the training corpus. The holdout pins
come from the contemporaneous Windows follow-up in #27573, since the
initial Linux PR did not record exact holdout revisions.

Each case used three independent profile pairs and 12 interleaved
fat/thin timing pairs per profile, for 720 timed runs in total. Runs
used eight pinned CPUs and eight threads, with balanced execution order
and warm caches. Diagnostics and formatting diffs were checked
separately from the quiet timed commands and matched across all six
final binaries.

Positive changes below mean thin training was slower. Intervals are
per-workload hierarchical bootstrap 95% confidence intervals, resampling
profile pairs and then paired timing observations; they are not adjusted
for multiple comparisons.

| Holdout | CPU change (95% CI) | Wall change (95% CI) |
| --- | ---: | ---: |
| Django lint | -0.21% [-0.77%, +0.49%] | +0.10% [-1.41%, +1.29%] |
| Django format | -0.10% [-1.18%, +0.72%] | -0.27% [-1.37%, +1.45%] |
| pandas lint | -0.01% [-0.45%, +0.39%] | -0.67% [-2.49%, +1.26%] |
| pandas format | -0.26% [-1.10%, +0.67%] | -0.84% [-2.81%, +2.12%] |
| scikit-learn lint | +0.02% [-0.62%, +1.12%] | -0.02% [-1.11%, +1.86%]
|
| scikit-learn format | -0.52% [-2.14%, +0.62%] | -0.79% [-2.08%,
+1.03%] |
| SciPy lint | -0.21% [-0.70%, +0.43%] | -0.64% [-4.59%, +1.52%] |
| SciPy format | -0.52% [-1.03%, +0.13%] | -2.47% [-6.20%, +1.61%] |
| SymPy lint | +0.01% [-0.30%, +0.27%] | -0.77% [-3.23%, +0.88%] |
| SymPy format | -0.18% [-1.89%, +1.84%] | -0.58% [-3.58%, +3.30%] |

A longer SymPy formatting repeat with the same three profile pairs and
48 timing pairs per profile measured -0.18% CPU [-1.96%, +1.66%] and
-0.28% wall time [-2.19%, +1.58%]. Individual profiles varied in both
directions, so the near-flat average should not be interpreted as
identical performance for every trained binary. The pandas formatting
wall-time interval remains inconclusive at a 2% slowdown threshold.

All six profiles had matching function/CFG identities and counter
layouts, with no missing-profile or profile-mismatch compiler
diagnostics. Their counter values and `.profdata` bytes were not
identical.

Measurements used the Rust/Cargo source at
`7c6f8fc8fc5655dfa47e125b05453007b9e7a6c7`, the standard release
codegen-unit settings, and Ohm Rust 1.99.0-dev (`08bbd755a`, LLVM
23.1.1) with experimental defaults disabled. The benchmark was not rerun
on this PR's newer base revision. This is an exploratory Linux x86-64
study with three profiles per mode, not a guarantee of zero runtime cost
or equivalent results on macOS, Windows, or ARM.
2026-10-10 08:26:15 -04:00
astral-automations-bot[bot]andBrent Westbrook 5b27c1ccfa Bump version to 0.17.0 (#29259)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2026-10-09 13:45:29 -04:00
David Peter d1d6a5d4ae [ty] Upgrade hyperfine to 2.0.0 for benchmarks (#29219)
Follow-up to #29151: upgrade the benchmark dependency from hyperfine
2.0.0-alpha.2 to 2.0.0.

Use `--env RAYON_NUM_THREADS=1` instead of an `env` command prefix for
the single-threaded formatter benchmarks. This avoids measuring the
overhead of `env` itself (~5 ms).
2026-10-08 13:34:20 +02:00
David Peter db2a4a9216 [ty] Upgrade hyperfine to 2.0.0-alpha.2 for benchmarks (#29151)
## Summary

Upgrade hyperfine to 2.0.0-alpha.2, which now includes support for
memory measurements (see output below).

Tested:

<details>
<summary>Formatter benchmark output</summary>

```text
Benchmark 1: ./target/release/ruff format ./fixture (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     12.8 ms  ±     0.2 ms      12.7 ms  …    12.9 ms
  Memory        16.6 MiB ±     0.1 MiB     16.5 MiB …    16.6 MiB
 
Benchmark 2: black --no-cache ./fixture --safe (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    207.1 ms  ±     4.8 ms     203.7 ms  …   210.5 ms         +1517.9%  (16.2x slower)
  Memory        37.3 MiB ±     0.1 MiB     37.2 MiB …    37.4 MiB         +125.2%  (2.3x more)
 
Benchmark 3: black --no-cache ./fixture --fast (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    226.6 ms  ±     7.4 ms     221.4 ms  …   231.9 ms         +1670.7%  (17.7x slower)
  Memory        37.4 MiB ±     0.1 MiB     37.3 MiB …    37.5 MiB         +126.2%  (2.3x more)
 
Benchmark 4: black --no-cache ./fixture --safe --preview (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    250.1 ms  ±    41.8 ms     220.6 ms  …   279.6 ms         +1854.3%  (19.5x slower)
  Memory        37.4 MiB ±     0.3 MiB     37.2 MiB …    37.6 MiB         +126.0%  (2.3x more)
 
Benchmark 5: black --no-cache ./fixture --fast --preview (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    186.2 ms  ±     2.5 ms     184.5 ms  …   188.0 ms         +1355.1%  (14.6x slower)
  Memory        37.3 MiB ±     0.0 MiB     37.3 MiB …    37.3 MiB         +125.2%  (2.3x more)
 
Benchmark 6: autopep8 ./fixture --recursive --in-place (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     41.6 ms  ±     0.7 ms      41.1 ms  …    42.1 ms          +225.2%  (3.3x slower)
  Memory        26.3 MiB ±     0.0 MiB     26.3 MiB …    26.3 MiB          +58.8%  (1.6x more)
 
Benchmark 7: yapf ./fixture --parallel --recursive --in-place (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    142.1 ms  ±     4.8 ms     138.7 ms  …   145.5 ms         +1010.6%  (11.1x slower)
  Memory        29.1 MiB ±     0.3 MiB     28.9 MiB …    29.4 MiB          +76.0%  (1.8x more)
 
Benchmark 1: env RAYON_NUM_THREADS=1 ./target/release/ruff format ./fixture (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     12.7 ms  ±     2.5 ms      10.9 ms  …    14.4 ms
  Memory        14.6 MiB ±     0.1 MiB     14.6 MiB …    14.7 MiB
 
Benchmark 2: black --no-cache ./fixture --workers=1 --safe (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     95.2 ms  ±     3.2 ms      92.9 ms  …    97.4 ms          +650.6%  (7.5x slower)
  Memory        38.4 MiB ±     0.0 MiB     38.4 MiB …    38.4 MiB         +162.6%  (2.6x more)
 
Benchmark 3: black --no-cache ./fixture --workers=1 --fast (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     89.6 ms  ±     2.8 ms      87.6 ms  …    91.6 ms          +606.5%  (7.1x slower)
  Memory        38.4 MiB ±     0.0 MiB     38.4 MiB …    38.4 MiB         +162.7%  (2.6x more)
 
Benchmark 4: black --no-cache ./fixture --workers=1 --safe --preview (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     94.7 ms  ±     0.6 ms      94.3 ms  …    95.2 ms          +646.9%  (7.5x slower)
  Memory        38.5 MiB ±     0.1 MiB     38.5 MiB …    38.6 MiB         +163.5%  (2.6x more)
 
Benchmark 5: black --no-cache ./fixture --workers=1 --fast --preview (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     92.7 ms  ±     0.6 ms      92.3 ms  …    93.2 ms          +631.4%  (7.3x slower)
  Memory        38.4 MiB ±     0.1 MiB     38.3 MiB …    38.4 MiB         +162.5%  (2.6x more)
 
Benchmark 6: autopep8 ./fixture --in-place --recursive --jobs=1 (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     43.5 ms  ±     3.7 ms      40.9 ms  …    46.1 ms          +242.7%  (3.4x slower)
  Memory        26.5 MiB ±     0.1 MiB     26.4 MiB …    26.6 MiB          +81.2%  (1.8x more)
 
Benchmark 7: yapf ./fixture --recursive --in-place (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     56.8 ms  ±     1.2 ms      55.9 ms  …    57.6 ms          +347.8%  (4.5x slower)
  Memory        27.3 MiB ±     0.1 MiB     27.2 MiB …    27.4 MiB          +86.5%  (1.9x more)
 
Benchmark 1: ./target/release/ruff format ./fixture --check (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     10.2 ms  ±     0.3 ms       9.9 ms  …    10.4 ms
  Memory        16.9 MiB ±     0.1 MiB     16.8 MiB …    17.0 MiB
 
Benchmark 2: black --no-cache ./fixture --check --safe (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    193.8 ms  ±     6.2 ms     189.5 ms  …   198.2 ms         +1806.9%  (19.1x slower)
  Memory        37.3 MiB ±     0.1 MiB     37.2 MiB …    37.4 MiB         +120.8%  (2.2x more)
 
Benchmark 3: black --no-cache ./fixture --check --fast (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    188.2 ms  ±     1.9 ms     186.8 ms  …   189.6 ms         +1751.4%  (18.5x slower)
  Memory        37.3 MiB ±     0.1 MiB     37.3 MiB …    37.4 MiB         +121.1%  (2.2x more)
 
Benchmark 4: black --no-cache ./fixture --check --safe --preview (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    202.1 ms  ±     6.3 ms     197.7 ms  …   206.6 ms         +1888.5%  (19.9x slower)
  Memory        37.2 MiB ±     0.0 MiB     37.2 MiB …    37.2 MiB         +120.5%  (2.2x more)
 
Benchmark 5: black --no-cache ./fixture --check --fast --preview (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    197.0 ms  ±    20.1 ms     182.8 ms  …   211.2 ms         +1837.8%  (19.4x slower)
  Memory        37.2 MiB ±     0.0 MiB     37.2 MiB …    37.2 MiB         +120.5%  (2.2x more)
 
Benchmark 6: autopep8 ./fixture --recursive --diff (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     38.5 ms  ±     0.1 ms      38.4 ms  …    38.6 ms          +278.4%  (3.8x slower)
  Memory        26.3 MiB ±     0.0 MiB     26.3 MiB …    26.4 MiB          +56.0%  (1.6x more)
 
Benchmark 7: yapf ./fixture --parallel --recursive --quiet (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    144.8 ms  ±     7.4 ms     139.6 ms  …   150.1 ms         +1324.8%  (14.2x slower)
  Memory        29.0 MiB ±     0.2 MiB     28.9 MiB …    29.1 MiB          +71.6%  (1.7x more)
```

</details>

<details>
<summary>ty_benchmark (multi-threaded, then single-threaded)</summary>

```text
jinja
-----

Benchmark 1: ty (2 runs)
                mean     ±       σ          min     …     max
  Wall Time     73.0 ms  ±     5.6 ms      69.0 ms  …    76.9 ms
  Memory       100.1 MiB ±     0.1 MiB    100.0 MiB …   100.2 MiB
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 2: Pyrefly (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    118.3 ms  ±     6.0 ms     114.0 ms  …   122.5 ms           +62.1%  (1.6x slower)
  Memory       130.9 MiB ±     0.2 MiB    130.8 MiB …   131.1 MiB          +30.8%  (1.3x more)
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 3: mypy (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    802.9 ms  ±     8.0 ms     797.2 ms  …   808.6 ms         +1000.2%  (11.0x slower)
  Memory       143.4 MiB ±     0.1 MiB    143.3 MiB …   143.5 MiB          +43.2%  (1.4x more)
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 4: mypy (warm) (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    208.8 ms  ±     5.6 ms     204.9 ms  …   212.7 ms          +186.1%  (2.9x slower)
  Memory        51.9 MiB ±     0.2 MiB     51.8 MiB …    52.0 MiB          -48.2%  (1.9x less)
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 5: Pyright (2 runs)
                mean     ±       σ          min     …     max
  Wall Time   1198.8 ms  ±    12.0 ms    1190.3 ms  …  1207.3 ms         +1542.8%  (16.4x slower)
  Memory       251.6 MiB ±     1.3 MiB    250.7 MiB …   252.5 MiB         +151.3%  (2.5x more)
 
  Warning: Ignoring non-zero exit code.

jinja
-----

Benchmark 1: ty (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    226.1 ms  ±    14.0 ms     216.2 ms  …   236.0 ms
  Memory        84.9 MiB ±     0.2 MiB     84.8 MiB …    85.0 MiB
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 2: Pyrefly (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    210.4 ms  ±     6.9 ms     205.5 ms  …   215.3 ms            -7.0%
  Memory        92.3 MiB ±     0.1 MiB     92.2 MiB …    92.4 MiB           +8.8%
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 3: mypy (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    896.4 ms  ±    88.8 ms     833.7 ms  …   959.2 ms          +296.4%  (4.0x slower)
  Memory       144.3 MiB ±     0.6 MiB    143.9 MiB …   144.7 MiB          +70.0%  (1.7x more)
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 4: mypy (warm) (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    213.0 ms  ±     3.7 ms     210.4 ms  …   215.6 ms            -5.8%
  Memory        51.7 MiB ±     0.1 MiB     51.6 MiB …    51.8 MiB          -39.1%  (1.6x less)
 
  Warning: Ignoring non-zero exit code.
 
Benchmark 5: Pyright (2 runs)
                mean     ±       σ          min     …     max
  Wall Time    981.9 ms  ±    13.0 ms     972.6 ms  …   991.1 ms          +334.2%  (4.3x slower)
  Memory       315.4 MiB ±     5.9 MiB    311.2 MiB …   319.5 MiB         +271.6%  (3.7x more)
 
  Warning: Ignoring non-zero exit code.
```

</details>

I also tested most of the other hyperfine invocations mentioned
somewhere in this repo.
2026-10-07 13:43:40 +00:00
Alex Waygood 8ecfc92734 Check script lockfiles in CI (#29021) 2026-10-02 14:28:56 +01:00
astral-automations-bot[bot]andBrent Westbrook 3265ed1f94 Bump version to 0.16.10 (#29055)
Co-authored-by: astral-automations-bot[bot] <305554984+astral-automations-bot[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2026-10-01 13:40:09 -04:00
Alex Waygood f7446a5970 [ty] Update ecosystem pins and refresh script lockfiles (#29017) 2026-09-30 10:38:07 +00:00
David Peter 55c48b1e76 [ty] Enable strict rules for even more projects and scripts (#28980)
## Summary

I found some more scripts / projects that were not using these rules.

## Test strategy

Ran `ty_benchmark` locally
2026-09-29 06:44:29 +02:00
David Peter de9071547c [ty] Enable None-union truthiness checks in Python scripts and tools (#28911)
## Summary

Enable `truthiness-test-of-none-union` in the standalone scripts' inline
configuration and in the `py-fuzzer` and `ty_benchmark` projects,
following #28889.
2026-09-28 11:47:03 +00:00