## Summary
Use a shared ty override for Ruff’s standalone scripts now that PEP 723
scripts can inherit project settings. Fix the diagnostics in the two PR
automation scripts so they can use the same checks.
## Notes to reviewers
All 28 standalone scripts passed ty with locked dependencies, and the
changed-file hooks and mocked workflow checks passed.
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
## Summary
Users can now suppress an attribute error that affects only some union
members and still catch names that are missing from every member. For
example, both lines below previously reported `unresolved-attribute`:
```python
def clean(value: str | int) -> str:
value.stirp() # missing-attribute: neither str nor int has stirp
return value.strip() # missing-union-attribute: str has strip, int does not
```
The motivation is that a project adopting ty may want `value.strip()` on
`str | None` to only warn while a typo remains an error-severity
diagnostic.
Both rules default to errors. The same distinction applies to optional
values such as `str | None` and to attribute deletion. Attributes
defined only conditionally on a class continue to use
`possibly-missing-attribute`.
### Comparison to other type checkers
Pyright distinguishes between `reportOptionalMemberAccess` (which is
specifically for unions with `None`) and `reportAttributeAccessIssue`,
which is for all other unions. mypy, meanwhile, distinguishes between
`union-attr` (which is an error code for all attribute-access errors on
union types) and `attr-defined` (which is an error code for all
attribute-access errors on non-union types). So this new split doesn't
_exactly_ align with either precedent -- but I _do_ think it's probably
the most useful split we can do, and it's the one that makes the most
sense _a priori_ in my opinion.
### Compatibility
`unresolved-attribute` remains an alias for `missing-attribute` in
configuration and inline ignores. To preserve a setting such as
`unresolved-attribute = "warn"` for both cases, also set
`missing-union-attribute = "warn"`. On an access that fails for only
some union members, replace `# ty: ignore[unresolved-attribute]` with `#
ty: ignore[missing-union-attribute]`. Tools that consume diagnostic
codes should recognize both new names.
Fixesastral-sh/ty#2839 and fixesastral-sh/ty#2471.
## Test Plan
A comparison with ty 0.0.96 on typeshed-stats, docstring-adder,
sphinx-lint, and typing_extensions found the same messages, locations,
and severities. Eight diagnostics received `missing-union-attribute`
across those four projects (two in sphinx-lint and six in
typing_extensions).
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
Enable `require-build-hashes` for the Python projects and inline scripts
so isolated builds require declared hashes for their build dependencies.
Pin the build backends and their dependencies with hashes.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
## Summary
Enable `truthiness-test-of-none-union` in the standalone scripts' inline
configuration and in the `py-fuzzer` and `ty_benchmark` projects,
following #28889.
Summary
--
This PR updates our ruff-pre-commit dependency to 0.16.5, which supports
the new category selectors,
and then enables `lint.preview` and the new default categories so that
we can dogfood them on our
code. There were 66 diagnostics initially, which I also fixed or
ignored:
```
28 FURB113 [ ] repeated-append
12 ASYNC240 [ ] blocking-path-method-in-async-function
6 RUF105 [*] noqa-comments
5 PLW1510 [ ] subprocess-run-without-check
3 PLW0108 [ ] unnecessary-lambda
2 EXE001 [ ] shebang-not-executable
2 RUF039 [*] unraw-re-pattern
1 B024 [ ] abstract-base-class-without-abstract-method
1 FURB142 [*] for-loop-set-mutations
1 PERF102 [ ] incorrect-dict-iterator
1 PLC0207 [*] missing-maxsplit-arg
1 PLW0127 [ ] self-assigning-variable
1 TRY004 [ ] type-check-without-type-error
1 PLW0120 [*] useless-else-on-loop
1 TRY201 [ ] verbose-raise
```
Of these, I ignored `repeated-append` and `blocking-path-method-in-async-function`. I think they are
still reasonable defaults, just less useful/important in our uses.
0.16.5 also includes https://github.com/astral-sh/ruff-pre-commit/pull/159, so I removed `markdown` from our main `ruff-format` hook, leaving it for the later mdtest formatting stage.
I also had to make a few changes related to config discovery in tests:
- move our cached ecosystem checkouts to `$RUNNER_TEMP` instead of inside of our checkout
- convert some CLI tests to use the `CliTest` helpers
- add `configurationPreference: editorOnly` to some notebook tests
In these cases, our own Ruff settings were being inherited by test code. This likely affects many other tests, but I only updated the obviously failing ones for now. I'm least confident about the notebook tests, but as far as I can tell from the history (https://github.com/astral-sh/ruff/pull/11944, https://github.com/astral-sh/ruff/pull/22810), the specific diagnostics are not the important parts being asserted.
Test Plan
--
CI on this PR
## Summary
This PR introduces clippy-style rule categories that can be used
alongside existing linter-based categories in preview. The categories
have a lower `specificity` than linter groups, meaning that `select =
UP` and `ignore = correctness` will not ignore `UP` correctness rules.
In general this leads to more rules being enabled than the alternative
we considered where categories and linter groups had equal precedence,
which makes it seem like a safer default. We can revisit this if it
proves to be unintuitive, or we can add a Cargo-style `priority` field
in the worst case.
I've tried to organize the commits such that this can be reviewed
commit-by-commit and could also be split into multiple PRs. The first
two commits update the `ViolationMetadata` macro to allow providing a
`category` argument and add such arguments for all rules. This part
could be a standalone change without exposing the categories to users
but still depends on reaching consensus on all rule categories. The
second pair of commits allows using the new categories as selectors
alongside existing selectors, including some precedence tests with other
selector types.
This mostly addresses #1774, but there are some sub-issues there to
resolve before closing the issue.
## Test Plan
New CLI tests and a new mdtest suite around rule precedence
Python tooling can currently build third-party source distributions. Set
`no-build` in the maintained Python projects and PEP 723 scripts,
retaining named exceptions for Git-pinned tooling and the local
ecosystem runner. Pin the isolated build dependencies for those
exceptions, including their transitive requirements.
Follows merged astral-sh/ruff#27856, which gives the legacy `ruff-lsp`
suite a dedicated, locked Python 3.12 test environment.
Follow-up to astral-sh/ruff#27843. Replaces astral-sh/ruff#27840, which
closed when the merged docs branch was deleted.
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
Summary
--
This PR updates our rooster config to replace `NEXT_RUFF_VERSION` with
the new
version when doing releases. This will be helpful for the
`preview_since`
entries (and even others like `stable_since`) in the
`violation_metadata` macro,
which we currently update manually.
This is the variable name that we discussed briefly on Discord, but
something shorter might be nice too? ~~I guess we may also want a check
somewhere that there's not a typo preventing the version from ever being
updated?~~
I went ahead and added this validation to the macro.
Test Plan
--
The first commit is the entirety of the rooster config change. The
second commit
modifies an existing rule to use the `NEXT_RUFF_VERSION` field, and then
the
third commit shows the result of a `./scripts/release.sh` run that
successfully
updates the rule metadata. The fourth commit reverts the test release.
## Summary
This PR sets up CI jobs to run ty from the `main` branch on the files
and subdirectories in our `scripts` directory
## Test Plan
Both these commands pass for me locally:
- `uv run --project=./scripts cargo run -p ty check --project=./scripts`
- `uv run --project=./scripts/ty_benchmark cargo run -p ty check
--project=./scripts/ty_benchmark`
Summary
--
Inspired by #20859, this PR adds the version a rule was added, and the
file and line where it was defined, to `ViolationMetadata`. The file and
line just use the standard `file!` and `line!` macros, while the more
interesting version field uses a new `violation_metadata` attribute
parsed by our `ViolationMetadata` derive macro.
I moved the commit modifying all of the rule files to the end, so it
should be a lot easier to review by omitting that one.
As a curiosity and a bit of a sanity check, I also plotted the rule
numbers over time:
<img width="640" height="480" alt="image"
src="https://github.com/user-attachments/assets/75b0b5cc-3521-4d40-a395-8807e6f4925f"
/>
I think this looks pretty reasonable and avoids some of the artifacts
the earlier versions of the script ran into, such as the `rule`
sub-command not being available or `--explain` requiring a file
argument.
<details><summary>Script and summary data</summary>
```shell
gawk --csv '
NR > 1 {
split($2, a, ".")
major = a[1]; minor = a[2]; micro = a[3]
# sum the number of rules added per minor version
versions[minor] += 1
}
END {
tot = 0
for (i = 0; i <= 14; i++) {
tot += versions[i]
print i, tot
}
}
' ruff_rules_metadata.csv > summary.dat
```
```
0 696
1 768
2 778
3 803
4 822
5 848
6 855
7 865
8 893
9 915
10 916
11 924
12 929
13 932
14 933
```
</details>
Test Plan
--
I built and viewed the documentation locally, and it looks pretty good!
<img width="1466" height="676" alt="image"
src="https://github.com/user-attachments/assets/5e227df4-7294-4d12-bdaa-31cac4e9ad5c"
/>
The spacing seems a bit awkward following the `h1` at the top, so I'm
wondering if this might look nicer as a footer in Ruff. The links work
well too:
- [v0.0.271](https://github.com/astral-sh/ruff/releases/tag/v0.0.271)
- [Related
issues](https://github.com/astral-sh/ruff/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20airflow-variable-name-task-id-mismatch)
- [View
source](https://github.com/astral-sh/ruff/blob/main/crates%2Fruff_linter%2Fsrc%2Frules%2Fairflow%2Frules%2Ftask_variable_name.rs#L34)
The last one even works on `main` now since it points to the
`derive(ViolationMetadata)` line.
In terms of binary size, this branch is a bit bigger than main with
38,654,520 bytes compared to 38,635,728 (+20 KB). I guess that's not
_too_ much of an increase, but I wanted to check since we're generating
a lot more code with macros.
---------
Co-authored-by: GiGaGon <107241144+MeGaGiGaGon@users.noreply.github.com>
## Summary
When using `add_rule.py`, it produces the following line in `codes.rs`
```
(Flake8Async, "102") => (RuleGroup::Stable, rules::flake8-async::rules::BlockingOsCallInAsyncFunction),
```
Causing a syntax error.
This PR resolves that issue so that the script can be used again.
## Test Plan
Tested manually in new rule creation
## Summary
Implement `write-whole-file` (`FURB103`), part of #1348. This is largely
a copy and paste of `read-whole-file` #7682.
## Test Plan
Text fixture added.
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
## Summary
The `add_rule.py` script would create a test case that pointed to a file
that didn't exist when the linter is set to `"pylint"`. This PR fixes
that.
## Test Plan
`python scripts/add_rule.py --name DoTheThing --prefix PL --code C0999
--linter pylint`
* Document codes.rs
* Refactor codes.rs before merging
Helper script:
```python
# %%
from pathlib import Path
codes = Path("crates/ruff/src/codes.rs").read_text().splitlines()
rules = Path("a.txt").read_text().strip().splitlines()
rule_map = {i.split("::")[-1]: i for i in rules}
# %%
codes_new = []
for line in codes:
if ", Rule::" in line:
left, right = line.split(", Rule::")
right = right[:-2]
line = left + ", " + rule_map[right] + "),"
codes_new.append(line)
# %%
Path("crates/ruff/src/codes.rs").write_text("\n".join(codes_new))
```
Co-authored-by: Jonathan Plasse <13716151+JonathanPlasse@users.noreply.github.com>
## Summary
This PR moves `Diagnostic`, `DiagnosticKind`, and `Fix` into their own crate, which will enable us to further split up Ruff, since sub-linter crates (which need to implement functions that return `Diagnostic`) can now depend on `ruff_diagnostics` rather than Ruff.
Currently the define_rule_mapping! macro generates both the Rule enum as
well as the RuleCodePrefix enum and the mapping between the two. After
this commit series the macro will only generate the Rule enum and the
RuleCodePrefix enum and the mapping will be generated by a new map_codes
proc macro, so we rename the macro now to fit its new purpose.
```console
❯ cargo run rule B017
Finished dev [unoptimized + debuginfo] target(s) in 0.13s
Running `target/debug/ruff rule B017`
no-assert-raises-exception
Code: B017 (flake8-bugbear)
### What it does
Checks for `self.assertRaises(Exception)`.
## Why is this bad?
`assertRaises(Exception)` can lead to your test passing even if the
code being tested is never executed due to a typo.
Either assert for a more specific exception (builtin or custom), use
`assertRaisesRegex` or the context manager form of `assertRaises`.
```
- optional `prefix` argument for `add_plugin.py`
- rules directory instead of `rules.rs`
- pathlib syntax
- fix test case where code was added instead of name
Example:
```
python scripts/add_plugin.py --url https://pypi.org/project/example/1.0.0/ example --prefix EXA
python scripts/add_rule.py --name SecondRule --code EXA002 --linter example
python scripts/add_rule.py --name FirstRule --code EXA001 --linter example
python scripts/add_rule.py --name ThirdRule --code EXA003 --linter example
```
Note that it breaks compatibility with 'old style' plugins (generation works fine, but namespaces need to be changed):
```
python scripts/add_rule.py --name DoTheThing --code PLC999 --linter pylint
```
"origin" was accurate since ruff rules are currently always modeled
after one origin (except the Ruff-specific rules).
Since we however want to introduce a many-to-many mapping between codes
and rules, the term "origin" no longer makes much sense. Rules usually
don't have multiple origins but one linter implements a rule first and
then others implement it later (often inspired from another linter).
But we don't actually care much about where a rule originates from when
mapping multiple rule codes to one rule implementation, so renaming
RuleOrigin to Linter is less confusing with the many-to-many system.