## Summary
This is a first step towards #20411. After this, we would work to either
upstream changes or adjust things within Ruff.
Performance improvements have already been upstreamed
(https://github.com/rust-lang/annotate-snippets-rs/pull/437,
https://github.com/rust-lang/annotate-snippets-rs/pull/440)
Re-implemented the following from the fork:
- Hidden snippets
(https://github.com/rust-lang/annotate-snippets-rs/issues/448)
- Renderer-level disabling of hyperlinks
- Jupyter cell indices
(https://github.com/rust-lang/annotate-snippets-rs/issues/449)
- Fixable mark
(https://github.com/rust-lang/annotate-snippets-rs/issues/450)
- `lineno_offset` (maybe not needed if #20648 is taken care of)
- anonymized lines affects origin
(https://github.com/rust-lang/annotate-snippets-rs/issues/451)
- cut indicator (maybe not needed if we can switch to unicode renderer)
Additional hacks to minimize behavior change include
- Adjust upstream's hidden severity behavior to match Ruff's
(https://github.com/rust-lang/annotate-snippets-rs/issues/452)
- How newlines are annotated
(https://github.com/rust-lang/annotate-snippets-rs/issues/454)
- To not indent for line numbers when none will be present
(https://github.com/rust-lang/annotate-snippets-rs/issues/453)
- Enabling `simd` by default
All of the above represents work that needs to be resolved to address
#20411.
`lib.name` hack is used to simplify maintenance with the assumption that
we'll continue working towards #20411.
Behavior changes that were included
- Tweak when implicit padding lines are present
- Fix Origin's line/column is now derived from the primary annotation
- Make consistent the rendering of primary/context annotations
- Fix snippets to show up for empty source (but may degrade #24528)
- Fix handling of term width
- Fix rendering of annotations between newline and first character
(granted, maybe the caller is wrong)
- Tweak multiline annotation markers
New or upcoming features annotate-snippets features that we will be able
to leverage by moving this forward
- `AnnotationKind::Visible` to improve the quality of built-in
code-folding if Ruff were to switch to it
- `DecorStyle::Unicode`
- *(upcoming)* No-graphiscs mode for screen readers (may also be useful
for AIs)
Deferred:
- Moving related `Group`s into the same `Report`
- Using `secondary_title` for any `Group` after the first in a `Report`
- Using a `Snippet`, instead of titles, for "virtual code" like
https://github.com/astral-sh/ruff/pull/27033#discussion_r3622727236
- Switching hidden snippets to `Origin`
## Test Plan
Within cpython, I ran
```console
$ hyperfine -i "../../ruff/target-baseline/release/ruff check" "../../ruff/target-update/release/ruff check" -N -m20 --output=null
Benchmark 1: ../../ruff/target-baseline/release/ruff check
Time (mean ± σ): 27.5 ms ± 1.1 ms [User: 33.0 ms, System: 108.4 ms]
Range (min … max): 25.0 ms … 31.7 ms 94 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: ../../ruff/target-update/release/ruff check
Time (mean ± σ): 28.1 ms ± 2.0 ms [User: 33.4 ms, System: 105.7 ms]
Range (min … max): 24.8 ms … 38.9 ms 102 runs
Warning: Ignoring non-zero exit code.
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use t
he '--warmup' or '--prepare' options.
Summary
../../ruff/target-baseline/release/ruff check ran
1.03 ± 0.08 times faster than ../../ruff/target-update/release/ruff check
```
Before:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/2d94dabf-b28d-48c1-b17c-52aac59fd57e"
/>
After:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/b388ada3-e3d7-4ba0-b90b-1330eb81f4be"
/>
## Summary
Prior to this change, when a slice bound was not an integer literal,
boolean literal, or `None`, we inferred a bare `slice[Any, Any, Any]`.
This erased enough information that subscription checking accepted
invalid slices even when `__getitem__` requires `SupportsIndex | None`
bounds:
```python
def get(values: list[int], start: float) -> list[int]:
return values[start:] # error: [invalid-argument-type]
```
(This may have been waiting on
https://github.com/astral-sh/ruff/commit/cb58704e085fe7e70d72842294575de47e47bead?)
We now preserve the inferred type of each provided slice bound in
`slice[...]`, so ordinary argument checking diagnoses invalid bounds for
built-in sequences and user-defined `__getitem__` methods with typed
slice parameters.
Closes https://github.com/astral-sh/ty/issues/3547.
## Summary
Previously, narrowing an enum by excluding one or more members eagerly
expanded the enum into a union of all remaining enum literals. That's
fine for small enums, but can cause large enums to blow up during
repeated equality or membership narrowing. And large enums are not
uncommon...
This PR keeps those narrowed types in a compact form, like `Color &
~Literal[Color.RED]`, instead of immediately expanding them to
`Literal[Color.GREEN, Color.BLUE, ...]`. The type is expanded only when
we need to inspect the remaining enum members individually (like for
overload matching or attribute lookup). The net effect is that we retain
the existing behavior while avoiding a brutal performance penalty for
large enums.
Note that we do vary some behavior by enum size. For example, with more
than 7 elements, we use (as you can see in the ecosystem diff) the
compact representation (`SupportedBlockchain &
~Literal[SupportedBlockchain.ETHEREUM_BEACONCHAIN`).
For the benchmark in the linked issue, `main` doesn't finish within 30s,
but this PR completes in 128ms.
Closes https://github.com/astral-sh/ty/issues/1588.
---------
Co-authored-by: Codex <noreply@openai.com>
- Updated the `prek` hook to replace `blacken-docs` with ruff
- Added some formatting directives to preserve obvious intentional
formatting choices
- Made some choice edits to content for better readability with new format
- Formatted md tests and updated snapshots
## Summary
Follow up from https://github.com/astral-sh/ruff/pull/21411. Again,
there are more things that could be improved here (like the diagnostics
for `lists`, or extending what we have for `dict` to `OrderedDict` etc),
but that will have to be postponed.
## Summary
Currently our diagnostic only covers the range of the thing being
subscripted:
<img width="1702" height="312" alt="image"
src="https://github.com/user-attachments/assets/7e630431-e846-46ca-93c1-139f11aaba11"
/>
But it should probably cover the _whole_ subscript expression (arguably
the more "incorrect" bit is the `["foo"]` part of this expression, not
the `x` part of this expression!)
## Test Plan
Added a snapshot
Co-authored-by: Brent Westbrook
<36778786+ntBre@users.noreply.github.com>
## Summary
Further improve subscript assignment diagnostics, especially for
`dict`s:
```py
config: dict[str, int] = {}
config["retries"] = "three"
```
<img width="1276" height="274" alt="image"
src="https://github.com/user-attachments/assets/9762c733-8d1c-4a57-8c8a-99825071dc7d"
/>
I have many more ideas, but this looks like a reasonable first step.
Thank you @AlexWaygood for some of the suggestions here.
## Test Plan
Update tests
## Summary
We synthesize a (potentially large) set of `__setitem__` overloads for
every item in a `TypedDict`. Previously, validation of subscript
assignments on `TypedDict`s relied on actually calling `__setitem__`
with the provided key and value types, which implied that we needed to
do the full overload call evaluation for this large set of overloads.
This PR improves the performance of subscript assignment checks on
`TypedDict`s by validating the assignment directly instead of calling
`__setitem__`.
This PR also adds better handling for assignments to subscripts on union
and intersection types (but does not attempt to make it perfect). It
achieves this by distributing the check over unions and intersections,
instead of calling `__setitem__` on the union/intersection directly. We
already do something similar when validating *attribute* assignments.
## Ecosystem impact
* A lot of diagnostics change their rule type, and/or split into
multiple diagnostics. The new version is more verbose, but easier to
understand, in my opinion
* Almost all of the invalid-key diagnostics come from pydantic, and they
should all go away (including many more) when we implement
https://github.com/astral-sh/ty/issues/1479
* Everything else looks correct to me. There may be some new diagnostics
due to the fact that we now check intersections.
## Test Plan
New Markdown tests.
## Summary
Adds validation to subscript assignment expressions.
```py
class Foo: ...
class Bar:
__setattr__ = None
class Baz:
def __setitem__(self, index: str, value: int) -> None:
pass
# We now emit a diagnostic on these statements
Foo()[1] = 2
Bar()[1] = 2
Baz()[1] = 2
```
Also improves error messages on invalid `__getitem__` expressions
## Test Plan
Update mdtests and add more to `subscript/instance.md`
---------
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: David Peter <mail@david-peter.de>