## Summary
Given a `Movie` TypedDict with a `title` field, ty currently reports
`invalid-key` for three cases. This PR gives each its own rule.
| Example | Rule |
| --- | --- |
| `movie["titel"]` reads a misspelled key | `invalid-typed-dict-key` |
| `movie[key]` reads a key whose type is `str` and whose value is
unknown | `non-literal-typed-dict-key` |
| `Movie(title="Alien", titel="Alien")` supplies an unknown key during
construction | `unknown-typed-dict-constructor-key` |
Both subscript rules apply to reads and assignments. An invalid
non-string key such as `movie[0]` reports `invalid-typed-dict-key`. The
constructor rule reports the extra `titel` key in `movie: Movie =
{"title": "Alien", "titel": "Alien"}`. For a closed TypedDict, it flags
`{key: value}` when `key` has type `str` and might introduce an extra
item.
Projects migrating to ty lose typo detection if they disable
`invalid-key` to allow dynamic TypedDict lookups. In mypy, disabling
`literal-required` allows `movie[key]` while `movie["titel"]` still
produces a `typeddict-item` error. Pyright allows the dynamic lookup by
default and also flags the typo. To keep that behavior in ty, set
`non-literal-typed-dict-key` to `ignore` and leave
`invalid-typed-dict-key` enabled.
All three rules default to `error`. With the alias handling from #29960,
`invalid-key` maps to `invalid-typed-dict-key` in configuration, inline
ignores, and existing baselines. Previously baselined diagnostics can
therefore reappear under either of the other two rule names. To preserve
a custom severity across the split, set all three rules to the same
level. For inline ignores, use the rule or rules reported on that line.
---------
Co-authored-by: David Peter <mail@david-peter.de>
When `typing.Type` is imported with a fallback to the builtin `type`, ty
can reject a valid `Type[T]` annotation and infer `Unknown` for a
generic call that should return `int`. A `typing.Tuple` import with a
`tuple` fallback has the same problem.
Use the same annotation handling for each pair of spellings, including
when ty sees multiple possible bindings for the subscripted name.
Fixesastral-sh/ty#4907.
## Test plan
Checked the issue reproducer and a dictionary constructor call against
the `frozendict` 2.4.7 stub with Python 3.12 selected. The generic call
reveals `int`, and `frozendict(x)` reveals `frozendict[str, int]`.
## Summary
We now infer `tuple[int, str]` for slices of tuple-based `NewType`s like
this one:
```python
from typing import NewType
MyTuple = NewType("MyTuple", tuple[int, str, None])
def example(value: MyTuple) -> tuple[int, str]:
return value[:2]
```
Previously, we fell back to the general `__getitem__` signature and
inferred `tuple[int | str | None, ...]`. We now apply tuple slicing to
the `NewType`'s concrete base, returning a plain tuple with the sliced
element types.
Closes https://github.com/astral-sh/ty/issues/4632.
## Summary
Closes https://github.com/astral-sh/ty/issues/1738
(While this PR also resolves the error in the code provided in
https://github.com/astral-sh/ty/issues/1846, I consider the broader
issue under discussion to still be unresolved)
To fully support implicit type aliases, this PR introduces `Recursive`
types. These represent *structural recursive types*. In other words,
they are anonymous recursive types implicitly constructed through type
inference, rather than a nominally defined recursive type like PEP 695
recursive type aliases.
This is what was being discussed in
https://github.com/astral-sh/ruff/pull/24252#issuecomment-4172655350. We
are not introducing recursive types solely for implicit type aliases;
this is the first step toward completely replacing the current
`Divergent`-based approximate recursive type inference with a full
implementation.
Initially, I planned to completely remove the `Divergent` type alongside
the introduction of `Recursive` types, but because the scope of the
impact was far too large, I decided to replace it incrementally. Both
will coexist until the replacement is complete, but eventually,
everything should be replaced by `Recursive` types.
Following this PR, I plan to use the newly introduced `Recursive` types
to replace various forms of recursive type inference.
## Design
Currently, we approximate and treat recursive type structures generated
by recursive type inference as gradual types containing `Divergent`. For
example, a subscript on `x: Divergent` results in `x[n]: Divergent`.
Consequently, we get:
```
x: list[Divergent]
x[n]: Divergent
x[n][n]: Divergent
```
As a result, the type after subscripting loses the information that `x`
was a recursive list type. We solve this problem by introducing
recursive types.
`RecursiveType` is a binder for a body type that contains a recursive
variable. In other words, it roughly has the following structure:
```rust
struct RecursiveType<'db> {
id: salsa::Id,
body: Type<'db>,
arguments: Option<Specialization<'db>>,
}
struct RecursiveVar<'db> {
id: salsa::Id,
arguments: Option<Specialization<'db>>,
}
enum Type<'db> {
Recursive(RecursiveType<'db>),
RecursiveVar(RecursiveVar<'db>),
...
}
```
For example, for the following implicit type alias:
```python
Tree = tuple[T, "Tree[list[T]] | None"]
# = μF. λT. tuple[T, F[list[T]] | None]
```
the constructed `RecursiveType` looks like this:
```rust
RecursiveType {
id,
body: tuple[T, RecursiveVar { id, arguments: [list[T]] } | None],
arguments: [T]
}
```
Unlike `Divergent`, the recursive variable `RecursiveVar` has no type
properties on its own. Attempting to access this variable immediately
unfolds the recursive type, so questioning the typing properties of the
variable itself is meaningless. Therefore, if any type operation on
`RecursiveVar` appears, it should be a bug.
To infer the container subscript `x[1]` for `x: Tree[int]`, first unfold
`x`'s type. Unfolding replaces references to the recursive binder with
the recursive type itself. Writing `B[a := R]` for capture-avoiding
substitution of `R` for `a` in `B`:
```text
unfold(μa. B) = B[a := μa. B]
```
For `Tree[int]`, substitute the constructor for `F`, then apply `T :=
int`:
```text
unfold((μF. λT. tuple[T, F[list[T]] | None])[int])
= tuple[int, (μF. λT. tuple[T, F[list[T]] | None])[list[int]] | None]
(= tuple[int, Tree[list[int]] | None])
```
Tuple subscripting then selects the element at index 1: `x[1]:
Tree[list[int]] | None`.
## Test Plan
mdtest updated
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
- Preserve a constrained type variable when subscripting each declared
constraint returns that same constraint.
- Share alternative handling between unions and constrained type
variables while retaining existing diagnostics and recursive-union
behavior.
- Infer precise unions when slicing constrained tuples.
Mypy handles these constrained typevar situations by actually checking
the entire function once per constraint type, I think. We may eventually
decide to do the same, but for now we already have precedent for this
approach in binary ops inference, and it seems to work well.
Fixesastral-sh/ty#4226.
## Test plan
- Mdtests cover overlapping `list` and `Sequence` constraints, distinct
slice-preserving implementations, and legacy constrained `TypeVar`
declarations on Python 3.10.
- Mdtests retain errors for type-changing slices, upper-bounded
`Sequence` variables, and unsupported constrained alternatives.
- Existing tuple-slicing coverage verifies the newly precise union of
slice results.
## Summary
PEP 484's numeric tower means we represent a `float` annotation as the
semantic union `int | float*` and `complex` as `int | float* |
complex*`. We previously rendered those internal unions directly, which
exposed an implementation detail in user-facing output:
```python
def f() -> float:
return 1
reveal_type(f()) # Before: int | float; after: float
```
This change recognizes complete numeric-tower unions during display and
renders them using their canonical annotation spelling, `float` or
`complex`.
The implementation does _not_ preserve explicit unions, e.g., if the
user writes out `int | float`, we still collapse that rather than trying
to preserve it, mostly for simplicity (as per
https://github.com/astral-sh/ty/issues/2184#issuecomment-4345219164).
Closes https://github.com/astral-sh/ty/issues/2184.
## Summary
This is a first step towards #20411. After this, we would work to either
upstream changes or adjust things within Ruff.
Performance improvements have already been upstreamed
(https://github.com/rust-lang/annotate-snippets-rs/pull/437,
https://github.com/rust-lang/annotate-snippets-rs/pull/440)
Re-implemented the following from the fork:
- Hidden snippets
(https://github.com/rust-lang/annotate-snippets-rs/issues/448)
- Renderer-level disabling of hyperlinks
- Jupyter cell indices
(https://github.com/rust-lang/annotate-snippets-rs/issues/449)
- Fixable mark
(https://github.com/rust-lang/annotate-snippets-rs/issues/450)
- `lineno_offset` (maybe not needed if #20648 is taken care of)
- anonymized lines affects origin
(https://github.com/rust-lang/annotate-snippets-rs/issues/451)
- cut indicator (maybe not needed if we can switch to unicode renderer)
Additional hacks to minimize behavior change include
- Adjust upstream's hidden severity behavior to match Ruff's
(https://github.com/rust-lang/annotate-snippets-rs/issues/452)
- How newlines are annotated
(https://github.com/rust-lang/annotate-snippets-rs/issues/454)
- To not indent for line numbers when none will be present
(https://github.com/rust-lang/annotate-snippets-rs/issues/453)
- Enabling `simd` by default
All of the above represents work that needs to be resolved to address
#20411.
`lib.name` hack is used to simplify maintenance with the assumption that
we'll continue working towards #20411.
Behavior changes that were included
- Tweak when implicit padding lines are present
- Fix Origin's line/column is now derived from the primary annotation
- Make consistent the rendering of primary/context annotations
- Fix snippets to show up for empty source (but may degrade #24528)
- Fix handling of term width
- Fix rendering of annotations between newline and first character
(granted, maybe the caller is wrong)
- Tweak multiline annotation markers
New or upcoming features annotate-snippets features that we will be able
to leverage by moving this forward
- `AnnotationKind::Visible` to improve the quality of built-in
code-folding if Ruff were to switch to it
- `DecorStyle::Unicode`
- *(upcoming)* No-graphiscs mode for screen readers (may also be useful
for AIs)
Deferred:
- Moving related `Group`s into the same `Report`
- Using `secondary_title` for any `Group` after the first in a `Report`
- Using a `Snippet`, instead of titles, for "virtual code" like
https://github.com/astral-sh/ruff/pull/27033#discussion_r3622727236
- Switching hidden snippets to `Origin`
## Test Plan
Within cpython, I ran
```console
$ hyperfine -i "../../ruff/target-baseline/release/ruff check" "../../ruff/target-update/release/ruff check" -N -m20 --output=null
Benchmark 1: ../../ruff/target-baseline/release/ruff check
Time (mean ± σ): 27.5 ms ± 1.1 ms [User: 33.0 ms, System: 108.4 ms]
Range (min … max): 25.0 ms … 31.7 ms 94 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: ../../ruff/target-update/release/ruff check
Time (mean ± σ): 28.1 ms ± 2.0 ms [User: 33.4 ms, System: 105.7 ms]
Range (min … max): 24.8 ms … 38.9 ms 102 runs
Warning: Ignoring non-zero exit code.
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use t
he '--warmup' or '--prepare' options.
Summary
../../ruff/target-baseline/release/ruff check ran
1.03 ± 0.08 times faster than ../../ruff/target-update/release/ruff check
```
Before:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/2d94dabf-b28d-48c1-b17c-52aac59fd57e"
/>
After:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/b388ada3-e3d7-4ba0-b90b-1330eb81f4be"
/>
## Summary
Split `ty_extensions` into a public part (`ty_extensions`), and an
internal part (`ty_extensions._internal`). For now, move everything to
`_internal`, except for:
- `static_assert`
- `Unknown`
- `AlwaysTruthy`, `AlwaysFalsy`
- `Intersection`, `Not`
- `Top`, `Bottom`
- `JustFloat`, `JustComplex`
- `NamedTupleLike`
## Summary
This PR implements variable-length tuple (VLT) slicing (previously not
implemented and fell back to `@Todo`).
What makes this more complicated than fixed-length tuple slicing is that
VLT is divided into a prefix part, variable part, and suffix part. While
the length of the prefix/suffix part is fixed, the length of the
variable part is not, so element type unioning is required depending on
the slice. Also, the tuple after slicing may be of fixed-length.
Consider the following VLT:
````
t: tuple[P0, P1, P2, *tuple[V, ...], S0, S1, S2]
````
Tuples after slicing can be classified by the sign of the step, whether
it is a non-empty range, and whether it can be represented as a fixed
length or not. The decision tree is:
```mermaid
flowchart TD
A["VLT slicing"] --> B{"step == 0?"}
B -->|Yes| Z["error"]
B -->|No| C{"backward slice <br> (step < 0)?"}
C -->|Yes| R["reverse tuple and bounds"]
C -->|No| F["use tuple as-is"]
R --> F
F --> E{"always empty?"}
E -->|Yes| Empty["Empty"]
E -->|No| BP{"result length is fixed<br/>from front/back positions?"}
BP -->|Yes| FixedFront["Fixed"]
BP -->|No| M{"representable as one<br/>variable-length tuple?"}
M -->|Yes| Mixed["Mixed"]
M -->|No| H["Homogeneous"]
```
### Empty
```
t[5:2]: tuple[()]
```
### Fixed
The slice result can be expressed as a fixed length.
#### Exact
Since the range required by the slice completely fits within the
prefix/suffix, it can be represented as a fixed-length tuple.
```
t[:2]: tuple[P0, P1]
t[-2:]: tuple[S1, S2]
```
#### Approximation
Since the total length required by the slice is shorter than the fixed
length (prefix length + suffix length), it can be represented as a fixed
length tuple, although it may contain elements of the variable part.
```
t[:4]:
if variable length == 0: tuple[P0, P1, P2, S0]
if variable length > 0: tuple[P0, P1, P2, V]
=> tuple[P0, P1, P2, V | S0]
```
### Mixed
The slice result cannot be expressed as a fixed length.
#### Exact
It is possible to separate the fixed length part and variable length
part, and it can be expressed by cutting a part of the original VLT.
```
t[:]: tuple[P0, P1, P2, *tuple[V, ...], S0, S1, S2]
t[1:]: tuple[P1, P2, *tuple[V, ...], S0, S1, S2]
t[:-1]: tuple[P0, P1, P2, *tuple[V, ...], S0, S1]
```
#### Approximation
The prefix/suffix part and the variable part cannot be separated, and
some parts are fused.
```
t[4:]:
if variable length == 0: tuple[S1, S2]
if variable length == 1: tuple[S0, S1, S2]
if variable length == 2: tuple[V, S0, S1, S2]
...
=> tuple[*tuple[V | S0, ...], S1, S2]
t[::2]:
if variable length == 0: tuple[P0, P2, S1]
if variable length == 1: tuple[P0, P2, S0, S2]
if variable length == 2: tuple[P0, P2, V, S1]
...
=> tuple[P0, P2, *tuple[V | S0 | S1 | S2, ...]]
```
### Homogenous
Since none of the above patterns apply, it falls back to a homogenous
tuple that is the union of all elements.
```
t[2147483647::-1]: tuple[S2 | S1 | S0 | V | P2 | P1 | P0, ...]
```
## Caveat
This PR assumes that the type of the sliced VLT will be a single tuple
type. However, sometimes a union of tuples is more precise than a single
tuple type:
```
t[:4]: tuple[P0, P1, P2, V | S0]
t[:5]: tuple[P0, P1, P2, V | S0, V | S0 | S1]
```
There is no problem with a single tuple type up to `t[:4]`. But in the
case of `t[:5]`, it is more precise to divide it into a union.
```
t[:5]:
if variable length == 0: tuple[P0, P1, P2, S0, S1]
if variable length == 1: tuple[P0, P1, P2, V, S0]
if variable length > 2: tuple[P0, P1, P2, V, V]
=> tuple[P0, P1, P2, S0, S1] | tuple[P0, P1, P2, V, S0 | V]
```
However, I think that such a union division should not be introduced at
this time. This is because ty does not yet have an implementation for
strict normalization, subtyping, and equality of tuples when they
contain unions (see astral-sh/ty#493). Until it is implemented, it is
safe to assume that a sliced VLT type is still a single tuple type.
## Test Plan
mdtest updated
## Summary
A slice with a step size of zero always fails for an exact `list`, but
we currently only report `zero-stepsize-in-slice` for string, bytes, and
tuple literals:
```python
values = list(range(10))
result = values[1:10:0] # error: [zero-stepsize-in-slice]
reveal_type(result) # list[int]
```
We now report the diagnostic when the receiver type is `list` or another
known built-in while preserving the inferred result type for error
recovery.
(A hypothetical subclass could override `__getitem__` to accept a zero
step, but the `list` type includes exact `list` instances that are known
to reject it. Subclasses with their own nominal type and custom sequence
types continue through normal subscript dispatch without this
diagnostic.)
Closes https://github.com/astral-sh/ty/issues/3736.
## Summary
Add a new lint that detects generic types used without explicit type
parameters in annotations (e.g. `list` instead of `list[int]`).
This is equivalent to mypy's `--disallow-any-generics` and pyright's
`reportMissingTypeArgument`.
Can be enabled via:
```
[rules]
missing-type-argument = "warn"
```
It skips classes where all type parameters have PEP 696 defaults.
This should fix the following issue in ty:
- https://github.com/astral-sh/ty/issues/2442
## Test Plan
Added tests covering stdlib generics, user-defined generics, PEP 696
defaults, dotted names, function signatures, base classes, and variable annotations.
---------
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
## Summary
The way we special-cased `sys.version_info` accidentally also treated
`sys.implementation.version` as always being the Python version, but
this is only true for CPython. This was because both are instances of
`sys._version_info` class in typeshed, and our special-casing operated
at the level of that class.
Adjust things so that class is no longer treated as a singleton, and our
special-casing operates at the instance layer rather than the class
layer.
This also allows removing the now-unused
`ExpandedClassBaseEntry::Synthetic` variant, which allows simplifying
some diagnostic code.
Closes https://github.com/astral-sh/ty/issues/3640
## Testing
Added mdtest.
## Summary
Prior to this change, when a slice bound was not an integer literal,
boolean literal, or `None`, we inferred a bare `slice[Any, Any, Any]`.
This erased enough information that subscription checking accepted
invalid slices even when `__getitem__` requires `SupportsIndex | None`
bounds:
```python
def get(values: list[int], start: float) -> list[int]:
return values[start:] # error: [invalid-argument-type]
```
(This may have been waiting on
https://github.com/astral-sh/ruff/commit/cb58704e085fe7e70d72842294575de47e47bead?)
We now preserve the inferred type of each provided slice bound in
`slice[...]`, so ordinary argument checking diagnoses invalid bounds for
built-in sequences and user-defined `__getitem__` methods with typed
slice parameters.
Closes https://github.com/astral-sh/ty/issues/3547.
## Summary
Like Mypy and Pyright, we now ignore annotations on non-name (i.e.,
subscript and attribute) targets, and emit a diagnostic for the invalid
annotation -- with the exception of annotated assignments on `self` and
class attributes (i.e., receivers).
Closes https://github.com/astral-sh/ty/issues/509.
## Summary
Previously, narrowing an enum by excluding one or more members eagerly
expanded the enum into a union of all remaining enum literals. That's
fine for small enums, but can cause large enums to blow up during
repeated equality or membership narrowing. And large enums are not
uncommon...
This PR keeps those narrowed types in a compact form, like `Color &
~Literal[Color.RED]`, instead of immediately expanding them to
`Literal[Color.GREEN, Color.BLUE, ...]`. The type is expanded only when
we need to inspect the remaining enum members individually (like for
overload matching or attribute lookup). The net effect is that we retain
the existing behavior while avoiding a brutal performance penalty for
large enums.
Note that we do vary some behavior by enum size. For example, with more
than 7 elements, we use (as you can see in the ecosystem diff) the
compact representation (`SupportedBlockchain &
~Literal[SupportedBlockchain.ETHEREUM_BEACONCHAIN`).
For the benchmark in the linked issue, `main` doesn't finish within 30s,
but this PR completes in 128ms.
Closes https://github.com/astral-sh/ty/issues/1588.
---------
Co-authored-by: Codex <noreply@openai.com>
## Summary
Part of https://github.com/astral-sh/ty/issues/1240
Stop unioning `Unknown` into the types of un-annotated container
literals.
We discussed perhaps continuing to union `Unknown` if the inferred type
is a singleton type like `None`. I'd like to explore this as a separate
change so we can see the ecosystem impact more clearly.
## Test Plan
Adjusted many mdtest expectations.
There's one test case that regresses with this change, because we don't
fully support union type contexts (it can require a lot of repeat
inference in pathological cases). So `x10: list[int | str] | list[int |
None] = [1, 2, 3]` previously passed only because we inferred the RHS as
`list[Unknown | int]` -- now we infer it as `list[int]` and the
assignment fails due to invariance. I've kept this test as a TODO since
it's not trivial to fix. Mypy errors in the same way we now do,
suggesting it's not necessarily a huge priority either.
## Ecosystem
This change is expected to cause new diagnostics and some false
positives, since we are replacing very-forgiving gradual types with
non-gradual inference heuristics.
Many of these issues could be solved or significantly mitigated by
https://github.com/astral-sh/ty/issues/1473, depending how far we are
able to go with that, and particularly whether we can afford to apply it
also to container literals which are not empty at construction. The
downside of broad application of this approach is that in some cases it
could cause us to widen container types when the user actually just made
a mistake and added the wrong thing to a container, and would prefer an
error at that location.
Some categories of new error that show up in the ecosystem report:
### Implicit TypedDicts
These are cases where the dictionary is heterogeneous and would ideally
be typed as a `TypedDict` but isn't, for example:
```py
def make_person(photo: bytes | None):
person = {"name": "Pat", age: 29}
if photo is not None:
person["photo"] = photo
```
We (and pyrefly, and pyright in strict mode) error on the last line here
because we already inferred `dict[str, str | int]`, so we can't add a
`bytes` value.
Mypy prefers common-base joins over union joins, so it infers `dict[str,
object]`, which avoids the error adding a `bytes` value. This means the
value type is less precise, which theoretically means potentially more
errors using values from the dict later. But in practice with this
heterogeneous pattern, either `object` or the union will cause similar
problems when using values from the dict -- in either case you'd
probably have to cast or narrow.
Pyright (in non-strict mode) has a special case where it falls back to
`Unknown` when it sees heterogenous value types, so it infers this as
`dict[str, Unknown]`.
I think we could consider either the mypy or pyright approaches here,
but we don't need to do it in this PR; we can file an issue and consider
it as a follow-up.
Another symptom of this same root cause is repetitive diagnostics
arising from a large union inferred as value type; the same fixes would
address this.
### Negative intersections, particularly with e.g. `~AlwaysFalsy` or
`~None`.
Example:
```py
class A: ...
def _(a: A | None) -> dict[str, A]:
if a:
d = {"a": a}
return d
return {}
```
We error on `return d` because "expected `dict[str, A]`, found
`dict[str, A & ~AlwaysFalsy]`". This is an issue specific to
intersection types, so no other type checker has this problem.
I think when we "promote literals" (we may need to give this operation a
broader name -- it's really "type promotion to give a better inferred
type when invariance means too-precise is bad") we should also eliminate
all negative types from intersections. I would prefer to do this as a
separate PR for easier review and better visibility of ecosystem impact,
but I think it's high priority to land soon after this PR (ideally
before a release).
### Overly-precise inference for singleton `None`
This did show up, to the tune of ~100 new diagnostics
([example](https://github.com/pytorch/ignite/blob/b73a4c20e991b3e14949f2a69651ed2a7219f2fd/tests/ignite/engine/test_engine.py#L158)),
so I think it is worth addressing as a follow-up.
## Summary
For a couple of diagnostics currently, we add a "Did you mean...?"
diagnostic hint if it appears like there's an obvious typo that caused
us to emit an error. The "Did you mean...?" suggestion is generated via
the `strsim` Levenshtein implementation on `crates.io`.
This PR replaces the `strsim` implementation of Levenshtein used to
create these hints with a custom Levenshtein implementation based on the
one that CPython itself uses to create these hints:
```pycon
>>> class Foo:
... xyxy = 42
...
>>> Foo.xyxyz
Traceback (most recent call last):
File "<python-input-1>", line 1, in <module>
Foo.xyxyz
AttributeError: type object 'Foo' has no attribute 'xyxyz'. Did you mean: 'xyxy'?
```
The added tests are also derived from CPython's test suite.
The motivation for copying CPython's implementation almost exactly is
that CPython has had this feature for several Python versions now, and
during that time many bug reports have been filed regarding incorrect
suggestions, which have since been fixed. This implementation is thus
very well "battle-tested" by this point; we can say with a reasonable
degree of confidence that it gives good suggestions for typos in the
Python context.
The ecosystem report on this PR bears out that this is an improvement.
We see bad suggestions going away:
```diff
- [error] invalid-key - Unknown key "pair" for TypedDict `RPCAnalyzedDFMsg` - did you mean "data"?
+ [error] invalid-key - Unknown key "pair" for TypedDict `RPCAnalyzedDFMsg`: Unknown key "pair"
```
and good suggestions being added:
```diff
- [error] invalid-key - Unknown key "old_entity_id" for TypedDict `_EventEntityRegistryUpdatedData_CreateRemove`: Unknown key "old_entity_id"
+ [error] invalid-key - Unknown key "old_entity_id" for TypedDict `_EventEntityRegistryUpdatedData_CreateRemove` - did you mean "entity_id"?
```
This Levenshtein implementation was originally proposed in #18705, and
then again in #18751. Those PRs also made other changes to use the
Levenshtein implementation in certain other areas, however, where
computing the list of suggestions to pass into the Levenshtein algorithm
turned out to be prohibitively expensive. This PR therefore _only_
updates the Levenshtein implementation being used for our existing
subdiagnostics, rather than expanding the callsites of the Levenshtein
implementation.
## Test plan
Unit tests have been added in `levenshtein.rs`. Some mdtests and
snapshots were updated to ensure that they still test what they're meant
to be testing, even with the new Levenshtein implementation.
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
- Updated the `prek` hook to replace `blacken-docs` with ruff
- Added some formatting directives to preserve obvious intentional
formatting choices
- Made some choice edits to content for better readability with new format
- Formatted md tests and updated snapshots
## Summary
I think `__class_getitem__` is a bit different than (e.g.)
`__contains__` because it's called on the class itself, not instances...
So given `Identity[0]` where `Identity` is a class, using
`try_call_dunder` would call `member_lookup_with_policy` with
`NO_INSTANCE_FALLBACK`, which would look at the metaclass, but not
`Identity` itself.
I added `try_call_dunder_on_class` for consistency which calls `member`
(without `NO_INSTANCE_FALLBACK`), though as far as this PR is concerned,
I think our current behavior was ok?
Closes https://github.com/astral-sh/ty/issues/190.
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
Fixes https://github.com/astral-sh/ty/issues/2015. We weren't recursing
into the value of a type alias when we should have been.
There are situations where we should also be recursing into the
bounds/constraints of a typevar. I initially tried to do that as well in
this PR, but that seems... trickier. For now I'm cutting scope; this PR
does, however, add several failing tests for those cases.
## Test Plan
added mdtests
## Summary
Follow up from https://github.com/astral-sh/ruff/pull/21411. Again,
there are more things that could be improved here (like the diagnostics
for `lists`, or extending what we have for `dict` to `OrderedDict` etc),
but that will have to be postponed.
## Summary
Currently our diagnostic only covers the range of the thing being
subscripted:
<img width="1702" height="312" alt="image"
src="https://github.com/user-attachments/assets/7e630431-e846-46ca-93c1-139f11aaba11"
/>
But it should probably cover the _whole_ subscript expression (arguably
the more "incorrect" bit is the `["foo"]` part of this expression, not
the `x` part of this expression!)
## Test Plan
Added a snapshot
Co-authored-by: Brent Westbrook
<36778786+ntBre@users.noreply.github.com>
## Summary
Further improve subscript assignment diagnostics, especially for
`dict`s:
```py
config: dict[str, int] = {}
config["retries"] = "three"
```
<img width="1276" height="274" alt="image"
src="https://github.com/user-attachments/assets/9762c733-8d1c-4a57-8c8a-99825071dc7d"
/>
I have many more ideas, but this looks like a reasonable first step.
Thank you @AlexWaygood for some of the suggestions here.
## Test Plan
Update tests
## Summary
We synthesize a (potentially large) set of `__setitem__` overloads for
every item in a `TypedDict`. Previously, validation of subscript
assignments on `TypedDict`s relied on actually calling `__setitem__`
with the provided key and value types, which implied that we needed to
do the full overload call evaluation for this large set of overloads.
This PR improves the performance of subscript assignment checks on
`TypedDict`s by validating the assignment directly instead of calling
`__setitem__`.
This PR also adds better handling for assignments to subscripts on union
and intersection types (but does not attempt to make it perfect). It
achieves this by distributing the check over unions and intersections,
instead of calling `__setitem__` on the union/intersection directly. We
already do something similar when validating *attribute* assignments.
## Ecosystem impact
* A lot of diagnostics change their rule type, and/or split into
multiple diagnostics. The new version is more verbose, but easier to
understand, in my opinion
* Almost all of the invalid-key diagnostics come from pydantic, and they
should all go away (including many more) when we implement
https://github.com/astral-sh/ty/issues/1479
* Everything else looks correct to me. There may be some new diagnostics
due to the fact that we now check intersections.
## Test Plan
New Markdown tests.
## Summary
Add (snapshot) tests for subscript assignment diagnostics. This is
mainly intended to establish a baseline before I hope to improve some of
these messages.
## Summary
Part of https://github.com/astral-sh/ty/issues/168. Infer more precise types for collection literals (currently, only `list` and `set`). For example,
```py
x = [1, 2, 3] # revealed: list[Unknown | int]
y: list[int] = [1, 2, 3] # revealed: list[int]
```
This could easily be extended to `dict` literals, but I am intentionally limiting scope for now.
## Summary
Adds validation to subscript assignment expressions.
```py
class Foo: ...
class Bar:
__setattr__ = None
class Baz:
def __setitem__(self, index: str, value: int) -> None:
pass
# We now emit a diagnostic on these statements
Foo()[1] = 2
Bar()[1] = 2
Baz()[1] = 2
```
Also improves error messages on invalid `__getitem__` expressions
## Test Plan
Update mdtests and add more to `subscript/instance.md`
---------
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: David Peter <mail@david-peter.de>
Summary
--
Fixes#19640. I'm not sure these are the exact fixes we really want, but
I
reproduced the issue in a 32-bit Docker container and tracked down the
causes,
so I figured I'd open a PR.
As I commented on the issue, the `goto_references` test depends on the
iteration
order of the files in an `FxHashSet` in `Indexed`. In this case, we can
just
sort the output in test code.
Similarly, the tuple case depended on the order of overloads inserted in
an
`FxHashMap`. `FxIndexMap` seemed like a convenient drop-in replacement,
but I
don't know if that will have other detrimental effects. I did have to
change the
assertion for the tuple test, but I think it should now be stable across
architectures.
Test Plan
--
Running the tests in the aforementioned Docker container