ty could panic when checking a decorated class if it inferred the return
type of an inner decorator and an outer decorator returned a class from
another module. During cycle recovery, ty could use that replacement
class's scope to look up names in the file that defined the original
class.
Preserve the original class before applying decorators and use it for
class-definition checks. The decorated name still takes the type
returned by the outer decorator.
Fixesastral-sh/ty#5021.
## Test plan
Checked the reported `EnumCheck` example with return type inference
enabled. Before this change it panicked. With the fix, ty reports the
invalid argument to `@d2` and finishes checking the file.
## Summary
Given a `Movie` TypedDict with a `title` field, ty currently reports
`invalid-key` for three cases. This PR gives each its own rule.
| Example | Rule |
| --- | --- |
| `movie["titel"]` reads a misspelled key | `invalid-typed-dict-key` |
| `movie[key]` reads a key whose type is `str` and whose value is
unknown | `non-literal-typed-dict-key` |
| `Movie(title="Alien", titel="Alien")` supplies an unknown key during
construction | `unknown-typed-dict-constructor-key` |
Both subscript rules apply to reads and assignments. An invalid
non-string key such as `movie[0]` reports `invalid-typed-dict-key`. The
constructor rule reports the extra `titel` key in `movie: Movie =
{"title": "Alien", "titel": "Alien"}`. For a closed TypedDict, it flags
`{key: value}` when `key` has type `str` and might introduce an extra
item.
Projects migrating to ty lose typo detection if they disable
`invalid-key` to allow dynamic TypedDict lookups. In mypy, disabling
`literal-required` allows `movie[key]` while `movie["titel"]` still
produces a `typeddict-item` error. Pyright allows the dynamic lookup by
default and also flags the typo. To keep that behavior in ty, set
`non-literal-typed-dict-key` to `ignore` and leave
`invalid-typed-dict-key` enabled.
All three rules default to `error`. With the alias handling from #29960,
`invalid-key` maps to `invalid-typed-dict-key` in configuration, inline
ignores, and existing baselines. Previously baselined diagnostics can
therefore reappear under either of the other two rule names. To preserve
a custom severity across the split, set all three rules to the same
level. For inline ignores, use the rule or rules reported on that line.
---------
Co-authored-by: David Peter <mail@david-peter.de>
Use `# ok` as the marker for mdtest statements whose expectation is that
no diagnostic is emitted. Update the guidance in `AGENTS.md` and replace
existing `# no diagnostic` comments in Ruff and ty mdtests.
## Summary
Generalize/promote types in inferred return types that refer to local
nested definitions.
For example, consider this unannotated decorator:
```python
def register(event: str):
def wrapper(func):
# ...
return func
return wrapper
```
We do not want to infer `def wrapper(func) -> Unknown` as the return
type of `register`. This would not just expose internal details, it
would also incorrectly suggest that each call to `register` returns the
same function object. Instead, we generalize to an anonymous `(func) ->
Unknown` callable.
For nested classes, we cannot easily do the same, so we return `Unknown`
instead. In the future, to improve this, we might want to synthesize a
protocol that has the same attributes and methods as the nested class.
## Test strategy
Updated existing mdtests
## Summary
<img width="331" height="155" alt="image"
src="https://github.com/user-attachments/assets/8e1b5c40-57b7-44ec-a197-3770e9fc3236"
align=right />
This PR adds return type inference as a new feature to ty. The feature
is currently *disabled by default*. Users can activate the feature by
setting `analysis.return-type-inference` to `"always"`.
In a standalone function, the inferred return type is simply the union
of the types of all reachable `return`ed expressions, as shown in the
image on the right. If control flow can reach the end of the function
scope, we add `None` to the union. In a (non-final) method on a
(non-final) class, we additionally union with `Unknown` to allow child
classes to widen the intended return type. For example, imagine
```py
class TaskBase:
def compute(self):
return None
class IntTask(TaskBase):
def compute(self):
return 42
```
Here, we infer a return type of `None | Unknown` for `TaskBase.compute`.
Otherwise, if we inferred `None`, we would prohibit child classes from
returning something else, which seems undesirable. Another interesting
scenario is when the base class method is annotated, but the child class
method is not:
```py
class Base:
def method(self) -> int:
return 0
class Child(Base):
def method(self):
return 1
```
Here, we respect the base class definition and infer `Child.method` as
returning `int`, not `int | Unknown`.
This work is based on https://github.com/astral-sh/ruff/pull/17371 by
@mtshiba (thank you!)
There are a few notable things that are _not_ implemented here:
- The type of the function itself currently stays unchanged, that is: we
do not infer a signature with the inferred return type. Concretely:
`my_function()` may be inferred as `int`, but `my_function` itself would
still be inferred as `def my_function() -> Unknown`. We have existing
tests with TODOs for this and I plan to tackle this in a follow up.
- We do not yet have return type inference for generator functions or
async functions
- We do not attempt to solve the problem of "escaping" inner types here
(see tests with TODOs)
- We do not add LSP/playground features related to this feature, but
plan to do so soon (display return type hints).
- There is a limitation around `functools.wraps` that's present in the
ecosystem report. This will hopefully be solved by
https://github.com/python/typeshed/pull/16515
- There is a [limitation around
`NotImplemented`](https://github.com/astral-sh/ty/issues/1150#issuecomment-6140701931)
which hasn't been resolved yet. The solution is described
[here](https://github.com/astral-sh/ty/issues/1150#issuecomment-6147899750).
- Fixpoint iteration currently uses `Divergent` as a provisional value.
I did think about using the new μ-types, but that currently leaves too
many questions unanswered: How would we represent an anonymous μ-type?
How would the (provisional) recursive marker `R` interact with other
types during inference (especially since those operations require
"closed" recursive types, and `R` would still be open)?
closes https://github.com/astral-sh/ty/issues/128
## Test Plan
- Changed assertions in the previously reviewed & merged mdtests
- Studied the ecosystem impact and addressed many problems
---------
Co-authored-by: Shunsuke Shibayama <sbym1346@gmail.com>
## Summary
Introduce a new configuration option `analysis.return-type-inference`
which can be used to control the return type inference feature, which
we'll introduce in #29253. I will not merge this PR without the one on
top, but it seemed like a good idea to split it out for easy review.
## Summary
Establish the behavior that we want for return type inference. I added
inline comments for the things that seem potentially worth discussing.
towards https://github.com/astral-sh/ty/issues/128
## Summary
We now infer and validate variance through direct, mutual, and expanding
recursive protocol references. A recursive member no longer suppresses
the variance mismatch in this protocol:
```python
from typing import Protocol, TypeVar
T_co = TypeVar("T_co", covariant=True)
class Sink(Protocol[T_co]): # error: should be contravariant
def write(self, value: T_co) -> None: ...
def next(self) -> "Sink[T_co]": ...
```
We separate constructing variance equations from evaluating them.
Recursive references name another parameter’s equation instead of
expanding its definition, so even references like `P[list[T]]` produce a
finite problem.
To validate a declaration, we solve mutually dependent parameters
together, starting at bivariance and updating until the results stop
changing. This prevents a protocol’s declared variance from serving as
evidence for its own correctness. Declarations outside that recursive
group remain authoritative. Dependencies also respect argument erasure:
an alias that ignores its type argument does not create a recursive
dependency through that argument. Ordinary type inference continues to
honor declared variance, including for unused parameters.
The symbolic terms and fixed-point iteration follow the general approach
described in the [Rust compiler’s variance-inference
documentation](https://rustc-dev-guide.rust-lang.org/variance.html#the-algorithm),
adapted here to validating Python protocol declarations.
This also enables structural variance inference for recursive PEP 695
protocols, including invariance from writable attributes. Recursive type
aliases and descriptor setters whose accepted values cannot be
represented by a single type remain deferred.
## Summary
Follow-up to #27534.
We now validate explicitly declared generic protocol variance against
its structural interface:
```python
from typing import Protocol, TypeVar
T = TypeVar("T")
class Source(Protocol[T]): # error: [invalid-protocol]
def read(self) -> T: ...
```
A type variable appearing only in a return position must be covariant,
while writable protocol attributes must remain invariant. We derive both
declared and inferred variance from the same member read/write
capabilities already used for structural compatibility, including
inferred legacy and PEP 695 protocol parameters:
```python
class Writable[T](Protocol):
_value: T
def overwrite(value: Writable[object]) -> None:
value._value = object()
def unsound(value: Writable[int]) -> None:
overwrite(value) # error: [invalid-argument-type]
```
For now, I've limited this to the easier cases, so recursive interfaces,
custom descriptors with unrepresentable setter domains, generic
defaults, and inherited or malformed headers are unchanged.
## Summary
- Treat generic aliases as fully static when their actual type arguments
are fully static, even if their type parameters have gradual bounds,
constraints, or defaults.
- Visit only a generic alias's concrete specialization when checking
dynamic content, matching the existing protocol-specific traversal.
- Cover modern and legacy generic declarations, generator return
statements, and genuinely gradual specializations.
This fixes the false negatives for the `unsound-*` rules identified in
https://github.com/astral-sh/ty/issues/4238.
## Summary
In this PR, I explored the proposal in
https://github.com/astral-sh/ty/issues/3251. The idea was to treat a
function like `def f(x="foo"): ...` as if the `x` parameter was declared
as `str`. This is what pyright does, and would follow what we do for
unannotated class attributes with default values. However, implementing
this showed a huge ecosystem impact of 8700 new diagnostics.
I tried various things to improve this. For example, I promoted `None`
to `Unknown | None` because it seems unreasonable to assume that you
only ever want callers to pass in `None`. Then there's a lot of code
that takes `x=()` as a default value and wants callers to be able to
pass in arbitrary tuples. I tried patching that by promoting `tuple[()]
-> tuple[Unknown, ...]`.
But the biggest problem is with functions that take a default int value
(e.g. `x=0`), but want to allow their callers to pass in float values. I
tried promoting the type of `x` from `int` to `float`/`complex`, but
that leads to the opposite problem: now the function body of *every*
function that takes `x=0` must also handle the possibility that `x` is
`float`/`complex`.
Ultimately, I think a scenario like this:
```py
def some_computation(..., tolerance=0): ...
some_computation(..., tolerance=1e-3)
```
is just far more common than the corresponding scenario with class
attributes.
I think it's unfortunate that our behavior is inconsistent with what we
do for class attributes, but I don't think it's worth introducing ~8000
diagnostics (no variant that I tried went below that limit). The
corresponding PR for attributes introduced an order of magnitude fewer
new diagnostics (https://github.com/astral-sh/ruff/pull/24531).
So in this PR, all I do is to document the current behavior.
closes https://github.com/astral-sh/ty/issues/3251
## Summary
Add a new lint that detects generic types used without explicit type
parameters in annotations (e.g. `list` instead of `list[int]`).
This is equivalent to mypy's `--disallow-any-generics` and pyright's
`reportMissingTypeArgument`.
Can be enabled via:
```
[rules]
missing-type-argument = "warn"
```
It skips classes where all type parameters have PEP 696 defaults.
This should fix the following issue in ty:
- https://github.com/astral-sh/ty/issues/2442
## Test Plan
Added tests covering stdlib generics, user-defined generics, PEP 696
defaults, dotted names, function signatures, base classes, and variable annotations.
---------
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
## Summary
With this change, we now record reachability constraints for certain
*text ranges* in a given scope. Using text ranges (as opposed to AST
nodes) simplifies the implementation and allows us to easily record
reachability for basic blocks by simply extending the range:
```py
print("hello") # record reachability for the full text range of this statement
print("world") # reachability hasn't changed => extend the range to also include this statement
some_call() # records a new reachability statement (this call could fail)
1 + "a" # reachabilty has changed, so create a new range and track it's reachability
x: int = "foo" # reachability is still the same, so extend the previous range
```
This mostly works on the statement level, but there are some special
syntactic constructs which require us to also track single expressions,
like ternary if statements. Here, we record reachability for both the `1
+ "a"` expression and the `"this is fine"` expression:
```py
1 + "a" if False else "this is fine"
```
Combining this range-based reachability with the pre-existing
scope-based reachability allows us to unconditionally silence
diagnostics in unreachable code (with the exception of `reveal_type`
diagnostics, which we still want to emit).
fixes https://github.com/astral-sh/ty/issues/2891
fixes https://github.com/astral-sh/ty/issues/1165
## Performance
This is actually a (small) win in terms of both performance and memory!
## Ecosystem
Looks great. Lots of removed diagnostics in `if not TYPE_CHECKING`
blocks.
## Conformance
Well, this test is somewhat controversial, but we now pass it.
## Test Plan
Updated tests
## Summary
Perform return type checking for generator functions.
part of https://github.com/astral-sh/ty/issues/1718
## Test Plan
The two newly added diagnostics in ecosystem result are correct the rest
were marked as flaky. So looks good.
I updated the tests for `return_types.md`.
---------
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
## Summary
* Pull out the test-only changes from
https://github.com/astral-sh/ruff/pull/23849 (with 2 TODOs for now)
* A few more preparations for when we silence all diagnostics in
unreachable code
- Updated the `prek` hook to replace `blacken-docs` with ruff
- Added some formatting directives to preserve obvious intentional
formatting choices
- Made some choice edits to content for better readability with new format
- Formatted md tests and updated snapshots
## Summary
Fixes https://github.com/astral-sh/ty/issues/2161
* Infer parameter default expressions using the parameter annotation as
type context.
* Fix `invalid-parameter-default` false positives for `TypedDict`
defaults (literals `{...}` and calls `dict(...)`).
* Add mdtests covering valid and invalid `TypedDict` defaults plus a
non-`TypedDict` assignability case.
## Test Plan
* Assert no diagnostics for `Foo` defaults via `{"x": 42}` and
`dict(x=42)`
* Assert `invalid-parameter-default` for missing key, wrong value type,
extra key in `Foo` defaults and non-assignable default (`tuple[int] =
()`)
## Summary
Split the "Generator functions" tests into two parts. The first part
(synchronous) refers to a function called `i` from a function `i2`. But
`i` is later redeclared in the asynchronous part, which was probably not
intended.
## Summary
Resolves https://github.com/astral-sh/ty/issues/339
Supports having a blank function body inside `if TYPE_CHECKING` block or
in the elif or else of a `if not TYPE_CHECKING` block.
```py
if TYPE_CHECKING:
def foo() -> int: ...
if not TYPE_CHECKING: ...
else:
def bar() -> int: ...
```
## Test Plan
Update `function/return_type.md`
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Partially implement https://github.com/astral-sh/ty/issues/538,
```py
from pathlib import Path
def setup_test_project(registry_name: str, registry_url: str, project_dir: str) -> Path:
pyproject_file = Path(project_dir) / "pyproject.toml"
pyproject_file.write_text("...", encoding="utf-8")
```
As no return statement is defined in the function `setup_test_project`
with annotated return type `Path`, we provide the following diagnosis :
- error[invalid-return-type]: Function **always** implicitly returns
`None`, which is not assignable to return type `Path`
with a subdiagnostic :
- note: Consider changing your return annotation to `-> None` or adding a `return` statement
## Test Plan
mdtests with snapshots to capture the subdiagnostic. I have to mention
that existing snapshots were modified since they now fall in this
category.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>