## Summary
This PR splits `mismatched-type-name` into eight independently
configurable warnings for `NewType`, functional enums, `TypeVar`,
`ParamSpec`, `TypeVarTuple`, `namedtuple`/`NamedTuple`, `TypedDict`, and
`TypeAliasType`.
~~This PR is actually quite a bit less breaking for users than other
rule splits we've done over the last few days. Because
`mismatched-type-name` is no longer the canonical name for _any_ rule on
this branch, we're able to make it a deprecated alias for _all_ the new
rules, meaning that it will continue to work on the CLI, in ty
configuration files, and ty suppression comments (we'll just emit a
warning diagnostic telling users to upgrade their configuration
file/suppression comment).~~
~~A more specific rule takes precedence over the deprecated name if both
are given within the same configuration table. On the command line, the
last matching flag wins (e.g. `--error=mismatched-enum-name
--ignore=mismatched-type-name` will suppress diagnostics for mismatched
enum names).~~
(After review from @MichaReiser, I backed out these changes. It's now a
somewhat breaking change, because the rule is now marked as "removed"
rather than "deprecated".)
As well as splitting the rule into eight separate rules, it also cleans
up our error recovery for several of these type factories. For example,
in the following snippet, we fail to recognize `T` as a `TypeVar` due to
bad error recovery from the non-literal name. But we don't really need a
nonliteral name in this case to determine that it's a validly
constructed TypeVar. It's probably a mistake, but there's no _soundness_
issue here:
```py
from typing import TypeVar
def f(x: str):
# main emits an error: `invalid-typevar`
# this PR emits a warning: `mismatched-typevar-name`
T = TypeVar(x)
def g(x: T) -> T:
return x
reveal_type(g(42)) # main: Todo. This PR: `Literal[42]`.
```
## Motivation
The main motivation here is that having a namedtuple or enum with the
"wrong name", for example, just seems much less "obviously wrong" than
having a typevar or paramspec with the "wrong name". The [ecosystem
report](https://github.com/astral-sh/ruff/pull/29957#issuecomment-6603964612)
is evidence of this. namedtuples and enums define runtime classes that
can be used in runtime contexts and instantiated; these constructs
aren't _just_ for type checkers.
TypeVars/ParamSpecs/TypeVarTuples/NewTypes/TypedDicts are all _only_ for
type checkers and there's no plausible reason why you'd want to give the
type a different name to the variable it's assigned to.
`mismatched-namedtuple-name` and `mismatched-enum-name` are therefore
now disabled by default, matching the default severity we already
settled on for `mistmatched-sentinel-name` in
https://github.com/astral-sh/ruff/pull/27206. `mismatched-typevar-name`,
`mismatched-paramspec-name`, `mismatched-typevartuple-name`,
`mismatched-enum-name`, `mismatched-namedtuple-name` and
`mismatched-newtype-name` all remain enabled by default with `warning`
severity.
19 KiB
Cycles
Recursive lambda in a loop condition
A lambda is always truthy. Determining whether the final assignment is reachable must not require inferring the lambda's return type, which depends on that same assignment.
(f := lambda: f)
while lambda: f: # error: [redundant-condition] "always truthy"
pass
f = 0
Recursive lambda in a conditional
The same cycle can arise when a conditional filters the bindings visible to a recursive lambda.
f = lambda: f
if not (lambda: f): # error: [redundant-condition] "always truthy"
f = 0
Function signature
Deferred annotations can result in cycles in resolving a function signature:
from __future__ import annotations
# error: [invalid-type-form]
def f(x: f):
pass
reveal_type(f) # revealed: def f(x: Unknown) -> Unknown
A TypeVar name depends on its own __name__
The name passed to TypeVar comes from T.__name__ in another module. Inferring T.__name__ leads
back to the name argument, so ty falls back to str for the attribute.
parameter.py:
from typing import TypeVar
from typing_extensions import reveal_type
from names import name
T = TypeVar(name) # error: [mismatched-typevar-name]
reveal_type(T.__name__) # revealed: str
names.py:
from parameter import T
name = T.__name__
Unpacking a recursively growing tuple
This is a regression test for https://github.com/astral-sh/ty/issues/3838.
while 1:
# error: [possibly-unresolved-reference]
# error: [possibly-unresolved-reference]
x = (*x, x)
while 1:
y = (y, *y)
Concatenating a recursively growing tuple
A loop can append any number of elements. The inferred tuple becomes variable-length instead of accumulating a new fixed-length alternative on each inference iteration.
def repeat(flag: bool) -> None:
value = ()
while flag:
value += (1,)
reveal_type(value) # revealed: tuple[Literal[1], ...]
Alternating between fixed-length tuples
A loop can alternate between a finite set of tuple shapes without growing the tuple. The inferred type preserves those shapes:
def alternate(flag: bool):
value = (1,)
while flag:
value = ("a", "b") if len(value) == 1 else (1,)
reveal_type(value) # revealed: tuple[Literal[1]] | tuple[Literal["a"], Literal["b"]]
Unpacking a value with a growing type argument
Each time through the loop, the first unpacked value has another callable nested in its type.
Inference falls back to Unknown for that value. The initial Box[int] also reaches the read after
the loop.
[environment]
python-version = "3.14"
from collections.abc import Callable
class Box[T]:
next: tuple[Box[Callable[[], T]], bool]
def read(more: bool):
value = Box[int]()
while more:
value, more = value.next
reveal_type(value) # revealed: Box[int] | Unknown
Generic NamedTuple with recursive fields
This is a regression test for https://github.com/astral-sh/ty/issues/3872. Computing the
NamedTuple fields while building the class's MRO must not try to determine whether the same class
is a TypedDict.
[environment]
python-version = "3.14"
from typing import NamedTuple
class Node[KT, VT](NamedTuple):
children: tuple[Node[KT, VT], ...] | tuple[Leaf[VT], ...]
class Leaf[VT](NamedTuple):
values: tuple[VT, ...]
Literal reduction during cycle recovery
This is a regression test for https://github.com/astral-sh/ty/issues/3851. Constructing a union during cycle recovery must not run redundancy checks between a literal and a protocol instance. Resolving the protocol interface can depend on the expression inference query that is already being recovered, which would introduce a new Salsa cycle.
[environment]
python-version = "3.14"
from typing import Protocol, runtime_checkable
_: Any
@property
def prop(self) -> A:
raise NotImplementedError
@runtime_checkable
class B(Protocol):
_: A
x = 5
while isinstance(x, B):
x = B() # error: [abstract-class-instantiation]
type(x)
x = 2
from typing import Any, assert_type
assert_type(prop, property)
if bool:
x = 5
while isinstance(x, B):
x = B() # error: [abstract-class-instantiation]
class A: ...
Literal widening during cycle recovery
Once a recursively growing group of integer literals widens to int, later iterations must not
reintroduce individual literals. Otherwise, the inferred type continues changing and the cycle never
converges. This is a reduced regression test from SciPy's iterative sparse solvers.
def solve(maxiter, a, b, c, d, e):
iteration = 0
stop = 0
while iteration < maxiter:
iteration = iteration + 1
if iteration >= maxiter:
stop = 7
if a:
stop = 6
if b:
stop = 5
if c:
stop = 4
if d:
stop = 3
if e:
stop = 2
if stop > 0:
break
return stop
Self-referential bare type alias
[environment]
python-version = "3.12" # typing.TypeAliasType
from typing import Union, TypeAliasType, Sequence, Mapping
A = list["A | None"]
def f(x: A):
reveal_type(x) # revealed: A
reveal_type(x[0]) # revealed: A | None
JSONPrimitive = Union[str, int, float, bool, None]
JSONValue = TypeAliasType("JSONValue", 'Union[JSONPrimitive, Sequence["JSONValue"], Mapping[str, "JSONValue"]]')
def _(x: JSONValue):
reveal_type(x) # revealed: Sequence[JSONValue] | float | None | Mapping[str, JSONValue]
Self-referential legacy type variables
from typing import Generic, TypeVar
B = TypeVar("B", bound="Base") # error: [missing-type-argument]
class Base(Generic[B]):
pass
Parameter default values
This is a regression test for https://github.com/astral-sh/ty/issues/1402. When a parameter has a
default value that references the callable itself, we currently prevent infinite recursion by simply
falling back to Unknown for the type of the default value, which does not have any practical
impact except for the displayed type. We could also consider inferring Divergent when we encounter
too many layers of nesting (instead of just one), but that would require a type traversal which
could have performance implications. So for now, we mainly make sure not to panic or stack overflow
for these seemingly rare cases.
Functions
class C:
def f(self: "C"):
def inner_a(positional=self.a):
return
self.a = inner_a
# revealed: def inner_a(positional=...) -> Unknown
reveal_type(inner_a)
def inner_b(*, kw_only=self.b):
return
self.b = inner_b
# revealed: def inner_b(*, kw_only=...) -> Unknown
reveal_type(inner_b)
def inner_c(positional_only=self.c, /):
return
self.c = inner_c
# revealed: def inner_c(positional_only=..., /) -> Unknown
reveal_type(inner_c)
def inner_d(*, kw_only=self.d):
return
self.d = inner_d
# revealed: def inner_d(*, kw_only=...) -> Unknown
reveal_type(inner_d)
We do, however, still check assignability of the default value to the parameter type:
class D:
def f(self: "D"):
# error: [invalid-parameter-default] "Default value of type `(a: int = ...) -> Unknown` is not assignable to annotated parameter type `int`"
def inner_a(a: int = self.a): ...
self.a = inner_a
Lambdas
class C:
def f(self: "C"):
self.a = lambda positional=self.a: positional
self.b = lambda *, kw_only=self.b: kw_only
self.c = lambda positional_only=self.c, /: positional_only
self.d = lambda *, kw_only=self.d: kw_only
# revealed: (positional=...) -> Unknown | ((positional=...) -> Divergent)
reveal_type(self.a)
# revealed: (*, kw_only=...) -> Unknown | ((*, kw_only=...) -> Divergent)
reveal_type(self.b)
# revealed: (positional_only=..., /) -> Unknown | ((positional_only=..., /) -> Divergent)
reveal_type(self.c)
# revealed: (*, kw_only=...) -> Unknown | ((*, kw_only=...) -> Divergent)
reveal_type(self.d)
Self-referential decorated functions
Resolving a decorated function's callable signature must not eagerly infer its default values. Otherwise, a default that refers back to the decorated name can re-enter the reachability check for an earlier assertion and prevent inference from converging. This is a regression test for https://github.com/astral-sh/ty/issues/4308.
f = lambda: f
assert f # error: [redundant-condition] "always truthy"
@property
def f(x=lambda: f): ...
The same cycle must converge when the parameter and return type are annotated:
g = lambda: g
assert g # error: [redundant-condition] "always truthy"
@property
def g(x: object = lambda: g) -> None: ...
Diagnostics for self-referential decorated functions
We reject a decorator that expects an integer instead of a function. Displaying the function's signature in that diagnostic can infer its self-referential default value. We report the error after function inference finishes, so diagnostic formatting does not create a cycle through the reachability check for the earlier assertion. This is a regression test for https://github.com/astral-sh/ty/issues/4440.
def decorator(value: int) -> int:
return value
f = lambda: f
assert f # error: [redundant-condition] "always truthy"
# error: [invalid-argument-type] "Expected `int`, found `def f(x=...) -> Unknown`"
@decorator
def f(x=lambda: f): ...
Self-referential property construction
Constructing a property explicitly has the same behavior as decorator syntax:
f = lambda: f
assert f # error: [redundant-condition] "always truthy"
def getter(x=lambda: f): ...
f = property(getter)
Self-referential callable decorators
The cycle is not specific to properties. A decorator that returns a callable with a fixed signature must also terminate:
from collections.abc import Callable
from typing import Any
def decorator(fn: Callable[[Any], Any]) -> Callable[[Any], Any]:
return fn
f = lambda: f
assert f # error: [redundant-condition] "always truthy"
@decorator
def f(x=lambda: f): ...
Self-referential ParamSpec decorators
A decorator can capture a function's parameters and return a callable with a different signature. Capturing those parameters must not evaluate a self-referential default.
[environment]
python-version = "3.12"
from collections.abc import Callable
def decorator[**P](fn: Callable[P, None]) -> Callable[[], None]:
return lambda: None
f = lambda: f
assert f # error: [redundant-condition] "always truthy"
@decorator
def f(x=lambda: f) -> None: ...
reveal_type(f) # revealed: () -> None
Self-referential generic properties
A generic getter's annotations are inferred in its type-parameter scope. Constructing the property must not pull its self-referential default into that inference.
[environment]
python-version = "3.12"
f = lambda: f
assert f # error: [redundant-condition] "always truthy"
@property
def f[T](value: T, callback=lambda: f) -> T:
return value
reveal_type(f) # revealed: property
Class decorator annotations depend on the decorated class
A decorator's annotations can refer to an instance of the decorated class. We report the invalid annotations and preserve the class binding, so constructor calls can still be checked.
from typing_extensions import reveal_type
# error: [invalid-type-form] "Variable `annotation` cannot be used as a type"
# error: [invalid-type-form] "Variable `annotation` cannot be used as a type"
def identity(value: annotation) -> annotation: ...
@identity
class Example:
def __init__(self, name: str) -> None: ...
annotation = Example("example")
reveal_type(Example) # revealed: <class 'Example'>
Example(123) # error: [invalid-argument-type] "Expected `str`, found `Literal[123]`"
Class decorator annotations depend on the decorated class through a type alias
A type alias can introduce the same dependency on the decorated class. Importers still see the class and can check its constructor arguments.
[environment]
python-version = "3.12"
example.pyi:
type Alias = annotation
def identity(value: Alias) -> Alias: ...
@identity
class Example:
def __init__(self, name: str) -> None: ...
annotation = type[Example]
main.py:
from example import Example
reveal_type(Example) # revealed: <class 'Example'>
Example("example")
Example(123) # error: [invalid-argument-type] "Expected `str`, found `Literal[123]`"
Dataclass outside a decorator whose annotations depend on the class
An outer @dataclass still generates an initializer when the inner decorator's annotations depend
on the decorated class.
from dataclasses import dataclass
from typing_extensions import reveal_type
# error: [invalid-type-form] "Variable `annotation` cannot be used as a type"
# error: [invalid-type-form] "Variable `annotation` cannot be used as a type"
def identity(value: annotation) -> annotation: ...
@dataclass
@identity
class Example:
name: str
annotation = Example("example")
reveal_type(Example) # revealed: <class 'Example'>
reveal_type(Example("example").name) # revealed: str
Example() # error: [missing-argument] "No argument provided for required parameter `name`"
Example(123) # error: [invalid-argument-type] "Expected `str`, found `Literal[123]`"
Recursive lambda used as a class decorator
The lambda returns a name that can refer back to the lambda itself. Inferring the decorated class converges even when another binding of that name has an unknown type.
make = lambda cls: result
try:
raise Exception
except Exception:
@make
class result: ...
result = make
finally:
from unknown_module import member as result # error: [import-not-found]
Decorated methods with implicit class attributes
This is a regression test for https://github.com/astral-sh/ty/issues/3471.
from collections.abc import Callable
from typing import TypeVar
class A: ...
T = TypeVar("T")
U = TypeVar("U", bound=A)
C = Callable[[T, U], object]
def d() -> Callable[[C[U, A]], object]:
raise NotImplementedError
class B:
@d()
def m1(self, p):
pass
@d()
def m2(self, p):
self.__slots__ # error: [missing-attribute]
Function annotation and dynamic NamedTuple / NewType
This is a regression test for https://github.com/astral-sh/ty/issues/3485 and
https://github.com/astral-sh/ty/issues/3682. Type traversal during cycle recovery should not force
the lazy base of a NewType.
class C:
pass
def f():
pass
def g() -> T: # error: [unresolved-reference]
pass
g()
from typing import NamedTuple, NewType
X = NamedTuple("X", [("x", "X")]), None # error: [invalid-type-form]
list(X)
min(X) # error: [invalid-argument-type]
T = f()
X = NewType("X", C)
The runtime callable returned by NewType also carries the lazy base and must use the same
cycle-safe traversal.
class C: ...
def f(): ...
def g() -> T: ...
g()
from typing import NamedTuple, NewType
X = NewType("X", C)
Y = NamedTuple("Y", [("a", "Y")]), X # error: [invalid-type-form]
min(Y) # error: [invalid-argument-type]
T = f()
Decorator defined on a base class with constrained typevars, accessed from a subclass with decorated generic parameters
This example was minimized from
a real issue in robotframework.
It created
a complicated cycle with multiple cycle heads,
which also involved
a tricky Salsa behavior that comes up when a query oscillates between being a cycle head and not being one.
entry.py:
from derived import Derived
Derived.decorate
# revealed: bound method <class 'Derived'>.decorate[T](item_class: type[T]) -> type[T]
reveal_type(Derived.decorate)
derived.py:
from ty_extensions._internal import reveal_mro
import bases
class Derived(bases.GenericBase["Foo", "Bar"]): ...
@Derived.decorate
class Foo(bases.Foo): ...
# revealed: <class 'Foo'>
reveal_type(Foo)
# revealed: (<class 'derived.Foo'>, <class 'bases.Foo'>, <class 'object'>)
reveal_mro(Foo)
@Derived.decorate
class Bar(bases.Bar): ...
# revealed: <class 'Bar'>
reveal_type(Bar)
# revealed: (<class 'derived.Bar'>, <class 'bases.Bar'>, <class 'object'>)
reveal_mro(Bar)
bases.py:
from typing import Generic, TypeVar, Type
from ty_extensions._internal import reveal_mro
T = TypeVar("T")
B1 = TypeVar("B1", bound="Foo")
B2 = TypeVar("B2", bound="Bar")
class GenericBase(Generic[B1, B2]):
@classmethod
def decorate(cls, item_class: Type[T]) -> Type[T]:
return item_class
# revealed: <class 'GenericBase'>
reveal_type(GenericBase)
# revealed: (<class 'GenericBase[Unknown, Unknown]'>, typing.Generic, <class 'object'>)
reveal_mro(GenericBase)
# revealed: (<class 'GenericBase[Foo, Bar]'>, typing.Generic, <class 'object'>)
reveal_mro(GenericBase["Foo", "Bar"])
class Foo: ...
class Bar: ...
Known class instances with a shadowed typing module
[environment]
python-version = "3.10"
String members retain their types when a local typing.py introduces an inference cycle. Resolving
str's bases looks up Sequence in that module. Determining whether the assignment is reachable
requires inferring trigger's return annotation. Resolving C.attribute requires determining C's
metaclass. Checking a call to that unknown metaclass constructs a class namespace with str keys,
completing the cycle. The consumer is checked before the shadowing module.
This is a regression test for https://github.com/astral-sh/ty/issues/4456.
m.py:
reveal_type("a".encode()) # revealed: bytes
typing.py:
class C(metaclass=missing): ... # error: [unresolved-reference]
def trigger() -> C.attribute: ...
trigger()
Sequence = object
Known class instances after checking the shadowing module
[environment]
python-version = "3.10"
String members retain their types when the shadowing module is checked first, as they do when the consumer is checked first.
typing.py:
class C(metaclass=missing): ... # error: [unresolved-reference]
def trigger() -> C.attribute: ...
trigger()
Sequence = object
m.py:
reveal_type("a".encode()) # revealed: bytes
Dynamic class metaclasses in loop conditions
Determining whether a loop body is reachable can depend on the metaclass of a dynamic class created in the loop, whose base in turn depends on that reachability.
type
value = 0
while value:
value = type("C", (value,), {})
types.new_class
from types import new_class
value = 0
while value:
value = new_class("C", (value,))
Generic call constraining a list initializer
[environment]
python-version = "3.12"
A list passed to a generic function retains the type inferred from its elements, even when the call's return type supplies an incompatible context for that list.
def keep[U](values: list[U]) -> list[U]:
return values
def make[T](arg: T) -> list[T]:
values = [42]
reveal_type(values) # revealed: list[int]
return keep(values) # error: [invalid-explicit-return]