## Summary
This PR splits `mismatched-type-name` into eight independently
configurable warnings for `NewType`, functional enums, `TypeVar`,
`ParamSpec`, `TypeVarTuple`, `namedtuple`/`NamedTuple`, `TypedDict`, and
`TypeAliasType`.
~~This PR is actually quite a bit less breaking for users than other
rule splits we've done over the last few days. Because
`mismatched-type-name` is no longer the canonical name for _any_ rule on
this branch, we're able to make it a deprecated alias for _all_ the new
rules, meaning that it will continue to work on the CLI, in ty
configuration files, and ty suppression comments (we'll just emit a
warning diagnostic telling users to upgrade their configuration
file/suppression comment).~~
~~A more specific rule takes precedence over the deprecated name if both
are given within the same configuration table. On the command line, the
last matching flag wins (e.g. `--error=mismatched-enum-name
--ignore=mismatched-type-name` will suppress diagnostics for mismatched
enum names).~~
(After review from @MichaReiser, I backed out these changes. It's now a
somewhat breaking change, because the rule is now marked as "removed"
rather than "deprecated".)
As well as splitting the rule into eight separate rules, it also cleans
up our error recovery for several of these type factories. For example,
in the following snippet, we fail to recognize `T` as a `TypeVar` due to
bad error recovery from the non-literal name. But we don't really need a
nonliteral name in this case to determine that it's a validly
constructed TypeVar. It's probably a mistake, but there's no _soundness_
issue here:
```py
from typing import TypeVar
def f(x: str):
# main emits an error: `invalid-typevar`
# this PR emits a warning: `mismatched-typevar-name`
T = TypeVar(x)
def g(x: T) -> T:
return x
reveal_type(g(42)) # main: Todo. This PR: `Literal[42]`.
```
## Motivation
The main motivation here is that having a namedtuple or enum with the
"wrong name", for example, just seems much less "obviously wrong" than
having a typevar or paramspec with the "wrong name". The [ecosystem
report](https://github.com/astral-sh/ruff/pull/29957#issuecomment-6603964612)
is evidence of this. namedtuples and enums define runtime classes that
can be used in runtime contexts and instantiated; these constructs
aren't _just_ for type checkers.
TypeVars/ParamSpecs/TypeVarTuples/NewTypes/TypedDicts are all _only_ for
type checkers and there's no plausible reason why you'd want to give the
type a different name to the variable it's assigned to.
`mismatched-namedtuple-name` and `mismatched-enum-name` are therefore
now disabled by default, matching the default severity we already
settled on for `mistmatched-sentinel-name` in
https://github.com/astral-sh/ruff/pull/27206. `mismatched-typevar-name`,
`mismatched-paramspec-name`, `mismatched-typevartuple-name`,
`mismatched-enum-name`, `mismatched-namedtuple-name` and
`mismatched-newtype-name` all remain enabled by default with `warning`
severity.
## Summary
Users can now suppress an attribute error that affects only some union
members and still catch names that are missing from every member. For
example, both lines below previously reported `unresolved-attribute`:
```python
def clean(value: str | int) -> str:
value.stirp() # missing-attribute: neither str nor int has stirp
return value.strip() # missing-union-attribute: str has strip, int does not
```
The motivation is that a project adopting ty may want `value.strip()` on
`str | None` to only warn while a typo remains an error-severity
diagnostic.
Both rules default to errors. The same distinction applies to optional
values such as `str | None` and to attribute deletion. Attributes
defined only conditionally on a class continue to use
`possibly-missing-attribute`.
### Comparison to other type checkers
Pyright distinguishes between `reportOptionalMemberAccess` (which is
specifically for unions with `None`) and `reportAttributeAccessIssue`,
which is for all other unions. mypy, meanwhile, distinguishes between
`union-attr` (which is an error code for all attribute-access errors on
union types) and `attr-defined` (which is an error code for all
attribute-access errors on non-union types). So this new split doesn't
_exactly_ align with either precedent -- but I _do_ think it's probably
the most useful split we can do, and it's the one that makes the most
sense _a priori_ in my opinion.
### Compatibility
`unresolved-attribute` remains an alias for `missing-attribute` in
configuration and inline ignores. To preserve a setting such as
`unresolved-attribute = "warn"` for both cases, also set
`missing-union-attribute = "warn"`. On an access that fails for only
some union members, replace `# ty: ignore[unresolved-attribute]` with `#
ty: ignore[missing-union-attribute]`. Tools that consume diagnostic
codes should recognize both new names.
Fixesastral-sh/ty#2839 and fixesastral-sh/ty#2471.
## Test Plan
A comparison with ty 0.0.96 on typeshed-stats, docstring-adder,
sphinx-lint, and typing_extensions found the same messages, locations,
and severities. Eight diagnostics received `missing-union-attribute`
across those four projects (two in sphinx-lint and six in
typing_extensions).
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
ty currently reports `unresolved-import` for both a missing module and a
missing member in a `from` import. But the word "unresolved" is heavily
overloaded in Python, and has been reported by users as being confusing
(see https://github.com/astral-sh/ty/issues/2471). This PR renames the
rule to `import-not-found`, which describes the lookup failure in either
case.
Configuration, command-line flags, and ignore comments accept either
spelling. If several flags set this rule’s severity using different
spellings, the last flag wins. Existing baseline entries still match,
and rewriting a baseline uses the new name. Links to the old rule
heading still work.
A generic type alias defined in a class could report a class-local name
as undefined or silently resolve it to a module-level name. For example,
in:
```py
class Outer:
class Inner:
type Pair[T] = tuple[T, Inner]
```
ty skipped the class scope that defined `Inner` because the alias's
type-parameter scope sat between its value and the class.
Name lookup now sees the enclosing class through consecutive annotation
scopes. The semantic index and type inference use the same visibility
rule.
Fixes https://github.com/astral-sh/ty/issues/4886.
Infer (most) union parameters through constraint-set assignability, so
every argument member must match a parameter alternative and all
arguments contribute to the same solution.
For now we keep the legacy constraint inference for variadics, recursive
aliases, and gradual evidence; these can be migrated over time.
- Fixesastral-sh/ty#1703
- Fixesastral-sh/ty#3695
- Fixesastral-sh/ty#4277
- Fixesastral-sh/ty#4364
- Fixesastral-sh/ty#4803
- Partially addresses astral-sh/ty#3557, but we still have some paths
that go through the legacy solver, so doesn't close this issue yet.
- Partially addresses astral-sh/ty#3022: fixes the bounded-variable
union example, but the original ParamSpec decorator example still fails.
- Partially addresses astral-sh/ty#3163: fixes synchronous inference,
but the asynchronous case still includes the coroutine type in its
result.
## Ecosystem report
Analyzed head: `203404e00c339e9cea5518103abe58a0adbdf1c2`. Generic union
inference improves across wrappers, collection adapters and numerical
APIs; more precise recovery also exposes downstream annotation
mismatches. The remaining regressions fall within the accepted deferrals
documented in the [complete ecosystem
analysis](https://github.com/astral-sh/ruff/pull/29236#issuecomment-6385786960).
The Colour slowdown did not recur in this run: raw job medians were
36.20s on the merge base and 35.29s on the PR, with two samples per
revision. This small sample does not establish a stable speedup.
Checking an eager annotation such as `A | B | C` can involve looking up
the same name twice. Ty needs both the type that the annotation denotes
and the runtime value on which Python calls `|`. In longer annotations,
it can also evaluate an inner union again while checking its parent.
This change reuses the runtime values of bare names, `None`, and nested
unions within the same eager annotation. A nested union computes its
runtime value only when the enclosing `|` can use it under the same
inference flags. Each expression still records its annotation type. For
example, `bool | int` denotes `int` but evaluates to a union object,
which matters for a subsequent `| None`.
This addresses the eager-annotation case from #29312 on top of #29727.
## Test plan
Measured CPU time for fresh, single-core `ty check` processes over 12
interleaved rounds. The comparison uses main at `18c5d27c41` and the
eager operand changes from #29312 rebased onto the same commit. Times
are medians in milliseconds for each whole input file.
| Input | Main | Rebased #29312 | This change |
| --- | ---: | ---: | ---: |
| Three ordinary classes, 1,500 signatures | 58.5 | 59.3 | 56.6 |
| Twelve ordinary classes, 450 signatures | 86.7 | 72.5 | 69.4 |
| Twelve specialized generics, 350 signatures | 144.5 | 100.3 | 100.0 |
| Direct `TypeForm` arguments | 113.9 | 98.3 | 91.7 |
The source-only checks of NumPy’s `_typing` package and Pydantic showed
no clear difference. Against rebased #29312, the stored 256-member
`TypeForm` case used 1.0% more CPU time, and the
`gradual_literal_union_equality` Criterion benchmark took 0.9% longer.
Compared the inferred types of all 14 AST nodes in a pair of left- and
right-nested annotations. Both alternatives and main yielded the same
type at every node.
Fixes a hang when a recursive alias combines positive and negative
references, such as `type A = list[A | ~A] | list[~A]`.
Retain negated recursive aliases whose union members resolve to nominal
instance types, so mapping does not repeatedly unfold the alias into
itself. Other union members still undergo normalization to preserve
literal exclusions and positive constraints.
Fixes https://github.com/astral-sh/ty/issues/4799.
## Test plan
Checked the original reproducer and equivalent spellings using `Not[A]`
and `Intersection[object, Not[A]]` with timeout-bounded debug binaries.
The patched checker completes on all three; the original reproducer
hangs on the baseline.
When ty checks a value assignment such as `Alias = A[int] | B[int] |
C[int]`, it currently normalizes `A[int] | B[int]` and then compares
those same members again while building the final union. Longer chains
repeat that work for every prefix, even when nothing ever uses the value
as a type.
Instead, this PR just stores the runtime operands of `|` in the
runtime-union object, and only builds and normalizes the actual type
union lazily, if or when the runtime union value needs to be understood
as a type. For example, checking a stored union against `TypeForm[Base]`
walks the operands once and caches the result. Checks such as
`isinstance` can still inspect the original operands.
This is an alternative to #29304, which keeps the type of every
intermediate union available. For `(A | B) | C`, it normalizes `A | B`,
then seeds a fresh `UnionBuilder` with those members. If the
construction policy and inference context match, the builder only needs
to compare `C` with the existing members. This PR stores the `A | B`
object as the left operand of the outer union. If only the final type is
requested, one union builder visits `A`, `B`, and `C`. A second request
for that same union uses the cached type. The two approaches take about
the same time when a 256-class union is consumed once.
- For a chain of assignments like `U2 = U1 | Literal[2]`, each
assignment provides a new type-variable binding context. In #29304 that
context is part of the cache key, so these repeated assignments defeat
the caching, and extending the union can revisit its prefixes even when
nothing ever asks for their types. This accounts for the slowness of
#29304 on the "Extend a literal union across 1024 assignments" case
below.
- If `Prefix` is a long union and `(Prefix | X, Prefix | Y, ...)` is
checked against `tuple[TypeForm[Base], ...]`, every extension needs its
type. #29304 can reuse the normalized `Prefix` across those extensions.
This PR walks all its members for each distinct result. With 32
extensions, #29304 is nearly twice as fast. If this case turns out to
occur commonly in practice, we could add intermediate prefix caching as
well.
This PR is larger than #29304, which leaves the runtime union
representation largely intact and adds a path in `UnionBuilder` for
reconstructing an already normalized prefix. This PR needs extra
complexity in `UnionTypeInstance`, where specialization, cycle recovery,
and runtime checks need to distinguish the original operands from the
denoted type. But I prefer keeping the complexity of these
runtime-unions in the runtime-union implementation, rather than allowing
it to leak into the core `UnionBuilder`, as it does in #29304.
This also fixes a bug (as seen in xarray in the ecosystem report) with a
runtime check list `isinstance(x, float | int)`. Previously we would
simplify this union, eliminating `int`, but `int` and `float` are
separate classes at runtime, so we would then miss that the `isinstance`
check catches integers as well. Now the `isinstance` narrowing can look
at the original runtime operands instead.
## Test plan
I timed optimized builds on Apple silicon using the same main revision
(`157caa55a5`) for all three. The #29304 build applies its reviewed
patch (`a0f998e2b4`) to that base. Each entry is the median elapsed time
in milliseconds for six fresh, single-worker CLI checks with warm
filesystem caches.
| Workload | Main | #29304 | This change |
| --- | ---: | ---: | ---: |
| Construct an unused union of 256 specialized classes | 326 | 56 | 34 |
| Check the stored union against `TypeForm[CommonBase]` | 328 | 57 | 60
|
| Extend a literal union across 1,024 assignments | 46 | 3,789 | 29 |
| Extend a union across 128 assignments and consume each prefix | 77 |
95 | 79 |
| Check 32 extensions of the same 128-class prefix as type forms | 112 |
57 | 105 |
## Ecosystem report
At `dcc4ce9890`, xarray loses two false positives because `isinstance`
now recognizes integer chunk sizes. Three existing xarray diagnostics
include the types that actually reach those branches. Two truthiness
warnings in aioredis and dd-trace-py display the same union members in a
different order. All seven changes are expected. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/29727#issuecomment-6422364992)
Builds on #29200.
When an invalid implicit alias is used as a type, the error now explains
why its original right-hand side is invalid and points to that
expression. For example:
```py
Called = type(1)
# snapshot: invalid-type-form
def inspect(called: Called):
reveal_type(called) # revealed: Unknown
```
```text
error[invalid-type-form]: Variable is not allowed in parameter annotations
--> src/mdtest_snippet.py:14:21
|
14 | def inspect(called: Called):
| ^^^^^^
info: Function calls are not allowed in type alias values
--> src/mdtest_snippet.py:11:10
|
11 | Called = type(1)
| ^^^^^^^
info: See the following page for a reference on valid type expressions:
info: https://typing.python.org/en/latest/spec/annotations.html#type-and-annotation-expressions
```
No other type checker provides this kind of detailed origin explanation
when a value is not usable as a type alias.
Follow imports and forwarding assignments to the original definition.
Collect these explanations independently of diagnostics in the defining
file, so unchecked dependencies and local suppressions do not hide them.
Ordinary assignments remain valid at runtime; the error is reported only
where the value is used as a type.
https://github.com/astral-sh/ruff/pull/28425 added a new way to infer
implicit and PEP 613 type aliases, which I'd previously described in
https://github.com/astral-sh/ty/issues/1711. Rather than using the value
type of the alias and converting it back to its type-annotation meaning,
this system goes back to the original definition and infers it directly
as a type expression. This gives us more robust validation that implicit
type aliases are valid type expressions, and allows us to emit accurate
diagnostics both for invalid-as-type-expression and for
invalid-at-runtime.
https://github.com/astral-sh/ruff/pull/28425 used this new approach only
for recursive type aliases; this PR expands it to all implicit and PEP
613 type aliases, so that we consistently get its benefits. This also
significantly improves our conformance results for implicit type
aliases; we now pass two more conformance files.
This is an example of the sort of thing that we previously accepted as a
type alias, and now reject (as the conformance suite says we should):
```py
BadAlias = (lambda: int)()
def f(value: BadAlias): # Main: int, Now: invalid-type-form
...
```
This also makes us more consistent (with ourselves, other type checkers
,and the conformance suite) when a generic type is used "bare" in an
alias. The core rule we use is that the RHS of an implicit or PEP 613
type alias is a type expression, therefore a use of a bare generic alias
in that type expression should be implicitly default-specialized.
```py
from typing import TypeVar
T = TypeVar("T", default=str)
GenAlias = list[T]
AnotherAlias = GenAlias | int
# this is now an error: AnotherAlias is not generic
def _(x: AnotherAlias[bytes]): ...
# The bare reference to GenAlias inside AnotherAlias is default-specialized:
def _(x: AnotherAlias):
reveal_type(x) # list[str] | int
```
We do make one carve-out from this rule: we consider a plain assignment
like `A = B` or `A = module.B` to be a direct "rename" or "forward" that
doesn't really treat the RHS as a type expression, and thus doesn't
default-specialize anything; instead `A` just becomes a new name for
`B`. This feels more intuitive, and matches pyrefly and pyright. This
carve-out does not apply to PEP 613 type aliases: `A: TypeAlias = B` is
explicitly a type alias, not a rename, so it does default-specialize
`B`.
Reject computed values used as types, including values that evaluate to
classes or `Any`, and preserve that validation when `invalid-type-form`
is disabled. Definition lookup stays behind a tracked query so unrelated
edits in an imported module do not invalidate consumers.
Fixes https://github.com/astral-sh/ty/issues/1711.
Fixes https://github.com/astral-sh/ty/issues/1846.
Prepares https://github.com/astral-sh/ty/issues/4683. Runtime `|`
behavior is not changed here.
Source-location diagnostic explanations for invalid aliases are added in
the follow-up #29654.
## Ecosystem
[Full ecosystem
analysis](https://github.com/astral-sh/ruff/pull/29200#issuecomment-6353595871)
for `b175c73` covers all 150 diagnostic changes across 21 projects: 61
additions, 58 removals, and 31 message changes. New diagnostics
primarily validate computed values used as types and default-specialized
explicit aliases. Improved alias inference removes cascading errors and
preserves forward references and generic parameters.
The nine new discord.py false positives match the known
[ty#4016](https://github.com/astral-sh/ty/issues/4016) callback-context
limitation and are accepted for this PR. No other new false positives
were identified.
Conformance gains six expected diagnostics (five in
`aliases_implicit.py`, one in `aliases_explicit.py`) and two fully
passing files, with no new conformance false positives.
Copying a dictionary whose value type contains a recursive
`Callable[..., T]` incorrectly reports `invalid-argument-type` for
`dict(value)`. Preserve materialization at recursive callable references
so the inferred dictionary type satisfies the constructor's `Self`
bound.
This also prevents stack overflows when checking negated recursive
aliases such as `type A = ~list[A] | list`, including the equivalent
legacy `TypeAlias` syntax.
Finalize alias materialization inside its recursion guard, and reuse
partial transformations only while the guards they depend on remain
active. Preserve the function guard when following `__call__` and share
the materialization-equivalence guard across nested overlap checks.
Fixes https://github.com/astral-sh/ty/issues/4744.
Fixes https://github.com/astral-sh/ty/issues/4768.
## Test plan
Compared deeply shared recursive callable graphs against the base
revision, including a graph enclosed by a recursive alias. Checking time
remains comparable to the base revision; added benchmark guards against
exponential recomputation.
## Summary
We now mark both branches of a condition with type `Never` as
unreachable:
```python
from typing import Never
def f(value: Never):
if value:
result = 1 # Unreachable.
else:
result = 2 # Also unreachable.
```
Previously, we treated `Never` as having ambiguous truthiness, allowing
either assignment to affect inference. In the reported loop, the
variables depend on each other, so including these impossible
assignments made their inferred types depend on which file we checked
first.
We add `Truthiness::Uninhabited` to represent the absence of either
boolean outcome. `Type::bool` and `Type::try_bool` return this variant
for `Never`, replacing the opt-in `bool_if_inhabited` API and making
callers handle the distinction explicitly.
Closes https://github.com/astral-sh/ty/issues/4577.
## Summary
We now terminate inference when an `Annotated` value recursively wraps
its own inferred type:
```python
from typing import Annotated, reveal_type
from ty_extensions._internal import TypeOf
value = int
while True:
value = Annotated[TypeOf[value], "metadata"]
reveal_type(value)
```
We preserve the `Annotated` wrapper when simplifying the recursive type,
so inference can recognize the cycle and stop expanding it.
Addresses the runtime `Annotated` reproducer in
https://github.com/astral-sh/ty/issues/3837.
## Summary
Expressions such as `partial(f) | None` were treated as type unions even
though the partial is a value. In a loop, this can cause cycle inference
to exhaust its iteration limit:
```python
from functools import partial
def f(): ...
def example(flag: bool):
value = partial(f)
while flag:
value = value | None
```
We now apply the same type-union operand check when either operand is
`None`. Other values go through normal binary-operator inference, which
reports the unsupported operation and allows the cycle to converge.
Sentinel values remain valid union operands, including when combined
with another type.
## Summary
We now report `cyclic-type-alias-definition` instead of overflowing the
stack for PEP 613 aliases such as:
```python
from typing import TypeAlias
A: TypeAlias = "~A & A"
```
Cycle detection now follows both positive and negative intersection
elements, so intersections and negations cannot hide unguarded recursive
references. This also covers mutual recursion and PEP 695 aliases, while
recursion through a containing type remains valid.
Closes https://github.com/astral-sh/ty/issues/4601.
A type variable with a callable upper bound should be inferred as its
upper bound during callable inference:
```py
from typing import Callable
def apply[T](callback: Callable[[int], T]) -> T:
return callback(1)
def _[F: Callable[[int], str]](callback: F):
reveal_type(apply(callback)) # main: Unknown, now: str
```
## Summary
Closes https://github.com/astral-sh/ty/issues/1738
(While this PR also resolves the error in the code provided in
https://github.com/astral-sh/ty/issues/1846, I consider the broader
issue under discussion to still be unresolved)
To fully support implicit type aliases, this PR introduces `Recursive`
types. These represent *structural recursive types*. In other words,
they are anonymous recursive types implicitly constructed through type
inference, rather than a nominally defined recursive type like PEP 695
recursive type aliases.
This is what was being discussed in
https://github.com/astral-sh/ruff/pull/24252#issuecomment-4172655350. We
are not introducing recursive types solely for implicit type aliases;
this is the first step toward completely replacing the current
`Divergent`-based approximate recursive type inference with a full
implementation.
Initially, I planned to completely remove the `Divergent` type alongside
the introduction of `Recursive` types, but because the scope of the
impact was far too large, I decided to replace it incrementally. Both
will coexist until the replacement is complete, but eventually,
everything should be replaced by `Recursive` types.
Following this PR, I plan to use the newly introduced `Recursive` types
to replace various forms of recursive type inference.
## Design
Currently, we approximate and treat recursive type structures generated
by recursive type inference as gradual types containing `Divergent`. For
example, a subscript on `x: Divergent` results in `x[n]: Divergent`.
Consequently, we get:
```
x: list[Divergent]
x[n]: Divergent
x[n][n]: Divergent
```
As a result, the type after subscripting loses the information that `x`
was a recursive list type. We solve this problem by introducing
recursive types.
`RecursiveType` is a binder for a body type that contains a recursive
variable. In other words, it roughly has the following structure:
```rust
struct RecursiveType<'db> {
id: salsa::Id,
body: Type<'db>,
arguments: Option<Specialization<'db>>,
}
struct RecursiveVar<'db> {
id: salsa::Id,
arguments: Option<Specialization<'db>>,
}
enum Type<'db> {
Recursive(RecursiveType<'db>),
RecursiveVar(RecursiveVar<'db>),
...
}
```
For example, for the following implicit type alias:
```python
Tree = tuple[T, "Tree[list[T]] | None"]
# = μF. λT. tuple[T, F[list[T]] | None]
```
the constructed `RecursiveType` looks like this:
```rust
RecursiveType {
id,
body: tuple[T, RecursiveVar { id, arguments: [list[T]] } | None],
arguments: [T]
}
```
Unlike `Divergent`, the recursive variable `RecursiveVar` has no type
properties on its own. Attempting to access this variable immediately
unfolds the recursive type, so questioning the typing properties of the
variable itself is meaningless. Therefore, if any type operation on
`RecursiveVar` appears, it should be a bug.
To infer the container subscript `x[1]` for `x: Tree[int]`, first unfold
`x`'s type. Unfolding replaces references to the recursive binder with
the recursive type itself. Writing `B[a := R]` for capture-avoiding
substitution of `R` for `a` in `B`:
```text
unfold(μa. B) = B[a := μa. B]
```
For `Tree[int]`, substitute the constructor for `F`, then apply `T :=
int`:
```text
unfold((μF. λT. tuple[T, F[list[T]] | None])[int])
= tuple[int, (μF. λT. tuple[T, F[list[T]] | None])[list[int]] | None]
(= tuple[int, Tree[list[int]] | None])
```
Tuple subscripting then selects the element at index 1: `x[1]:
Tree[list[int]] | None`.
## Test Plan
mdtest updated
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Prior to this change, we treated imports as declarations, incorrectly
requiring later assignments to preserve the imported type:
```py
import os
from types import ModuleType
def normalize(module: ModuleType) -> ModuleType:
return module
os = normalize(os)
```
We now model regular imports, `from` imports, and wildcard imports as
bindings rather than declarations. This allows ordinary reassignment,
context-manager targets, and optional-import fallbacks while preserving
existing local annotations.
Imported `Final` qualifiers are tracked separately from declared types
through use-def flow. This preserves the qualifier across assignments,
re-exports, and branches without replacing explicit annotations or
treating imports as protocol-member declarations. The implementation
reuses declaration-flow storage and reachability information, with
separate queries for imported qualifiers.
Deprecation checks use the original imported value before assignment
recovery, so a conflicting local annotation does not suppress the
import's deprecation warning.
Closes https://github.com/astral-sh/ty/issues/1836.
## Test plan
- Added mdtests for import reassignment, optional-import fallbacks,
existing annotations, wildcard imports, and cyclic loop/match bindings.
- Added imported-`Final` coverage for re-exports, conditional and
unreachable imports, nested and forwarding scopes, class and protocol
members, annotated stub re-exports, and replacement by later imports or
annotations. Core tests check that imported qualifiers remain separate
from genuine declarations and participate in branch merging.
- Added a conflicting-annotation regression that checks both the
assignment error and import deprecation warning while preserving the
recovered binding type.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
This PR allows us to distribute `type[..]` over intersections, and
therefore let's us recognize that `type[A & B] = type[A] & type[B]`.
This implements what has been proposed here:
https://github.com/astral-sh/ruff/pull/25704#discussion_r3389309682.
## Ecosystem
The psycopg hit looks like it's related to the problems around `self`
receiver types that are intersections. It's a regression, but maybe
acceptable? If we think it's problematic, we can also defer this PR.
> Internally, ty represents `NamedTuple` as:
>
> ```python
> tuple[object, ...] & NamedTupleLike
> ```
>
> This PR distributes `type[...]` over that intersection:
>
> ```python
> type[NamedTuple]
> =
> type[tuple[object, ...]] & type[NamedTupleLike]
> ```
>
> When resolving `nt._make`, ty finds the method on `NamedTupleLike`.
However, it does not bind `Self` to the instance type of the complete
intersection receiver.
## Test Plan
New Markdown test
## Summary
Allow experimental intersection and negation syntax in PEP 695 type
aliases, e.g. `type Both = A & B`. These aliases are evaluated lazily,
but ty previously reported an unsupported runtime operation even though
it inferred the correct type.
Skip the runtime checks for `&` and `~` in PEP 695 alias scopes, and for
deferred annotations.
## Test Plan
New Markdown tests
## Summary
Add unsafe fixes to diagnostics that suggest `Literal`: wrap integer,
bytes, and boolean literals and strings that cannot be parsed as forward
annotations in `Literal[...]`, and replace parameterized `LiteralString`
with `Literal` when its arguments are valid literal parameters.
The fixes preserve literal spelling and reuse unshadowed import aliases.
They import from `typing` on Python 3.8+, or from `typing_extensions` on
Python 3.7 when it is a declared direct dependency and the installed
module exports `Literal`. The typing-module availability checks are
shared with the `reveal_type` and `assert_never` fixes.
## Summary
Previously, we allowed arbitrary attribute access on the alias objects
`type[T]` and `typing.Type[T]`. We now resolve missing attributes
against their known origin, `type`, instead of using the unrestricted
`GenericAlias.__getattr__` fallback.
```python
from typing import Any, Type, TypeAlias
Modern: TypeAlias = type[Any]
Legacy: TypeAlias = Type[Any]
# Before: no errors. After: unresolved-attribute on each line.
Modern.missing
Legacy.missing
def example(cls: type[Any]) -> None:
cls.missing # Before and after: OK.
```
The alias object has a known origin, whereas `cls` can refer to any
class with its own attributes. Attributes defined on the alias itself,
such as `__args__` and `__origin__`, remain accessible. The origin is
`type` regardless of the type argument, so attributes of `int`, such as
`bit_length`, are not available on the alias object `type[int]`.
Mypy 2.1.0, pyright 1.1.410, and pyrefly 1.3.0-dev.1 with the `default`
preset agree with the behavior shown above: they reject both missing
alias attributes and allow the access through `cls`.
This reports the two missing errors in
[`specialtypes_type`](https://github.com/python/typing/blob/cf943ccbea5596ef969eec6e2260097ff697b7ba/conformance/tests/specialtypes_type.py#L143-L146),
reducing its discrepancies from 11 to 9. The file remains Partial
because its remaining nine discrepancies reflect an intentional choice
in ty: we interpret bare `type` as `type[object]`, whereas the
conformance suite expects `type[Any]`. The other 144 conformance files
are unchanged.
Searching generic alias arguments for `Self` or a type variable starts a
fresh type walk for each argument. Shared nested specializations are
therefore visited repeatedly, which can make method binding take
exponential time even when its annotations contain no `Self`.
Use a shared visitor implementation for both searches, with one
recursion guard per search. It visits explicit alias arguments,
including arguments unused by the alias body, without evaluating alias
bodies or other lazy attributes. Different specializations of the same
alias remain distinct.
Follow-up to #28117.
## Test plan
Add mdtests for `Self` used only in an aliased parameter, including
accepted and rejected arguments, and nested uses of the same alias in an
inherited method. Simplify the protocol fixtures while retaining
separate method and property cases for structural `Self` binding.
A minimized example with 32 levels of shared alias arguments completes
in about 40 ms; the prior implementation hit a 12-second timeout in the
same debug-build setup.
## Summary
ty incorrectly rejects structural protocol implementations when `Self`
appears as an argument to a generic type alias:
```python
type Identity[T] = T
class Cloneable(Protocol):
def clone(self) -> Identity[Self]: ...
class Concrete:
def clone(self) -> Identity[Self]:
return self
```
`Self` binding uses a preliminary check to avoid applying a type mapping
when a type does not contain `Self`. Because type alias bodies are lazy,
this check did not inspect the alias specialization and therefore missed
the `Self` argument in `Identity[Self]`.
Inspect generic type alias arguments when checking for `Self`, without
evaluating the alias body. The existing type mapping can then bind
`Self` normally in method parameters and returns, property types, and
attribute annotations.
Fixesastral-sh/ty#4409.
## Test Plan
- Added mdtests for binding aliased `Self` in bound method parameters,
return types, and attribute annotations.
- Added protocol mdtests covering aliased and direct structural method
implementations and property return types.
- Ran the full `ty_python_semantic` test suite.
- Ran Clippy with all targets and features and warnings denied.
- Ran the applicable repository hooks.
ty can incorrectly treat a branch guarded by an impossible compound
condition as reachable when an operand has potentially mutable
truthiness. For example, `if value and False:` always skips its body in
CPython; the first falsy operand selects the control-flow path
immediately, it doesn't fully evaluate `value and False` and then
re-evaluate the truthiness of the result. Saving `value and False` to a
variable and then testing the saved object can instead call
`value.__bool__` twice and get different answers.
Distinguish expressions consumed directly as conditions from expressions
that produce values. Evaluate direct boolean and conditional expressions
structurally, and retain the combined truthiness of chained comparisons
even when a comparison returns an arbitrary object. Operands with types
equivalent to `Never` cannot produce a result; preserve that distinction
from ambiguous truthiness while composing conditions. Value contexts
continue to allow repeated truthiness checks to disagree.
To limit retained memory, comparison truthiness uses sparse overrides in
optional inference metadata, stored as boxed slices. Ordinary comparison
chains use the truthiness of their inferred value type, and root
chained-comparison conditions reuse expression inference without a
separate condition-analysis memo. Cycle widening preserves overrides
when needed to keep the effective truthiness monotonic. Tracking
operands that cannot produce a result adds only transient evaluation
state.
Prerequisite for #28045, which addresses astral-sh/ty#4380.
## Ecosystem results
Five expected diagnostic removals across pwndbg, pytest, and Spark; no
additions. Each comes from a branch that is unreachable under the
checked platform or declared types. See the [Codex ecosystem
analysis](https://github.com/astral-sh/ruff/pull/28082#issuecomment-5434341797)
for details and minimized reproducers.
## Test plan
- Mdtests cover compound conditions, nested boolean operations and
`not`, conditional expressions, and chained comparisons across branches,
loops, assertions, comprehension filters, and match guards.
- Reachability assertions use revealed variable types, including
reachable saved results, assignment expressions, nested value
expressions, and call arguments whose truthiness can be tested again.
- Cases with operands narrowed to `Never`, aliases and unions of aliases
of `Never`, and type variables bounded by `Never` assert branch
reachability using unrelated variables and check precise
conditional-expression result types.
- Additional cases cover unreachable loop assignments and conditional
expressions in stringified `Annotated` metadata.
- A Rust invariant test covers sparse comparison-truthiness overrides
appearing or disappearing across inference-cycle results, including
preservation of previous-only overrides.
## Summary
Add a new `SemanticMode::definitely_has_builtin_binding()` that allows
us to cheaply and easily query whether a builtin name such as `list` or
`int` has been shadowed by a variable in an enclosing scope when viewed
from a given node in the AST. Once this method is in place, it becomes
much more trivial to add autofixes to various diagnostics, which this PR
therefore does.
The new method does not attempt to fully model Python's name-lookup
semantics. It doesn't need to -- it's okay to be conservative when
offering autofixes. The method therefore errs on the side of caution and
only returns `true` if it can be certain that the name passed in was
definitely not shadowed in an enclosing scope.
The new method is similar to the `SemanticModel::has_builtin_binding()`
method that Ruff has had for a long time.
The new `SemanticModel` method is pulled out of
https://github.com/astral-sh/ruff/pull/27634, where I'm using it for
several more diagnostic autofixes that make it much easier to understand
how ty is trying to tell you to rewrite your code (in my opinion).
## Test Plan
mdtests and snapshots
## Summary
Previously, we rejected calls to inherited methods through unsubscripted
generic classes:
```python
def clear_containers(items: list[int], mapping: dict[str, int]) -> None:
list.clear(items)
list.reverse(items)
dict.clear(mapping)
```
Inherited member lookup applied the subclass's default type arguments to
ordinary signature types, but left class-scoped type variables inside
the upper bound of `Self`. For example, `list.clear` retained a bound of
`MutableSequence[_T@list]`, even though `_T@list` is not inferable at
the call site.
We now apply the owner's default specialization to `Self` bounds as
well. This accepts compatible receivers while preserving declared
defaults, incompatible-receiver errors, and concrete return types for
methods returning `Self`. Constructor inference remains unchanged.
Closes https://github.com/astral-sh/ty/issues/4353.
## Summary
Previously, suggestions for invalid parameterized `LiteralString`
annotations were based on the inferred runtime types of their arguments:
```python
from typing import Literal, LiteralString
Alias = Literal["value"]
alias: "LiteralString[Alias]"
value = "value"
variable: "LiteralString[value]"
```
We now check quoted arguments with the existing `Literal` argument
validator before deciding whether to suggest `Literal`. This adds the
suggestion for valid literal aliases and avoids suggesting it for
ordinary variables. Evaluated annotations retain their existing
behavior.
Valid calls such as `collections.abc.Mapping.register(...)` currently
produce missing-attribute errors. Using `register` as a class decorator
can also hide the registered class’s members and generic parameters,
causing further false positives and negatives.
This PR gives protocol classes declared in source or ordinary `.pyi`
files the metaclass `typing._ProtocolMeta`, which derives from
`ABCMeta`. That makes ABC methods available and preserves the types of
registered classes. It also lets ty report incompatible custom
metaclasses.
Typeshed’s standard-library stubs need a narrow compatibility exception:
they sometimes list `Protocol` bases that the runtime class does not
inherit. When those bases do not select a custom metaclass, ty uses
`ABCMeta` for attribute lookup but does not let that inferred metaclass
create artificial inheritance conflicts. This applies to bundled or
configured typeshed, not to other `.pyi` files. An explicitly chosen
custom metaclass continues to constrain subclasses. Known built-ins
retain their actual `type` metaclass.
Mypy applies a similar permissive fallback more broadly; this PR limits
the exception to typeshed.
Fixes https://github.com/astral-sh/ty/issues/1204.
## Ecosystem results
The [current ecosystem
run](https://github.com/astral-sh/ruff/actions/runs/32385816515/attempts/1)
removes missing-`register` errors in pandas, ppb-vector, Jinja, Spack,
and Ibis, plus two spurious metaclass conflicts in Spack’s legacy
`typing_extensions`. All 294 Ibis changes follow from recovering
registered classes’ members and generic parameters. The added
diagnostics expose existing annotation gaps, deliberately invalid tests,
and the known [virtual ABC subclass
limitation](https://github.com/astral-sh/ty/issues/2391).
The beartype library has an [old compatibility
workaround](https://github.com/beartype/beartype/blob/d6c056225f2e5632ea72006b43ffb066bd9a2c6b/beartype/typing/_typingpep544.py#L73-L79)
for the fact that `_ProtocolMeta` used to be missing from typeshed; it
substitutes `ABCMeta` under `if TYPE_CHECKING`, so it now produces a
metaclass conflict. We accept this false positive: typeshed now declares
`_ProtocolMeta`, and beartype can remove the workaround and derive from
`type(Protocol)` directly.
## Test plan
- Cover protocol metaclasses, bundled and custom typeshed, and
registration decorators that preserve generic classes.
- Check compatible and conflicting metaclasses, built-in collections,
and inheritance through ordinary and dynamically created classes.
- Protect subclass-type bounds, class-namespace attributes, quoted
unions, and dataclass-transform inheritance.
## Summary
Previously, `Annotated` metadata containing unpacked keyword arguments
could panic when the annotation was written as a string:
```python
from typing import Annotated
value: "Annotated[int, dict(**{})]"
```
Dictionary-key narrowing attempted to look up use-definition information
for the unpacked keyword argument, but parsed string-annotation nodes
are not present in the semantic index.
We now skip dictionary-key narrowing inside string annotations while
preserving ordinary call inference. This also handles unpacked
dictionaries in parameter and return annotations.
## Summary
Previously, invalid `Literal` and `LiteralString` arguments inside
string annotations could reach assignment expressions that are absent
from the semantic index:
```python
from typing import Literal, LiteralString
value: "LiteralString[(other := 0)]"
another: "Literal[int[(other := 0)]]"
```
We now infer both sides of assignment expressions in string annotations
without looking up nonexistent semantic-index definitions. This avoids
the panic while preserving unresolved-reference and other nested
diagnostics for invalid `Literal` and `LiteralString` arguments.
Evaluated annotations retain their existing inference and runtime
diagnostics.
## Summary
Follow-up to #27882.
Previously, the non-generic-class and `type[...]` fallback paths could
evaluate invalid subscript arguments inside string annotations, reaching
assignment expressions that are absent from the semantic index:
```python
value: "int[(other := 0)]"
another: "type[(other := 0)]"
```
We now skip expression inference in these error-recovery paths when the
arguments come from a string annotation. Non-generic classes retain
their existing `invalid-type-form` diagnostic, unsupported `type[...]`
forms retain their existing fallback type, and evaluated annotations
retain their runtime diagnostics. We do not add support for additional
`type[...]` forms.
This fix is independent of #27914 and #27913.
## Summary
Previously, we inferred an invalid subscript's value before checking
whether it was a valid type-expression operand. Inside a string
annotation, this could evaluate expressions that are absent from the
semantic index, panic on assignment expressions, or emit cascading
unresolved-reference diagnostics:
```python
value: "(lambda default=(name := int): None)[int]"
other: "TypedDict('T', {}, extra_items=missing)[int]"
```
We now infer invalid subscript operands only when annotations execute at
runtime. String annotations instead report the existing
`invalid-type-form` diagnostic without evaluating their invalid
children, matching existing error-recovery behavior in both Python and
stub files.
Supersedes https://github.com/astral-sh/ruff/pull/27852.
## Summary
Implement `to_meta_type()` for intersections, removing a Todo type. This
allows removing a special-cased workaround for intersections that was
introduced in https://github.com/astral-sh/ruff/pull/27644
- Implement intersection meta-type projection by combining positive
class constraints, preserving bounded type variables and `Self`,
discarding value-only refinements, and retaining exact enum
alternatives.
- Pass the complete receiver meta-type to descriptors, removing the
intersection-specific owner workaround introduced for enum attribute
inference.
- Preserve nominal class information when an optional type-variable
bound excludes `None`, alongside structural protocol behavior and the
runtime `dict` class of narrowed `TypedDict` values.
## Test plan
Added and updated mdtests cover:
- Nominal class intersections, `type()`/`__class__`, and constructor
round trips.
- Bounded type variables intersected with unrelated classes, optional
bounds narrowed by excluding `None`, and truthiness-narrowed `Self`.
- Positive and negative truthiness constraints, excluded literals, pure
negations, and intersections without a positive class bound.
- Descriptor owners requiring both sides of an intersection.
- Narrowed `TypedDict` values retaining `dict` as their runtime class.
- Callable qualified-name lookup and named-tuple classmethods preserving
their complete intersection owner.
## Ecosystem impact
Most new ecosystem diagnostics expose existing constructor, signature,
or attribute-checking behavior that was previously hidden by `@Todo`:
SciPy includes inherited `object.__init__` errors and duplicate
diagnostics; rotki exposes superclass-constructor selection and strict
generic narrowing; pandas, Sphinx, and SymPy expose existing constructor
or flow-analysis limitations; and Pydantic assigns an undeclared class
attribute. All these same underlying limitations reproduce on `main`, if
you express the class-type intersection directly instead of relying on
`to_meta_type()` to create it. So I consider these out-of-scope for this
PR.
One known false positive remains in koda-validate: a
`NamedTuple`-bounded class passed to `inspect.signature` is rejected.
General callable-intersection support is intentionally deferred to keep
this change focused.
In [Porcupine’s lexer
comparison](https://github.com/Akuli/porcupine/blob/8e34077345317427aa4664d6b1ec556c3c6812e7/porcupine/plugins/highlight/pygments_highlighter.py#L51),
the more precise class type also exposes an existing method-identity
limitation. ty treats `type(self._lexer).get_tokens_unprocessed ==
RegexLexer.get_tokens_unprocessed` as always false, even though a
`PythonLexer` makes it true at runtime. This incorrectly removes an
`unsound-return-statement` warning in the reachable branch. The same
behavior reproduces on the merge base when the class-object intersection
is written explicitly. Handling method identity correctly when
subclasses can override a method is tracked in
[astral-sh/ty#2428](https://github.com/astral-sh/ty/issues/2428) and is
deferred to a separate change.
## Summary
Infer `TypeVarTuple` from arguments passed to `*args`, including
unpacked tuples and iterables, fixed prefixes and suffixes, and nested
type variable tuples.
This PR contains only the core call-binding change. Callback forwarding
is handled separately in #27371 to keep this PR focused and avoid adding
more work to the old constraint solver. Moving `TypeVarTuple` and
`ParamSpec` to the new solver is separate follow-up work.
Continues #25240.
## Test plan
- Pack inference for direct arguments, unpacked arguments, fixed
boundaries, nested packs, and legacy `Unpack`.
- Invalid starred arguments, bounded type variables, and diagnostic
locations.
## Summary
- Delegate bounded and constrained type-variable member lookup to the
complete lookup behavior of each bound or constraint while preserving
the original receiver for descriptors and `Self`.
- Restore enum-derived `name`, `_name_`, `value`, and `_value_`
attributes for implicit `self`, explicit `Self`, and enum-bounded or
enum-constrained type variables.
- Preserve class-object lookup, mixed instance/class-object constraints,
custom enum-property overrides, and classmethod binding for narrowed
`Self` and mixin receivers.
- Fixes https://github.com/astral-sh/ty/issues/4229.
## Test plan
- Extend enum mdtests to cover implicit `self`, explicit `Self`,
explicitly annotated enum receivers, enum-bounded and enum-constrained
type variables, special name/value attributes, explicitly annotated
`_value_`, custom `enum.property` overrides, strict return checking, and
literal-preserving `Self` return values.
- Add generic mdtest coverage for type variables constrained to both
ordinary instances and class objects.
- Extend `Self` mdtests to cover truthiness-narrowed classmethod
receivers and mixins narrowed to unrelated classes.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Previously, we treated unpacked `*args` annotations as ordinary variadic
parameters during call binding, allowing missing tuple elements, excess
positional arguments, and incompatible element types to pass unchecked:
```python
def callback(*args: *tuple[int, str]) -> None: ...
callback() # error: [missing-argument]
callback(1, 2) # error: [invalid-argument-type]
callback(1, "ok", 3) # error: [too-many-positional-arguments]
```
We now align positional arguments with the unpacked tuple's fixed
prefix, homogeneous variadic segment, and fixed suffix, preserve each
element's expected type through ordinary call checking, and enforce the
tuple's minimum and maximum arity.
The same path validates callable protocols and specializes bounded type
variables, including ordinary type variables beside an unresolved
`TypeVarTuple`.
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
## Summary
Fix `NewType` disjointness and `is` narrowing by distinguishing runtime
objects from typed inhabitants. A typed inhabitant consists of a runtime
object plus any invisible static tags, such as a generic specialization
or a `NewType`. Two types overlap when they admit the same typed
inhabitant, not merely the same runtime object.
`NewType` constructors return the original object unchanged, so distinct
tags can represent different typed inhabitants of the same runtime
object:
```py
from typing import NewType
UserId = NewType("UserId", int)
OrderId = NewType("OrderId", int)
NestedUserId = NewType("NestedUserId", UserId)
UserId(True) is OrderId(True) # True at runtime.
```
The inhabitants `(True, UserId)` and `(True, OrderId)` are distinct, so
`UserId` and `OrderId` remain disjoint. Ordinary types do not require
either tag, so `UserId` still overlaps `int`, `bool`, `Literal[True]`,
and other nominal, structural, or literal types that overlap its
concrete base. `NestedUserId` overlaps `UserId` because a nested tag
remains a subtype of its parent.
Apply these distinctions consistently to identity and narrowing:
transfer object-wide generic guarantees and runtime-value constraints,
but never transfer another operand's `NewType` tag or potentially tagged
type variable. Preserve each operand's own tags and negations. Narrowing
through `isinstance`, class patterns, type guards, enum equality, and
exhaustive enum matches retains the original `NewType` tag.
Fixesastral-sh/ty#4187.
## Test plan
- Disjointness and intersection mdtests cover nominal and structural
base overlaps, literals, boolean/numeric types, enum members, type
guards, distinct and nested `NewType`s, and invariant, covariant, and
gradual generic specializations.
- Identity mdtests cover operand-specific `NewType` tags, static versus
runtime negations, invariant and covariant generics,
unconstrained/bounded/constrained type variables, singleton values, and
compatibility with existing string-literal narrowing.
- Narrowing mdtests cover `isinstance`, `TypeIs`, class patterns,
branded enum members and complements, nested brands, cross-enum and
custom equality, exhaustive enum/`IntEnum` matching, and recursive
aliases.
## Summary
PEP 484's numeric tower means we represent a `float` annotation as the
semantic union `int | float*` and `complex` as `int | float* |
complex*`. We previously rendered those internal unions directly, which
exposed an implementation detail in user-facing output:
```python
def f() -> float:
return 1
reveal_type(f()) # Before: int | float; after: float
```
This change recognizes complete numeric-tower unions during display and
renders them using their canonical annotation spelling, `float` or
`complex`.
The implementation does _not_ preserve explicit unions, e.g., if the
user writes out `int | float`, we still collapse that rather than trying
to preserve it, mostly for simplicity (as per
https://github.com/astral-sh/ty/issues/2184#issuecomment-4345219164).
Closes https://github.com/astral-sh/ty/issues/2184.
## Summary
Reject `Self` in method signatures when an explicitly annotated receiver
is anything other than `Self` for an instance method or `type[Self]` for
a class method, matching the typing specification, mypy, and Pyright:
```python
from typing import Self, TypeVar
T = TypeVar("T")
class Example:
def invalid(self: T) -> Self: ... # error: [invalid-type-form]
@classmethod
def invalid_classmethod(cls: type[T]) -> Self: ... # error: [invalid-type-form]
def valid(self: Self) -> Self: ...
```
We infer explicitly annotated method receivers before the rest of the
signature and report incompatible occurrences through the existing
`Self` special-form checks. Receiver classification is shared with
implicit `self`/`cls` inference, and invalid annotations retain their
`Self` fallback type.
Each `Self` occurrence is diagnosed independently, including quoted
annotations, PEP 695 generic methods, normalized unions such as `Self |
object`, and lazy aliases such as `Identity[Self]`. Diagnostics
highlight the offending token and respect per-occurrence suppressions.