## Summary
Today, ty panics on `class C(*(int, Generic[T, U]))` when `T` has a
default and `U` is required. To locate the error, ty looks for the
second base in the class header. The header contains a single starred
expression that expands to two bases.
This PR changes the diagnostic to use the source expression ty recorded
for each base during expansion, following #24695 and #24699. The
diagnostic now underlines `T, U` and reports that a required type
parameter cannot follow one with a default.
Added an mdtest that checks the diagnostic on the unpacked generic base.
## Summary
This PR splits `mismatched-type-name` into eight independently
configurable warnings for `NewType`, functional enums, `TypeVar`,
`ParamSpec`, `TypeVarTuple`, `namedtuple`/`NamedTuple`, `TypedDict`, and
`TypeAliasType`.
~~This PR is actually quite a bit less breaking for users than other
rule splits we've done over the last few days. Because
`mismatched-type-name` is no longer the canonical name for _any_ rule on
this branch, we're able to make it a deprecated alias for _all_ the new
rules, meaning that it will continue to work on the CLI, in ty
configuration files, and ty suppression comments (we'll just emit a
warning diagnostic telling users to upgrade their configuration
file/suppression comment).~~
~~A more specific rule takes precedence over the deprecated name if both
are given within the same configuration table. On the command line, the
last matching flag wins (e.g. `--error=mismatched-enum-name
--ignore=mismatched-type-name` will suppress diagnostics for mismatched
enum names).~~
(After review from @MichaReiser, I backed out these changes. It's now a
somewhat breaking change, because the rule is now marked as "removed"
rather than "deprecated".)
As well as splitting the rule into eight separate rules, it also cleans
up our error recovery for several of these type factories. For example,
in the following snippet, we fail to recognize `T` as a `TypeVar` due to
bad error recovery from the non-literal name. But we don't really need a
nonliteral name in this case to determine that it's a validly
constructed TypeVar. It's probably a mistake, but there's no _soundness_
issue here:
```py
from typing import TypeVar
def f(x: str):
# main emits an error: `invalid-typevar`
# this PR emits a warning: `mismatched-typevar-name`
T = TypeVar(x)
def g(x: T) -> T:
return x
reveal_type(g(42)) # main: Todo. This PR: `Literal[42]`.
```
## Motivation
The main motivation here is that having a namedtuple or enum with the
"wrong name", for example, just seems much less "obviously wrong" than
having a typevar or paramspec with the "wrong name". The [ecosystem
report](https://github.com/astral-sh/ruff/pull/29957#issuecomment-6603964612)
is evidence of this. namedtuples and enums define runtime classes that
can be used in runtime contexts and instantiated; these constructs
aren't _just_ for type checkers.
TypeVars/ParamSpecs/TypeVarTuples/NewTypes/TypedDicts are all _only_ for
type checkers and there's no plausible reason why you'd want to give the
type a different name to the variable it's assigned to.
`mismatched-namedtuple-name` and `mismatched-enum-name` are therefore
now disabled by default, matching the default severity we already
settled on for `mistmatched-sentinel-name` in
https://github.com/astral-sh/ruff/pull/27206. `mismatched-typevar-name`,
`mismatched-paramspec-name`, `mismatched-typevartuple-name`,
`mismatched-enum-name`, `mismatched-namedtuple-name` and
`mismatched-newtype-name` all remain enabled by default with `warning`
severity.
When a generic function expects a callback that returns `Id[R]`, where
`type Id[T] = T`, ty can lose the relationship between `R` and a type
variable from the caller.
```py
from collections.abc import Callable
from typing import reveal_type
type Id[T] = T
def run[R](callback: Callable[[], Id[R]]) -> R:
return callback()
def check[T](callback: Callable[[], T]):
reveal_type(run(callback)) # Before: Unknown. After: T@check.
```
When an alias bound expands to a type variable, record the relationship
between the two variables. In this example, the solver can infer `R`
from the callback's return type `T`.
This is just an edge case Codex noticed, not something that showed up in
real code, so it's quite low priority; it just seemed like a very simple
and clearly correct fix.
## Summary
In this code on ty 0.0.107, `wrap(lambda n: n + 1)` alone works fine,
but passing that same result straight to `register` makes ty report an
error on `n + 1`.
```python
from collections.abc import Callable
def wrap[**P](f: Callable[P, int]) -> Callable[P, int]:
return f
def register[**P](f: Callable[P, int]) -> None: ...
wrap(lambda n: n + 1) # OK
register(wrap(lambda n: n + 1)) # Error in ty 0.0.107
```
`P` stands for all the parameters of the callback. Neither `wrap` nor
`register` says what type `n` has. In the nested call, we wrongly gave
`n` the type of the whole parameter list, `(*args: object, **kwargs:
object)`, and then rejected the addition.
With this fix, `n` stays `Unknown` in both calls. If the callback
signature does give a concrete type for a parameter, ty still uses it.
Fixesastral-sh/ty#5075.
ty could infer `Unknown` for a call when one overload left a parameter
unannotated and another supplied a type for it. In this example,
`first()` has no arguments to infer `T` from, so its return type
defaults to `int` unless the caller supplies an expected type.
```python
from typing import overload, reveal_type
from typing_extensions import Never, TypeVar
T = TypeVar("T", default=int)
def first(*values: T) -> T:
return values[0]
@overload
def choose(value, index: int) -> int: ...
@overload
def choose(value: Never, index: object) -> object: ...
def choose(value, index):
return index
reveal_type(choose(first(), 1)) # previously Unknown, now int
```
Both overloads initially match. The first gives `first()` no expected
type, so it infers `int`. The second infers `Never` for the same
expression. Previously, ty reused `Never` while checking the first
overload and treated the call as ambiguous. It now uses the argument
type inferred for each overload, so the first overload wins.
## Test plan
Checked the original example with a nested `create()` whose return type
is a type variable bounded by `int`. ty reveals `int` for
`choose(create(), 1)`, including when `create()` is replaced by
`next(iter([]))`.
On main, we hang on this snippet:
```python
class Property[T]:
value: T
class Either[T](Property[T]):
def __init__[A, B, C](
self: "Either[A | B | C]",
first: type[Property[A]],
second: type[Property[B]],
third: type[Property[C]],
) -> None: ...
result = Either(Property[float], Property[str | None], Property[bytes])
```
The three arguments determine `A`, `B`, and `C`, so `result` should have
type `Either[float | str | None | bytes]`. Previously, we kept
rebuilding the same union in different orders, treating each order as a
new result, and cycling forever.
ty now substitutes one type variable at a time. It lets other bindings
resolve a cycle before choosing a fixed point or falling back to
`Unknown`. When two variables refer back to each other, ty also checks
their bounds. For example, a constructor’s `int` bound still applies
when the class parameter itself is unbounded.
Fixesastral-sh/ty#5057
I tried all six orders of `A`, `B`, and `C` in the `__init__`
declaration above. With the call unchanged, ty inferred `Either[float |
str | None | bytes]` every time.
## Ecosystem report
Bokeh finished in 20 seconds after timing out at 180 seconds on the
base. Across five other projects, 58 existing diagnostics changed which
unresolved type variables ty displayed. Both revisions rejected the same
operations, and typed array inputs retained their concrete dtype. No
stable diagnostics were added or removed. [Full report for
`baf1faa7`](https://github.com/astral-sh/ruff/pull/30510#issuecomment-6965510325)
When inferring a specialization for a generic call, we try each element
of a union return context individually before falling back to the entire
union. If a nested call fails during one of those attempts, its error
recovery type can make the proposed specialization appear valid even
though it isn't.
There is an open constraint solver bug (astral-sh/ty#4696) where we
don't verify that all solutions collectively "cover" all of the valid
specializations of any _non-inferable_ typevars. This can cause us to
infer a provisional specialization that is incompatible with the actual
arguments of a nested call. We then incorrectly accept the narrowed
specialization because the nested call's recovery type satisfies the
expected parameter type.
For now, we work around this by recording when call binding fails, and
threading that information through `InferContext` so that `try_narrow`
can reject speculative inference attempts that depend on error recovery.
We track these failures even when their diagnostics are suppressed, but
ignore failures in unreachable code. For calls with a single candidate
signature, we also allow narrowing to succeed when the failed argument
corresponds to a parameter whose type contains no typevars, since that
argument cannot influence the proposed specialization.
There are still limitations around callable unions and overloads, which
#27942 will address.
Closesastral-sh/ty#4971
## Summary
This PR adds a new `project-manager` table:
```toml
[project-manager]
project = "uv" | "auto" | "none"
project-mode = "sync" | "metadata"
script = "uv" | "none"
```
This PR only implements the script part, but I listed all options to
outline where we want to go with this in the future.
`script = "uv"` opts in to installing PEP 723 script dependencies with
uv (to reduce the need for `TY_UV`). For now, `project-manager.script`
defaults to `"none"`, but I plan to change this to `"uv"` in the near
future. The new option follows the same configuration precedence as
other script settings:
* A script with its own `tool.ty` section uses the value from its own
table to control whether it uses uv to sync its dependencies or not.
* Scripts without a `tool.ty` section use the project level
`project-manager.script`
For now, overriding the setting based on file patterns isn't supported
(I have no strong reason against it, it's mainly to cut scope).
`TY_UV` takes precedence over any setting in the configuration (similar
to other CLI options that support environment variables).
The recommended way to check untrusted code is to set `TY_UV=off`, which
overrides `project-manager.script`.
For now, we keep supporting `experimental.useUv` for scripts, but the
in-editor `configuration.project-manager.script` takes precedence over
`experimental.useUv`.
## Why is the setting not under `environment.
For projects, setting `project-manager.project = "metadata"` also
changes how we discover the project (we prefer the workspace over the
closest ty project), and `metadata` doesn't initialize the Python
environment.
## LSP watch mode
Changing `project-manager.script` at the project level requires saving
any open script (or closing and reopening) for the sync to start. This
simplified the implementation a fair bit and seems an okay trade off.
The "gradual parameters" `*args: Any, **kwargs: Any` are special-cased
to match any possible set of arguments, even when they follow other
"normal" parameters in a signature. But they do not make the _entire_
signature gradual -- calls still must satisfy the other normal
parameters present.
We currently are "too gradual" in our treatment of these signatures,
allowing a callable with an initial typed parameter plus gradual
parameters to be assigned to a callable type that has the wrong type for
its first parameter:
```python
from typing import Any, Callable
def lowercase(value: str, *args: Any, **kwargs: Any) -> str:
return value.lower()
callback: Callable[[int], str] = lowercase # Previously accepted by ty
callback(1) # AttributeError
```
This PR rewrites callable assignability (with a significant net
reduction in Rust LOC!) and now correctly reports the error on this
assignment. The first positional argument binds to `value`, which
requires a `str`. The gradual `*args` and `**kwargs` only describe the
remaining arguments.
Closesastral-sh/ty#5052.
## Ecosystem report
All 104 public ecosystem changes on `8681ef368f` are expected. Most new
errors identify callbacks or overrides that cannot accept an argument
their types promise. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/30509#issuecomment-6966654399)
## Summary
Fixesastral-sh/ty#5035.
When targeting Python 3.12 or earlier, infer covariance for read-only
fields on Pydantic models frozen through a class keyword or
`ConfigDict`. Use the same setter lookup as assignment checking so a
custom `__setattr__` keeps the model invariant. On Python 3.13 and
later, the synthesized `__replace__` still makes these models invariant.
On main, `functools.partial` rejects a union-valued keyword when
different overloads accept its members and a required argument is left
for the caller. For example, ty accepts binding `option="text"` or
`option=None` separately, but reports `Expected str, found str | None`
here.
```python
from functools import partial
from typing import overload
@overload
def f(value: int, /, *, option: str) -> str: ...
@overload
def f(value: int, /, *, option: None) -> int: ...
def f(value: int, /, *, option: str | None) -> str | int:
return value if option is None else option
def apply(option: str | None) -> str | int:
bound = partial(f, option=option)
return bound(1)
```
When overload resolution retries with each union member, it matches
parameters again. This used to eliminate both overloads because `value`
was missing. The fix defers the missing-argument check on every match
and keeps all compatible overloads for the eventual call. In the
example, ty infers `str | int` for `bound(1)` because the result depends
on the stored keyword. An explicit override selects the corresponding
overload, so `bound(1, option=None)` returns `int`.
For an overloaded function whose two-positional-argument form requires
`option=None`, a partial `p` that stores an `option` of type `str |
None` rejects `p(1, 2)`. It accepts `p(1, 2, option=None)` because the
caller replaces the stored value.
## Test plan
Checked `partial(datetime.replace, tzinfo=tz)` in Pydantic with the
proposed typeshed sync from #30443, which splits `tzinfo` and `None`
into separate overloads. The fix removed the diagnostic on this call. A
direct check confirmed that the partial accepts a datetime receiver and
a `tzinfo=None` override, and rejects a missing or integer receiver.
[Pydantic call
site](https://github.com/pydantic/pydantic/blob/560f0f96a37d6913afdfddb2de22030ef08e68c3/pydantic/experimental/pipeline.py#L313)
ty could spend minutes inferring the type of a Bokeh `Either`
constructed from a property with 28 literal marker names and a property
accepting `str`. To rule out a candidate type, the solver explored
combinations of bounds for every marker. Rejecting both bounds for just
one marker is already a counterexample.
Before searching for a counterexample, ty now tries one path that makes
each atomic constraint on it false. It follows that path only while the
diagram already has the next node, so it never joins branches or
backtracks. If the path reaches false and its assignments are
consistent, ty can rule out the candidate immediately. Otherwise it uses
the existing exact search.
Fixesastral-sh/ty#5046.
## Test plan
I checked [Bokeh’s original
`dataspec.py`](https://github.com/bokeh/bokeh/blob/1c2309202d8623ede74a996a367f3b60eaf79411/src/bokeh/core/property/dataspec.py#L408-L409)
with the base and PR Linux binaries, the same checker configuration, and
Bokeh’s dependencies in a Python 3.12 environment. The base binary
exceeded a 20-second timeout. Removing just the reported constructor
call let it finish in 0.69 seconds. The PR binary checked the unmodified
file in 0.80 seconds with no diagnostics. Using Bokeh’s classes
directly, it also accepted `assert_type(Either(MarkerType,
Regex("^@.*$")), Either[str, str])`.
## Summary
ty previously accepted `call_with_int(callback)` below, even though a
caller can pass `forward` a function that takes no arguments.
```python
from typing import Callable, Concatenate
def call_with_int(callback: Callable[Concatenate[int, ...], None]) -> None:
callback(1)
def forward[**P](callback: Callable[P, None]) -> None:
call_with_int(callback) # now an error
```
Check the fixed positional parameters before the caller's `ParamSpec`
against the prefix required by `Concatenate`. This also rules out
overloads that require more positional parameters than the callback
guarantees.
This fixes a bug that otherwise caused new false-positive regressions on
https://github.com/astral-sh/ruff/pull/30423.
## Ecosystem report
The public run for `deb2cd0da5` found three expected changes across 165
projects. ty now reports a missing class receiver in a
pytest-autoprofile wrapper annotation and removes two incorrect
overload-related errors in discord.py. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/30470#issuecomment-6927397479).
ty used to abort with a stack overflow while checking this
`super().__init__` call. It now reports `invalid-argument-type` on
`value` in `super().__init__(value)`.
```python
class Box[T]:
def __init__[A, B](self: "Box[A]", value: A | tuple[B]) -> None: ...
class Child[T](Box[bytes]):
def __init__(self, value: T | tuple[T]) -> None:
super().__init__(value)
```
The receiver for that call is a `Child[T]`, which is already a
`Box[bytes]`. That fixes `A` to `bytes`. A one-element tuple is valid
because the initializer can infer `B = T`. But `Child[str]("hello")`
would forward a plain string where the initializer expects `bytes` or a
tuple.
ty now uses the receiver’s existing class type arguments when checking a
constructor method. Looking up the method on a bare generic class still
allows the call to infer those arguments.
The stack overflow happened while ty was trying to explain the failed
call. It combined type-variable bindings inferred from individual
arguments, then repeatedly substituted them into each other. A cycle
through a tuple element can add another layer of `tuple[...]` on every
substitution. The original reproducer had the same problem with a
callable’s return type. ty now drops the parts of a binding that would
keep growing and uses the remaining evidence to report the error.
Fixes https://github.com/astral-sh/ty/issues/5047
## Test plan
Compared the CLI on the merge base and this change using the example
above and the original reproducer in both PEP 695 and legacy `TypeVar`
syntax. All three aborted on the merge base and produced argument-type
diagnostics on this change. The example above passes with `Box[T]` as
the base.
## Ecosystem report
On `3c94d4a654a3c2cc6a05b4845d4057271599ebb0`, six new diagnostics
identify constructors that accept arguments incompatible with the
receiver’s specialization. The change also removes a false return
warning in PyMongo and simplifies two existing setuptools diagnostics.
All nine changes were reproduced, with no new regressions found. [Full
ecosystem
report](https://github.com/astral-sh/ruff/pull/30473#issuecomment-6931764426)
## Summary
The `ty-extensions` package lets Python evaluate and inspect
`Intersection[A, B]` and `Not[A]` at runtime. Update the stub docstrings
and the `experimental-syntax` documentation to point readers to the
package.
We internally declare the parameter type of the `assert_never` value to
be `Any`. However, this declared type should be ignored when inferring
the value expression, as it can sometimes prevent a nested generic call
from inferring `Never`, incorrectly silencing the assertion:
```py
from typing import Iterator, Never, assert_never
def _(items: Iterator[Never]):
assert_never(next(items)) # main: error[type-assertion-failure], now: ok
def result[T = Never]() -> T:
raise NotImplementedError
assert_never(result()) # main: error[type-assertion-failure], now: ok
```
Reported in
https://github.com/astral-sh/ty/issues/4988#issuecomment-6779572903.
An inferred attribute type should not provide a declared type preference
to later assignments, allowing it to be overwritten by a gradual type
without being restricted to the original concrete type:
```py
from typing import Any
def identity[T](value: T) -> T:
return value
class Fields:
class_value = None
def __init__(self, value: Any):
self.value = None
self.value = identity(value)
reveal_type(self.value) # main: None, now: Any
```
Resolves https://github.com/astral-sh/ty/issues/4974. Part of
https://github.com/astral-sh/ty/issues/4988.
The `TypeContextKind` of a given `TypeContext` should be preserved when
propagated to a nested expression. For example, an upper bound `T:
Payload` does not provide a solver preference for gradual types, and so
should not provide a preference to any fields of the `TypedDict`:
```py
from typing import TypedDict, Any
class Payload(TypedDict):
value: int
def payload[T: Payload](value: T) -> T:
return value
def identity[T](value: T) -> T:
return value
def _(value: Any):
payload(reveal_type(identity(value))) # revealed: Any
payload({"value": reveal_type(identity(value))}) # main: int, now: Any
```
There are a few places where we are incorrectly constructing a
`TypeContext` with the default `TypeContextKind` instead of preserving
it from the outer `TypeContext`, which this PR fixes.
This PR allows `type[...]` types to participate in equality narrowing
(unsoundly, in the face of hypothetical subclasses with metaclass
`__eq__`), in non-strict equality mode. This improves consistency with
the behavior of other types in non-strict mode, where we generally
assume that hypothetical subclasses are "well behaved" and don't
override dunder methods in unexpected ways.
```python
class Base: ...
class Child(Base): ...
def _(cls: type[Base]) -> Base:
if cls == Child:
reveal_type(cls) # revealed: <class 'Child'>
```
The same equality rule narrows `cls in (Child, OtherChild)` to the union
of the two exact class objects and narrows `x` after `type(x) == Child`.
We don't do this narrowing in strict-equality mode. We also don't do it,
even in non-strict mode, if we can see a concrete metaclass `__eq__` on
either metaclass.
Some codebases might see a loss of narrowing from
https://github.com/astral-sh/ruff/pull/29346, since it can cause nested
classes to infer as a `type[...]` type rather than a class-literal type.
This restores that narrowing.
## Ecosystem report
At `5d76dc306f`, the public ecosystem run showed 17 expected diagnostic
changes across four projects. Exact-class checks remove false positives
and improve return-type diagnostics. In static-frame, ty now flags an
unreachable branch in a test helper. All 17 changes were reproduced,
with no regressions found. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/30393#issuecomment-6843608044)
A class that defines `__eq__` and omits `__hash__` gets `__hash__ =
None` at class creation. ty previously found `object.__hash__` instead
and accepted instances of the class and its subclasses where `Hashable`
was required.
Account for the implicit `None` during class-member lookup. Subclasses
inherit it and can restore hashing by defining `__hash__`. Explicit and
generated hashes take precedence, including hashes supplied by frozen
dataclasses and Pydantic models. An equality protocol's own implicit
`None` also stays separate from the hash requirements it imposes on
structural implementations.
Fixes https://github.com/astral-sh/ty/issues/4941.
## Test plan
Compared ty with CPython on the issue reproducer. Python raised
`TypeError` for the equality-only class and its subclass, and accepted
the subclass that restored hashing. ty now rejects the same two
instances for a `Hashable` parameter.
Checked Pydantic 2.13.4 at runtime with frozen models that define
equality and with a base that provides a custom hash. Pydantic generated
hashes for the frozen models, preserved the inherited custom hash when
equality was inherited, and left `__hash__ = None` on the subclass that
redefined equality over the custom hash.
## Ecosystem report
The public run for
[`ac783ae`](https://github.com/astral-sh/ruff/commit/ac783ae505378aa9b46acd4e082f0edf6f5e678c)
found 23 stable diagnostic changes across five projects. We accept three
JAX false positives for this fix. ty loses the relationship between a
mask and the callable selected for its type. The other 20 changes follow
from recognizing equality-only classes as unhashable, including
corrected indexing overloads in static-frame and a missing hash
declaration in pandas-stubs. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/30145#issuecomment-6700457278)
ty could drop valid alternatives from a union when a protocol matched
only the first of several literals. For example, if `NamedRed` requires
a `name` of `Literal["RED"]`, ty simplified `Literal[Color.RED,
Color.GREEN] | NamedRed` to `NamedRed` and rejected a return of
`Color.GREEN`. Reversing the operands kept `Color.GREEN`. This was
because we failed to recognize that the protocol could distinguish
between the grouped literal elements, and wrongly allowed one element to
stand in for the whole group when compared against the protocol.
The union builder now checks each individual literal for redundancy when
the other operand is a protocol. This also handles protocols that
distinguish integers by their numerator or strings and bytes by their
length.
The builder defers comparisons between an added type alias and grouped
literals until it expands the alias. An alias of `int` can then subsume
an entire integer literal group in one check. An alias of e.g.
`NamedRed` still compares each enum member individually.
ty could panic when checking a decorated class if it inferred the return
type of an inner decorator and an outer decorator returned a class from
another module. During cycle recovery, ty could use that replacement
class's scope to look up names in the file that defined the original
class.
Preserve the original class before applying decorators and use it for
class-definition checks. The decorated name still takes the type
returned by the outer decorator.
Fixesastral-sh/ty#5021.
## Test plan
Checked the reported `EnumCheck` example with return type inference
enabled. Before this change it panicked. With the fix, ty reports the
invalid argument to `@d2` and finishes checking the file.
ty could reject `dict.fromkeys(...)` when the expected value type was a
union containing a bounded type variable. It could also reject a fresh
`dict(rows.items())` passed to `MappingProxyType` when the return
annotation supplied a wider key type.
Both cases lost a valid specialization while comparing an intersection
with an upper bound. For static types `A` and `B`, `(U & ~A) ≤ B` is
equivalent to `U ≤ B | A`. Checking the positive element alone gave the
solver the stronger constraint `U ≤ B`. This change keeps the excluded
types in the upper bound and checks the whole target before splitting a
union into its arms.
Fixesastral-sh/ty#4851Fixesastral-sh/ty#4967
## Test plan
Compared the reported examples across ty versions. The
`MappingProxyType` example passes in 0.0.98, fails in 0.0.99, and passes
with this change. With `unsound-return-statement` enabled, the reported
`list()` and `list((None,))` constructors also infer the annotated
element type.
A subclass can give a class variable from the base class a new default,
and it can annotate that assignment with bare `ClassVar`. Previously, we
rejected `Record` below because it inferred a one-element tuple for
`group_by`, which we considered an `invalid-mutable-override` (narrowing
the type of a mutable attribute), and then we also rejected the
subsequent assignment to `Record.group_by` as incompatible with the
inferred one-element tuple type.
```python
from typing import ClassVar
class Message:
group_by: ClassVar[tuple[str, ...]] = ()
class Record(Message):
# Previously we inferred `ClassVar[tuple[str]]` as the type of this attribute,
# which is both an `invalid-mutable-override`, and disallows the assignment below.
# Now we "fill in" the bare `ClassVar` with the inherited type instead of the inferred
# type, leaving the declared type of the attribute as `ClassVar[tuple[str, ...]]`.
group_by: ClassVar = ("record",)
Record.group_by = ("record", "source") # previously an error, now OK
```
Ty now checks the default against `Message.group_by`. Inside the
`Record` class body, ty knows that `group_by` is `("record",)`. Later
assignments through `Record` can use any tuple of strings.
A subclass can still redeclare the type explicitly with `ClassVar[...]`.
Follow-up to #30293.
## Ecosystem report
At
[`20ae89c`](https://github.com/astral-sh/ruff/commit/20ae89ca2d779e5cdbdb13d48cc0e26998731662),
ty removes 39 false-positive override errors across five projects. All
39 came from subclasses repeating a bare `ClassVar` with a valid
default. The public run found no new stable diagnostics or project
failures. [Full report and
examples](https://github.com/astral-sh/ruff/pull/30396#issuecomment-6841752291)
ty now accepts `Child.Meta` as an override when it inherits from
`Base.Meta`. Previously, ty rejected the override because the two class
objects are distinct.
```python
class Base:
class Meta: ...
class Child(Base):
class Meta(Base.Meta): ... # OK
class Other(Base):
class Meta: ... # Error: unrelated to Base.Meta
```
`Base.Meta` can still safely reveal the singleton class type of `Meta`,
since the expression `Base` has a singleton class type that doesn't
include subclasses. But a caller reading `obj.Meta` could get
`Child.Meta`, so ty allows for subclasses in the inferred type.
```python
reveal_type(Base.Meta) # <class 'Meta'>
def inspect(obj: Base, cls: type[Base]) -> None:
reveal_type(obj.Meta) # type[Meta]
reveal_type(cls.Meta) # type[Meta]
```
The nested class declaration is read-only. If `obj` is a `Child`,
assigning `Base.Meta` would replace its more specific `Child.Meta`;
making `Base.Meta` read-only allows `Child` to safely narrow its type.
```python
def update(obj: Base) -> None:
obj.Meta = Base.Meta # Error
del obj.Meta # Error
```
A subclass can explicitly declare a writable instance attribute when its
instances need to replace `Meta`.
```python
class Configurable(Base):
def __init__(self) -> None:
self.Meta: type[Base.Meta] = Base.Meta
def configure(obj: Configurable) -> None:
obj.Meta = Child.Meta # OK
```
The same read-only rule applies to nested classes declared on protocols
and metaclasses.
### Django
We have a special-case for Django in main allowing inner `Meta` class
overrides on Django models. This PR doesn't remove that special case.
Two abstract Django models can each declare an unrelated `Meta`. A model
inheriting from both uses the first base’s `Meta` options by default.
Ordinary override checking would report a conflict between those
classes, so ty keeps the Django exception for this case. [Django’s
`Meta` inheritance
rules](https://docs.djangoproject.com/en/5.2/topics/db/models/#meta-inheritance)
Closes https://github.com/astral-sh/ty/issues/4711.
## Test plan
Compared Rust type layouts on aarch64 with debug assertions enabled and
disabled. Widening `TypeQualifiers` to `u16` fit in existing padding in
`TypeAndQualifiers`, `PlaceAndQualifiers`, and `ProtocolMemberData`. All
three kept their original size.
## Ecosystem report
The public comparison found 168 stable diagnostic changes on
`c2d14afc02318e6fdf041a6983447aa7fb6008a6`. All follow from the new
nested-class behavior. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/29346#issuecomment-6172170919)
- Spark gained two default warnings and Alerta gained eight opt-in
warnings. Both use a nested class accessed through an instance as a
base, so ty can no longer determine the exact method resolution order.
- Three diagnostics disappeared in Spack and Freqtrade.
- 155 existing override errors now display the inferred class-object
types.
A bare `Final` or `ClassVar` leaves the value type to inference.
Previously, ty passed `Unknown` as the expected type of the initializer.
This could change the result of a generic call and let assignments to a
mutable class variable accept values incompatible with its initializer.
Track whether a qualifier has a type argument. For example, `header:
ClassVar = bytes.fromhex("fd")` exposes `bytes` on the class and accepts
subsequent `bytes` assignments through the class. An explicit
`ClassVar[Unknown]` still exposes `Unknown`.
A bare `Final` instance attribute gets its type from the declaring
class's initializer. A subclass cannot initialize that same final
attribute. For a bare final dataclass field with an ordinary default,
the generated constructor accepts the inferred field type.
A generic call can infer its type arguments from its inputs when it
appears in `yield` or `yield from` in a generator without a return
annotation. The same applies when the call is passed to a callable with
an unknown signature. An explicit parameter or yielded element type of
`Any` still supplies type context.
Split from #30244, which changes generic type-argument inference. The
ordinary dataclass default case overlaps #29576, which also handles
field specifiers and missing defaults. #30255 covers public reads of
module-level finals with an explicit type argument.
Fixesastral-sh/ty#4989.
## Ecosystem report
At
[`28a3e59`](https://github.com/astral-sh/ruff/commit/28a3e5901958260777e605b2ce27477f995dbbab),
ty detects 43 incompatible class-variable overrides across six projects,
removes two false return warnings in pytest, and updates the messages
for three existing errors. All 48 stable changes are expected. [Full
report and
examples](https://github.com/astral-sh/ruff/pull/30293#issuecomment-6782994357)
ty rejected this assignment even though `max` returns one of the two
input tuples and both contain only `"red"` and `"blue"`.
```python
from typing import Literal
short = ("red",)
long = ("red", "blue")
longest: tuple[Literal["red", "blue"], ...] = max([short, long], key=len)
```
ty widened the tuple elements to `str` while inferring the list passed
to `max`, so it reported `invalid-assignment` for a result of type
`tuple[str] | tuple[str, str]`. The annotation on `longest` already
supplies an upper bound that rules out that widening. This change checks
proposed promotions against that bound and preserves the literals in
this example.
When a union in the expected return type gives the solver several
constraint paths, ty widens type arguments together only if one path
permits all those promotions. If the paths disagree on which arguments
can widen, ty tries the promotions they share and checks that a complete
path still permits them. This preserves literals where needed and keeps
the result independent of type-variable declaration order.
Fixes astral-sh/ty#4968.
## Test plan
Bisected the original Python 3.12 example to #26680 and checked that
this change accepts it.
## Summary
unyt keeps a set of 118 NumPy functions and adds more with `|=`.
Checking this reduction with NumPy 2.5.4 drops from 12.5 seconds to 0.21
seconds in matching local builds. [unyt’s function
registry](https://github.com/yt-project/unyt/blob/fc1b1299e7b5aab58368ce92620dc7eec2a50db5/unyt/tests/test_array_functions.py#L42-L192)
```python
import numpy as np
functions = {np.linalg.multi_dot}
functions |= {np.linalg.multi_dot}
```
Several overloads of `multi_dot` declare their own type variables. When
ty collected the type variables in the set operation’s constraints, it
included those local variables. Following specialized alias arguments in
#30150 made this particularly expensive for NumPy. The solver explored
combinations involving variables that only need to be inferred when
`multi_dot` itself is called.
We now track which variables each callable signature binds and collect
only the free variables of a constraint bound. A callback can still
depend on a type variable from its enclosing scope. Checking all of unyt
3.2.0 produces the same 375 diagnostics as main.
Closes https://github.com/astral-sh/ty/issues/4990
---------
Co-authored-by: Douglas Creager <dcreager@dcreager.net>
Unary `+`, `-`, and `~` could report an invalid return type when a
method returned `Self`. For example, given `x: T` with `T` bounded by
`Base`, ty inferred `Base` for `-x` even though `x.__neg__()` returned
`T`.
Use the upper bound for literal folding and the original operand as the
receiver when binding the unary method. For a union bound, each method
binds to the part of the operand that belongs to its union member. This
also keeps the diagnostic that identifies a member missing the operator.
For an intersection bound, ty checks the call on each component and
combines the return types of the matching calls.
Fixes https://github.com/astral-sh/ty/issues/4983
## Ecosystem report
At `b79065f153a6c39d5c4ea2c53f6b11f0ee553a9b`, ty removes three false
positives in Ibis, PySpark, and SymPy. In SymPy’s matrix subtraction,
preserving the operand type also exposes an existing decorated-method
inference issue (astral-sh/ty#2520). The unsupported addition and its
return warning move from reflected subtraction to forward subtraction,
replacing an incorrect return-type error there. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/30242#issuecomment-6757597391)
## Test plan
Compared a debug build to the base revision on a file with 300 `with x`
statements and a 48-class union bound. Three runs of each build used
0.42–0.43 seconds of user CPU. This checks that other implicit method
calls on union-bounded type variables still benefit from cached lookup.
## Summary
ty can panic when two recursive `NamedTuple` fields share a wide union.
For example, both fields in the regression test simplify to `int |
Leaf`, but `Leaf` itself refers back to the two named tuples. Checking
whether we can drop `Leaf` takes us back to the union we're still
building. Its cached result can alternate between `int` and `int | Leaf`
until ty panics.
We now build the union directly in the caller when the cache lookup
encounters a cycle. Both fields resolve to `int | Leaf`.
ty 0.0.99 overflows its stack on the call to `map_leaves` inside `visit`
below. It should report a return-type error instead. If `V` is `int`,
for example, `visit` should return an `int` but `map_leaves` returns a
`Tree[int]`, which expands to `int | tuple[Tree[int]]` -- the tuple part
is not compatible with the required return type.
```python
from collections.abc import Callable
type Tree[T] = T | tuple[Tree[T], ...]
def map_leaves[T, U](fn: Callable[[T], U], value: T) -> Tree[U]:
raise NotImplementedError
def visit[V](value: V) -> V:
return map_leaves(lambda leaf: leaf, value)
```
## Why ty specializes alias arguments
Suppose ty knows `U = int` and encounters `Tree[U]`. Expanding the
definition would lead straight back to `Tree[U]` inside the tuple.
Before #24773, expanding that infinite recursion would just fall back to
`Tree[Any]` at the recursion point and lose type information. After
#24773, we can re-specialize `Tree[U]` as `Tree[int]` directly without
needing to expand the alias.
#24773 added the `as_specialization` method because substituting into an
alias's stored arguments requires a mapping with one entry per type
parameter. Given a partial mapping, the helper kept the original type
variable. But there are two cases where it did that, and only one of
them is correct:
**Missing slot.** Suppose we know only `T = int` for `map_leaves`. Its
callback type becomes `Callable[[int], U]` and its return type stays
`Tree[U]`. Keeping `U` lets ty specialize the return type to `Tree[str]`
if it later learns that the callback returns `str`. This is correct.
**Skipped slot.** Since #26680, ty uses both the arguments and the
expected return type to infer a call. In the reported example, that can
give `U` a binding that mentions itself.
```text
U = V | tuple[Tree[U], ...]
```
While resolving this binding, ty marks `U` as skipped. If
`as_specialization` keeps the inner `U` (as it does on main),
substituting this binding into another variable's solution copies the
same `U` one tuple level deeper on every pass.
We now call the existing `ApplySpecialization::get`, which already
distinguishes missing vs skipped, and supplies `Never` (instead of the
original type variable) for a skipped typevar. In the recursive
"skipped" case, `Never` works because the recursive occurrence of the
typevar doesn't add new evidence about the leaf type. This use of
`Never` pre-dates this PR.
Closesastral-sh/ty#4972.
## Test plan
Bisected the reporter's Python 3.12 example to #26680. Its parent
accepted the example in 0.04 seconds. The merge commit overflowed its
stack after 22 seconds. With this fix, the example passes with the cast
and reports `invalid-explicit-return` when the cast is removed.
## Summary
Closes astral-sh/ty#4961.
When a parameter accepts either `T` or a generic type containing `T`, ty
can infer extra layers of that generic type when it revisits an argument
expression whose type is context-sensitive. We do fixpoint iteration on
generic calls to allow contextual inference to propagate types from
sibling arguments, and this nested growth can prevent that iteration
from settling. An example:
```python
from typing import reveal_type
def as_list[T](value: T | list[T]) -> list[T]:
if isinstance(value, list):
return value
return [value]
def example(n: int, xs: list[int]) -> None:
# The type of `n` is always `int`.
reveal_type(as_list(n)) # list[int] (before and after)
# This case also already converges quickly because the type of `xs` is always `list[int]`.
# We might prefer to infer a type of `list[int]` in this case, but that would be a separate change:
reveal_type(as_list(xs)) # list[list[int] | int] (before and after)
# Both of these cases are context sensitive and we build repeatedly deeper nested types before this PR:
reveal_type(as_list([n])) # list[list[int] | int] with this change
reveal_type(as_list(as_list(n))) # list[list[int] | int] with this change
```
The arguments `xs`, `[n]`, and `as_list(n)` all start out with type
`list[int]`. Each can match the `T` arm with `T = list[int]` or the
`list[T]` arm with `T = int`. We then revisit the same argument
expression using the specialized parameter type as type context. This is
how one argument can help determine the type of another in a generic
call; this is particularly useful for e.g. typing a lambda in a call to
`map`, where the type of data passed to `map` can help us understand the
type of the untyped lambda.
On main, ty combines the two bindings into `int | list[int]`, so the
type context from the `list[T]` arm becomes `list[int | list[int]]`. For
the context-sensitive argument expressions (literal `[n]` or the inner
`as_list(n)`), that context can widen the inferred element type to `int
| list[int]`, which then feeds back into the outer call and can add yet
another list layer on each iteration.
With this PR, the hint from the `list[T]` arm is `list[int]`, so all
three argument expressions keep that type. Both bindings still
contribute to the result of the outer call, `list[list[int] | int]`.
The same feedback loop made the nested `factory.Maybe` calls in
astral-sh/ty#4961 slow to check.
I don't love this fix -- it's too narrowly targeted to unions and it can
miss some cases. But it seems like more general approaches will be quite
large/invasive, so I think this is the best fix for now to solve the
known problem case.
## Comparison with #30195#30195 detects when inference paths disagree on a type variable. If the
resulting parameter type already covers the previous argument type, it
reuses that whole argument type as context. This can discard information
from a sibling argument, including the parameter type needed to check a
lambda body.
This change computes a separate specialization for the nested union
members. For `T | list[T]`, ty tries matching the list arguments to
`list[T]` while retaining the constraints from the other arguments and
the expected return. It uses that hint only if one specialization works
across the call and preserves an already-compatible return. The nested
member must have a solution for its type variables, and the argument’s
previous type must be redundant with that specialized member. For
example, `list[int] | Unknown` is assignable to `Iterable[int]`, but
using that as the next hint can invalidate a nested call. The bare `T`
arm keeps its ordinary specialization.
The benchmark covers the issue's nested constructor call with no
expected type for the outer expression, including a variant where a type
alias hides the union members.
## Test plan
<details>
<summary>Dependency-free constructor reproducer</summary>
```python
class Declaration[K, V]:
def __init__(self, key: K | None = None, *choices: "V | Declaration[K, V]"): ...
class Attribute[V](Declaration[str, V]): ...
class SubFactory[V](Declaration[str, V]): ...
Declaration("name", Declaration("name", Attribute(), SubFactory()), None)
```
</details>
I checked this constructor example three times with each debug binary.
On `4c51cc646d`, all three runs hit a 10-second timeout. On this branch,
all three checks completed without diagnostics, with a median of 49 ms.
With the `types-factory-boy` 0.4.1 stubs, the reporter's full example
finished in 231–255 ms across three Linux ARM64 debug runs. The baseline
hit a 35-second timeout. The only diagnostics were the two `Meta`
override errors also present with the nested expression removed.
A descriptor that defines only `__set__` still controls writes to its
attribute. For example, ty could accept a subclass property whose setter
took only `str` when the inherited descriptor accepted `int`. Check that
an override accepts the inherited setter's values.
When a method decorator returns a setter-only descriptor on a protocol,
reading the member gives the descriptor itself and assigning to it
checks the setter's value type. An ordinary annotated attribute remains
writable when its type merely permits a descriptor.