Use `# ok` as the marker for mdtest statements whose expectation is that
no diagnostic is emitted. Update the guidance in `AGENTS.md` and replace
existing `# no diagnostic` comments in Ruff and ty mdtests.
Unpacking a tuple with repeated incompatible element types into a
variadic parameter can emit identical `invalid-argument-type`
diagnostics at the same starred argument. Report each identical mismatch
once, while retaining distinct errors for different source arguments,
parameters, or expected and provided types.
Deduplicate when emitting diagnostics so all parameter matches, type
checks, and failing-overload error counts remain intact.
## Ecosystem report
For head `ed5e760595e57b6f83c6bd66e9dbdc80d149fcad`, the ecosystem
comparison removes three duplicate `invalid-argument-type` diagnostics:
two in Prefect and one in SymPy. Each affected call retains one error,
and distinct diagnostics remain unchanged. Exact-run reproduction and
minimized examples confirm the expected behavior; all readiness checks
passed.
[Full ecosystem
report](https://github.com/astral-sh/ruff/pull/29337#issuecomment-6152510514).
Reject calls where every member of an unpacked tuple union supplies too
many arguments once later positional arguments are included:
```python
def f(a: int) -> None: ...
def g(args: tuple[int] | tuple[int, int]) -> None:
f(*args, 1) # too-many-positional-arguments
```
Combining unequal tuple shapes previously discarded their minimum
lengths, allowing argument matching to treat `args` as empty.
`TupleSpecBuilder::union` now preserves the smaller minimum length when
combining shapes, so `Type::iterate` retains that guarantee for all its
consumers. This also preserves required elements when unpacking tuple
unions into tuple literals or inferring a `TypeVarTuple`.
Addresses the [minimum-length review
comment](https://github.com/astral-sh/ruff/pull/29293#discussion_r4248949931)
on #29293.
## Ecosystem report
Analyzed head:
[`1b5eb5424c`](https://github.com/astral-sh/ruff/commit/1b5eb5424c7fad6edf6994d73a36691f358a97e2).
[Full
report](https://github.com/astral-sh/ruff/pull/29331#issuecomment-6150237369).
The five changes are understood and acceptable for this fix: two more
precise Bokeh messages, two rewrites of existing Meson cyclic-inference
warnings, and one duplicate SymPy argument error. No newly rejected
calls were found. The duplicate reflects pre-existing reporting behavior
for nonempty tuples and can be addressed independently.
## Summary
This PR fixes a bug where unpacking something like `*args` would eat up
all the remaining positional parameters. Because of this, if you passed
another explicit argument *after* the unpack (like `f(*args, x)`), the
type checker thought there was no space left for `x`. This caused false
positive `too-many-positional-arguments` or `invalid-argument-type`
errors.
To fix this, we updated the logic in `bind.rs`. Now, if a function
doesn't have a variadic parameter, we count how many explicit positional
arguments come after the unpack and reserve room for them so everything
fits properly.
Part of: astral-sh/ty#3649.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
This PR prepares our mdtests for return type inference. For example, we
had some definitions like
```py
def flag(): ...
```
which were supposed to be used in `if flag()` tests of ambiguous
truthiness. However, with return type inference, ty would understand
that this function returns `None`, which is falsy.
I'm pulling this out so as to reduce the diff on the upcoming return
type inference PR.
## Test Plan
Tested by running the return type inference implementation against these
changes.
Calls such as `pair(*[1, "two"])` and `pair(**{"x": 1, "y": "two"})`
lose their exact argument count and individual types during call
analysis.
Retain the contents of immediate list and tuple literals, and
dictionaries with explicit string keys. Use that information for
argument matching, overload selection, `TypeVarTuple` inference, and
signature help. Preserve it through `ParamSpec` forwarding, including
dictionaries that also supply wrapper parameters.
Adapted from [Tamir Duberstein’s original
change](https://github.com/tamird/ruff/commit/26e5603efa8e650108f332ac4ee84562c265c38f).
Related to
[astral-sh/ty#3792](https://github.com/astral-sh/ty/issues/3792), but
doesn't fully fix it. The current PR only works for literals constructed
directly in the call expression, but for example:
```python
params = {"sep": "foo", "end": "baz"}
print(*range(5), **params)
```
isn't recognized by this implementation (as the type of `params` is an
open `dict[str, str]`).
---------
Co-authored-by: Tamir Duberstein <tamird@gmail.com>
## Summary
Disable `invalid-legacy-positional-parameter` by default. Modern code
can use double-underscore-prefixed parameter names to avoid collisions
with `**kwargs`, without intending to use the legacy positional-only
convention.
The mdtest harness continues to enable the rule globally, and the
conformance runner already enables it with
`--error=invalid-legacy-positional-parameter`. Update the rule
documentation to explain the default, and regenerate the docs and
schema.
Follow-up to
https://github.com/astral-sh/ruff/pull/25079#issuecomment-5797494065.
## Summary
This is step 3 in rebooting #26712 as a series of focused PRs
(astral-sh/ty#3557).
`SpecializationBuilder` currently mixes two operations:
1. Collecting the constraints for a generic call.
2. Copying their solutions into the legacy type-variable mapping.
Copying solutions too early loses relationships between alternatives.
This PR separates the work into three explicit steps:
1. Analyze a constraint set and collect any useful diagnostic
information.
2. Record the constraint set for the whole call.
3. Update the legacy mapping only when a caller needs it.
This lets overloaded callbacks examine every alternative before accepted
alternatives update the mapping. Rejected overloads cannot affect valid
specializations, and the existing first-rejection diagnostic behavior is
preserved.
Existing fallbacks for contextual preferences, `ParamSpec`,
`TypeVarTuple`, and recursive specialization remain in place.
## Test plan
Added mdtests cover:
- Generic overloaded callbacks passed to constructors.
- Bounded and constrained type variables when an invalid callback
overload appears first or last.
- A `ParamSpec`-forwarding callable passed through an unpacked
`TypeVarTuple`.
Rust unit tests also cover independent analysis, recording, and
projection; valid and rejected inference paths; lower-bound
classification; and grouped declaration failures.
## Summary
Part of astral-sh/ty#3557. This is the first focused step toward
rebooting #26712 as a stack of smaller solver changes.
Introduce `ArgumentRelation` as the shared representation of a matched
argument/parameter pair and use it for generic specialization inference,
unsatisfiable-constraint diagnostic recovery, and ordinary,
positional-unpack, and keyword-unpack argument checking. Preserve
effective per-element formal types, each checking path's existing
actual-type lookup, synthetic receiver/source argument indices, gradual
variadic filtering, and existing `TypeVarTuple` handling without
changing solver behavior.
## Test plan
Added mdtests covering generic bound-method diagnostics after an
implicit receiver, positional and keyword source locations, homogeneous
and heterogeneous unpacked generic parameters, unpacked positional
element types, per-key `TypedDict` generic inference, and gradual
variadic arguments that must not affect inference.
The constraint solver has a fast-path that bypasses the full solver when
all type variables are bounded by concrete types. This was necessary to
fix exponential blowup in the sequent map with a large number of type
variables with no transitive relationships.
https://github.com/astral-sh/ty/issues/3989. However, introducing a
single relationship between type variables bypasses the fast path,
making it possible to observe similar exponential blowup, as seen in
https://github.com/astral-sh/ruff/pull/26680. This PR adds a
short-circuit in the full solver to avoid sequent discovery for the
subset of typevars that have concrete bounds.
Note that I attempted to remove the fast-path entirely now that we have
a more general fix, but it reintroduced a small regression due to the
general extra work that the full solver performs. A followup may be able
to close the gap entirely. There are some differences in behavior and
diagnostics between the two paths that should eventually be addressed as
well.
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
Previously, we treated unpacked `*args` annotations as ordinary variadic
parameters during call binding, allowing missing tuple elements, excess
positional arguments, and incompatible element types to pass unchecked:
```python
def callback(*args: *tuple[int, str]) -> None: ...
callback() # error: [missing-argument]
callback(1, 2) # error: [invalid-argument-type]
callback(1, "ok", 3) # error: [too-many-positional-arguments]
```
We now align positional arguments with the unpacked tuple's fixed
prefix, homogeneous variadic segment, and fixed suffix, preserve each
element's expected type through ordinary call checking, and enforce the
tuple's minimum and maximum arity.
The same path validates callable protocols and specializes bounded type
variables, including ordinary type variables beside an unresolved
`TypeVarTuple`.
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
## Summary
PEP 484's numeric tower means we represent a `float` annotation as the
semantic union `int | float*` and `complex` as `int | float* |
complex*`. We previously rendered those internal unions directly, which
exposed an implementation detail in user-facing output:
```python
def f() -> float:
return 1
reveal_type(f()) # Before: int | float; after: float
```
This change recognizes complete numeric-tower unions during display and
renders them using their canonical annotation spelling, `float` or
`complex`.
The implementation does _not_ preserve explicit unions, e.g., if the
user writes out `int | float`, we still collapse that rather than trying
to preserve it, mostly for simplicity (as per
https://github.com/astral-sh/ty/issues/2184#issuecomment-4345219164).
Closes https://github.com/astral-sh/ty/issues/2184.
## Summary
This is a first step towards #20411. After this, we would work to either
upstream changes or adjust things within Ruff.
Performance improvements have already been upstreamed
(https://github.com/rust-lang/annotate-snippets-rs/pull/437,
https://github.com/rust-lang/annotate-snippets-rs/pull/440)
Re-implemented the following from the fork:
- Hidden snippets
(https://github.com/rust-lang/annotate-snippets-rs/issues/448)
- Renderer-level disabling of hyperlinks
- Jupyter cell indices
(https://github.com/rust-lang/annotate-snippets-rs/issues/449)
- Fixable mark
(https://github.com/rust-lang/annotate-snippets-rs/issues/450)
- `lineno_offset` (maybe not needed if #20648 is taken care of)
- anonymized lines affects origin
(https://github.com/rust-lang/annotate-snippets-rs/issues/451)
- cut indicator (maybe not needed if we can switch to unicode renderer)
Additional hacks to minimize behavior change include
- Adjust upstream's hidden severity behavior to match Ruff's
(https://github.com/rust-lang/annotate-snippets-rs/issues/452)
- How newlines are annotated
(https://github.com/rust-lang/annotate-snippets-rs/issues/454)
- To not indent for line numbers when none will be present
(https://github.com/rust-lang/annotate-snippets-rs/issues/453)
- Enabling `simd` by default
All of the above represents work that needs to be resolved to address
#20411.
`lib.name` hack is used to simplify maintenance with the assumption that
we'll continue working towards #20411.
Behavior changes that were included
- Tweak when implicit padding lines are present
- Fix Origin's line/column is now derived from the primary annotation
- Make consistent the rendering of primary/context annotations
- Fix snippets to show up for empty source (but may degrade #24528)
- Fix handling of term width
- Fix rendering of annotations between newline and first character
(granted, maybe the caller is wrong)
- Tweak multiline annotation markers
New or upcoming features annotate-snippets features that we will be able
to leverage by moving this forward
- `AnnotationKind::Visible` to improve the quality of built-in
code-folding if Ruff were to switch to it
- `DecorStyle::Unicode`
- *(upcoming)* No-graphiscs mode for screen readers (may also be useful
for AIs)
Deferred:
- Moving related `Group`s into the same `Report`
- Using `secondary_title` for any `Group` after the first in a `Report`
- Using a `Snippet`, instead of titles, for "virtual code" like
https://github.com/astral-sh/ruff/pull/27033#discussion_r3622727236
- Switching hidden snippets to `Origin`
## Test Plan
Within cpython, I ran
```console
$ hyperfine -i "../../ruff/target-baseline/release/ruff check" "../../ruff/target-update/release/ruff check" -N -m20 --output=null
Benchmark 1: ../../ruff/target-baseline/release/ruff check
Time (mean ± σ): 27.5 ms ± 1.1 ms [User: 33.0 ms, System: 108.4 ms]
Range (min … max): 25.0 ms … 31.7 ms 94 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: ../../ruff/target-update/release/ruff check
Time (mean ± σ): 28.1 ms ± 2.0 ms [User: 33.4 ms, System: 105.7 ms]
Range (min … max): 24.8 ms … 38.9 ms 102 runs
Warning: Ignoring non-zero exit code.
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use t
he '--warmup' or '--prepare' options.
Summary
../../ruff/target-baseline/release/ruff check ran
1.03 ± 0.08 times faster than ../../ruff/target-update/release/ruff check
```
Before:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/2d94dabf-b28d-48c1-b17c-52aac59fd57e"
/>
After:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/b388ada3-e3d7-4ba0-b90b-1330eb81f4be"
/>
## Summary
Split `ty_extensions` into a public part (`ty_extensions`), and an
internal part (`ty_extensions._internal`). For now, move everything to
`_internal`, except for:
- `static_assert`
- `Unknown`
- `AlwaysTruthy`, `AlwaysFalsy`
- `Intersection`, `Not`
- `Top`, `Bottom`
- `JustFloat`, `JustComplex`
- `NamedTupleLike`
## Summary
An mdtest section is evaluated as a single Python module, even when its
examples are split across multiple fenced code blocks. Some existing
sections therefore define the same legacy `TypeVar` name more than once:
```python
T = TypeVar("T")
# ...
T = TypeVar("T")
```
#25854 will correctly diagnose the second definition, but these
duplicates are incidental to the tests. This separates self-contained
examples into distinct mdtest sections and gives the remaining type
variables distinct names where the examples depend on shared setup. The
expected types and diagnostics are unchanged.
This is the preparatory base for #25854 and doesn't change behavior.
## Summary
Add a new lint that detects generic types used without explicit type
parameters in annotations (e.g. `list` instead of `list[int]`).
This is equivalent to mypy's `--disallow-any-generics` and pyright's
`reportMissingTypeArgument`.
Can be enabled via:
```
[rules]
missing-type-argument = "warn"
```
It skips classes where all type parameters have PEP 696 defaults.
This should fix the following issue in ty:
- https://github.com/astral-sh/ty/issues/2442
## Test Plan
Added tests covering stdlib generics, user-defined generics, PEP 696
defaults, dotted names, function signatures, base classes, and variable annotations.
---------
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
## Summary
Open `TypedDict` values can contain hidden extra items with value type
`object`. This updates call binding to account for those possible values
when a `**TypedDict` argument is matched against a keyword-variadic
parameter, while continuing to ignore hidden items for callees without
`**kwargs`.
This means calls like the following now reject the possible hidden extra
items instead of only checking the declared keys:
```py
class Options(TypedDict):
name: str
def accepts_ints(name: str, **kwargs: int) -> None: ...
accepts_ints(**Options(name="x")) # Expected `int`, found `object`
```
The same implicit-open behavior also means `functools.partial(...,
**typed_dict)` cannot be normalized to a precise reduced signature,
because hidden extra items may bind additional keywords at runtime. We
now fall back to the broad `partial[ReturnType]` form for those calls,
matching Pyright for ordinary open `TypedDict`s.
## Summary
Catch infinite recursive expansion in constraint-set solving.
Closes https://github.com/astral-sh/ty/issues/3578
## Test Plan
Added test that runs forever before this fix.
This PR implements the bulk of the logic needed for
https://github.com/astral-sh/ty/issues/2799, by building up a
`ConstraintSet` in `SpecializationBuilder`. This constraint set holds
the "pending state" of the specialization being inferred. It will
eventually _replace_ the `types` hash map, which stores a simple mapping
from typevars to inferred types, and which is always updated by unioning
together types when we find multiple solutions for a typevar.
For now, we are building up both pending state representations, and
solving from the constraint set where we can. There are two main cases
where we must fall back on the old solver:
- Any specialization that involves a `ParamSpec`
- If the constraint set returns `Unsatisfiable` or `Unconstrained`
Follow-on PRs will address these remaining cases that use the old
solver; once they are all migrated over to the constraint set solver, we
will remove the `types` field.
Closesastral-sh/ty#2572Closesastral-sh/ty#2959Closesastral-sh/ty#3037Closesastral-sh/ty#3203Closesastral-sh/ty#3228Closesastral-sh/ty#3428Closesastral-sh/ty#3483
Remember when we wrote `x = 1 if flag else None` because we couldn't
infer types from annotated parameters and/or couldn't understand union
type annotations? This PR modernizes our mdtests to simply use annotated
function parameters in those cases (I manually rejected some rewrites
when the new style was less clear, for example when the generated union
type itself was hard to spell).
The motivation for this is to set a better precedent for agents, as I've
seen them repeating that `x if flag else y` pattern, which is really
unnecessary, longer and harder to read in most cases.
## Summary
Given:
```python
def accepts_anything(first: int, *args: Any, **kwargs: Any) -> None: ...
```
Calling this method with a large number of arguments, like:
```python
accepts_anything(0, ("field_0", 0), ..., ("field_1023", 1023))
```
...would lead all 1,024 positional arguments to bind to `*args`. On
main, that means we check each one against `Any`, and do a bunch of
other bookkeeping, only to throw it away. This PR short-circuits most of
the argument checking in these cases (while still validating other
positional arguments and the `**` key types).
## Summary
The guard condition for per-element union iteration in match_variadic
previously required all remaining positional parameters to be defaulted.
This was overly conservative. We only need parameters beyond the
shortest union element's length to be defaulted, since those positions
are guaranteed to be provided by every union element.
See: https://github.com/astral-sh/ruff/pull/23124/changes#r2806496807.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Closesastral-sh/ty#2653
Move the KeywordsNotAMapping check from bind.rs to builder.rs, matching
the existing pattern for starred argument iterability. This ensures
users see "must be a mapping type" instead of "no matching overload"
when passing a non-mapping **kwargs to an overloaded function.
## Test Plan
Added test case for **kwargs with non-mapping types on overloaded
functions in function.md.
## Summary
fixes: https://github.com/astral-sh/ty/issues/1584
This PR fixes a bug where ty would unpack a variadic argument consuming
all the remaining unmatched parameters. The way this fix is implemented
is by collecting the parameters for which an argument is provided via
keyword e.g., `func(foo=1)`. For example,
```py
def foo(x: str, y: int): ...
def _(args: list[str]):
foo(*args, y=1)
```
In the above example, `main` would currently raise
`parameter-already-assigned` because we don't know how many elements
would `args` unpack into so we consider assigning the type to all the
remaining parameters. But, in the above case `y` has been provided as
keyword argument explicitly.
Now, there are other cases like `foo(*args, 1)` or `foo(*args, *(1,))`
which is not considered in this PR i.e., this PR only handles explicit
keyword argument and we would still emit diagnostic for positional,
variadic, and keyword-variadic. Pyright behaves in the same way i.e., it
only handles keyword argument while other type checkers (mypy, pyrefly)
does not handle any cases. Refer to the new mdtest cases for all the
cases.
## Test Plan
Add new mdtests.
## Summary
Closes: https://github.com/astral-sh/ty/issues/551
This PR adds support for step 4 of the overload call evaluation
algorithm which states that:
> If the argument list is compatible with two or more overloads,
determine whether one or more of the overloads has a variadic parameter
(either `*args` or `**kwargs`) that maps to a corresponding argument
that supplies an indeterminate number of positional or keyword
arguments. If so, eliminate overloads that do not have a variadic
parameter.
And, with that, the overload call evaluation algorithm has been
implemented completely end to end as stated in the typing spec.
## Test Plan
Expand the overload call test suite.
## Summary
Closes: https://github.com/astral-sh/ty/issues/1236
This PR fixes a bug where the variadic argument wouldn't match against
the variadic parameter in certain scenarios.
This was happening because I didn't realize that the `all_elements`
iterator wouldn't keep on returning the variable element (which is
correct, I just didn't realize it back then).
I don't think we can use the `resize` method here because we don't know
how many parameters this variadic argument is matching against as this
is where the actual parameter matching occurs.
## Test Plan
Expand test cases to consider a few more combinations of arguments and
parameters which are variadic.
## Summary
This PR adds support for unpacking `**kwargs` argument.
This can be matched against any standard (positional or keyword),
keyword-only, or keyword variadic parameter that haven't been matched
yet.
This PR also takes care of special casing `TypedDict` because the key
names and the corresponding value type is known, so we can be more
precise in our matching and type checking step. In the future, this
special casing would be extended to include `ParamSpec` as well.
Part of astral-sh/ty#247
## Test Plan
Add test cases for various scenarios.
## Summary
This PR addresses an issue for a variadic argument when involved in
argument type expansion of overload call evaluation.
The issue is that the expansion of the variadic argument could result in
argument list of different arity. For example, in `*args: tuple[int] |
tuple[int, str]`, the expansion would lead to the variadic argument
being unpacked into 1 and 2 element respectively. This means that the
parameter matching that was performed initially isn't sufficient and
each expanded argument list would need to redo the parameter matching
again.
This is currently done by redoing the parameter matching directly,
maintaining the state of argument forms (and the conflicting forms), and
updating the `Bindings` values if it changes.
Closes: astral-sh/ty#735
## Test Plan
Update existing mdtest.
This PR updates our iterator protocol machinery to return a tuple spec
describing the elements that are returned, instead of a type. That
allows us to track heterogeneous iterators more precisely, and
consolidates the logic in unpacking and splatting, which are the two
places where we can take advantage of that more precise information.
(Other iterator consumers, like `for` loops, have to collapse the
iterated elements down to a single type regardless, and we provide a new
helper method on `TupleSpec` to perform that summarization.)
This PR updates our call binding logic to handle splatted arguments.
Complicating matters is that we have separated call bind analysis into
two phases: parameter matching and type checking. Parameter matching
looks at the arity of the function signature and call site, and assigns
arguments to parameters. Importantly, we don't yet know the type of each
argument! This is needed so that we can decide whether to infer the type
of each argument as a type form or value form, depending on the
requirements of the parameter that the argument was matched to.
This is an issue when splatting an argument, since we need to know how
many elements the splatted argument contains to know how many positional
parameters to match it against. And to know how many elements the
splatted argument has, we need to know its type.
To get around this, we now make the assumption that splatted arguments
can only be used with value-form parameters. (If you end up splatting an
argument into a type-form parameter, we will silently pass in its
value-form type instead.) That allows us to preemptively infer the
(value-form) type of any splatted argument, so that we have its arity
available during parameter matching. We defer inference of non-splatted
arguments until after parameter matching has finished, as before.
We reuse a lot of the new tuple machinery to make this happen — in
particular resizing the tuple spec representing the number of arguments
passed in with the tuple length representing the number of parameters
the splat was matched with.
This work also shows that we might need to change how we are performing
argument expansion during overload resolution. At the moment, when we
expand parameters, we assume that each argument will still be matched to
the same parameters as before, and only retry the type-checking phase.
With splatted arguments, this is no longer the case, since the inferred
arity of each union element might be different than the arity of the
union as a whole, which can affect how many parameters the splatted
argument is matched to. See the regression test case in
`mdtest/call/function.md` for more details.
## Summary
Having a recursive type method to check whether a type is fully static
is inefficient, unnecessary, and makes us overly strict about subtyping
relations.
It's inefficient because we end up re-walking the same types many times
to check for fully-static-ness.
It's unnecessary because we can check relations involving the dynamic
type appropriately, depending whether the relation is subtyping or
assignability.
We use the subtyping relation to simplify unions and intersections. We
can usefully consider that `S <: T` for gradual types also, as long as
it remains true that `S | T` is equivalent to `T` and `S & T` is
equivalent to `S`.
One conservative definition (implemented here) that satisfies this
requirement is that we consider `S <: T` if, for every possible pair of
materializations `S'` and `T'`, `S' <: T'`. Or put differently the top
materialization of `S` (`S+` -- the union of all possible
materializations of `S`) is a subtype of the bottom materialization of
`T` (`T-` -- the intersection of all possible materializations of `T`).
In the most basic cases we can usefully say that `Any <: object` and
that `Never <: Any`, and we can handle more complex cases inductively
from there.
This definition of subtyping for gradual subtypes is not reflexive
(`Any` is not a subtype of `Any`).
As a corollary, we also remove `is_gradual_equivalent_to` --
`is_equivalent_to` now has the meaning that `is_gradual_equivalent_to`
used to have. If necessary, we could restore an
`is_fully_static_equivalent_to` or similar (which would not do an
`is_fully_static` pre-check of the types, but would instead pass a
relation-kind enum down through a recursive equivalence check, similar
to `has_relation_to`), but so far this doesn't appear to be necessary.
Credit to @JelleZijlstra for the observation that `is_fully_static` is
unnecessary and overly restrictive on subtyping.
There is another possible definition of gradual subtyping: instead of
requiring that `S+ <: T-`, we could instead require that `S+ <: T+` and
`S- <: T-`. In other words, instead of requiring all materializations of
`S` to be a subtype of every materialization of `T`, we just require
that every materialization of `S` be a subtype of _some_ materialization
of `T`, and that every materialization of `T` be a supertype of some
materialization of `S`. This definition also preserves the core
invariant that `S <: T` implies that `S | T = T` and `S & T = S`, and it
restores reflexivity: under this definition, `Any` is a subtype of
`Any`, and for any equivalent types `S` and `T`, `S <: T` and `T <: S`.
But unfortunately, this definition breaks transitivity of subtyping,
because nominal subclasses in Python use assignability ("consistent
subtyping") to define acceptable overrides. This means that we may have
a class `A` with `def method(self) -> Any` and a subtype `B(A)` with
`def method(self) -> int`, since `int` is assignable to `Any`. This
means that if we have a protocol `P` with `def method(self) -> Any`, we
would have `B <: A` (from nominal subtyping) and `A <: P` (`Any` is a
subtype of `Any`), but not `B <: P` (`int` is not a subtype of `Any`).
Breaking transitivity of subtyping is not tenable, so we don't use this
definition of subtyping.
## Test Plan
Existing tests (modified in some cases to account for updated
semantics.)
Stable property tests pass at a million iterations:
`QUICKCHECK_TESTS=1000000 cargo test -p ty_python_semantic -- --ignored
types::property_tests::stable`
### Changes to property test type generation
Since we no longer have a method of categorizing built types as
fully-static or not-fully-static, I had to add a previously-discussed
feature to the property tests so that some tests can build types that
are known by construction to be fully static, because there are still
properties that only apply to fully-static types (for example,
reflexiveness of subtyping.)
## Changes to handling of `*args, **kwargs` signatures
This PR "discovered" that, once we allow non-fully-static types to
participate in subtyping under the above definitions, `(*args: Any,
**kwargs: Any) -> Any` is now a subtype of `() -> object`. This is true,
if we take a literal interpretation of the former signature: all
materializations of the parameters `*args: Any, **kwargs: Any` can
accept zero arguments, making the former signature a subtype of the
latter. But the spec actually says that `*args: Any, **kwargs: Any`
should be interpreted as equivalent to `...`, and that makes a
difference here: `(...) -> Any` is not a subtype of `() -> object`,
because (unlike a literal reading of `(*args: Any, **kwargs: Any)`),
`...` can materialize to _any_ signature, including a signature with
required positional arguments.
This matters for this PR because it makes the "any two types are both
assignable to their union" property test fail if we don't implement the
equivalence to `...`. Because `FunctionType.__call__` has the signature
`(*args: Any, **kwargs: Any) -> Any`, and if we take that at face value
it's a subtype of `() -> object`, making `FunctionType` a subtype of `()
-> object)` -- but then a function with a required argument is also a
subtype of `FunctionType`, but not a subtype of `() -> object`. So I
went ahead and implemented the equivalence to `...` in this PR.
## Ecosystem analysis
* Most of the ecosystem report are cases of improved union/intersection
simplification. For example, we can now simplify a union like `bool |
(bool & Unknown) | Unknown` to simply `bool | Unknown`, because we can
now observe that every possible materialization of `bool & Unknown` is
still a subtype of `bool` (whereas before we would set aside `bool &
Unknown` as a not-fully-static type.) This is clearly an improvement.
* The `possibly-unresolved-reference` errors in sockeye, pymongo,
ignite, scrapy and others are true positives for conditional imports
that were formerly silenced by bogus conflicting-declarations (which we
currently don't issue a diagnostic for), because we considered two
different declarations of `Unknown` to be conflicting (we used
`is_equivalent_to` not `is_gradual_equivalent_to`). In this PR that
distinction disappears and all equivalence is gradual, so a declaration
of `Unknown` no longer conflicts with a declaration of `Unknown`, which
then results in us surfacing the possibly-unbound error.
* We will now issue "redundant cast" for casting from a typevar with a
gradual bound to the same typevar (the hydra-zen diagnostic). This seems
like an improvement.
* The new diagnostics in bandersnatch are interesting. For some reason
primer in CI seems to be checking bandersnatch on Python 3.10 (not yet
sure why; this doesn't happen when I run it locally). But bandersnatch
uses `enum.StrEnum`, which doesn't exist on 3.10. That makes the `class
SimpleDigest(StrEnum)` a class that inherits from `Unknown` (and
bypasses our current TODO handling for accessing attributes on enum
classes, since we don't recognize it as an enum class at all). This PR
improves our understanding of assignability to classes that inherit from
`Any` / `Unknown`, and we now recognize that a string literal is not
assignable to a class inheriting `Any` or `Unknown`.