Commit Graph
48 Commits
Author SHA1 Message Date
Alex WaygoodandMicha Reiser 22e77392fb [ty] Split invalid-return-type into 3 focussed rules; rename empty-body and unsound-return (#29947)
Co-authored-by: Micha Reiser <micha@reiser.io>
2026-11-11 16:59:57 +00:00
David PeterandAlex Waygood 10775b0d4c [ty] Rename incompatible override rules (#30142)
Rename the method, attribute, property, and mutable override rules to
`incompatible-…-override`. The old names remain aliases in configuration
and inline suppressions:

| Before | After |
|---|---|
| `invalid-method-override` | `incompatible-method-override` |
| `invalid-attribute-override` | `incompatible-attribute-override` |
| `invalid-property-type-override` | `incompatible-property-override` |
| `invalid-mutable-override` | `incompatible-mutable-override` |

A follow-up PR will split `invalid-method-override` into two rules.

Fixes astral-sh/ty#2001.

---------

Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2026-11-09 12:14:13 +01:00
Alex WaygoodandBrent Westbrook ce0077c3e5 [ty] Distinguish attributes missing from some union members (#29944)
## Summary

Users can now suppress an attribute error that affects only some union
members and still catch names that are missing from every member. For
example, both lines below previously reported `unresolved-attribute`:

```python
def clean(value: str | int) -> str:
    value.stirp()          # missing-attribute: neither str nor int has stirp
    return value.strip()   # missing-union-attribute: str has strip, int does not
```

The motivation is that a project adopting ty may want `value.strip()` on
`str | None` to only warn while a typo remains an error-severity
diagnostic.

Both rules default to errors. The same distinction applies to optional
values such as `str | None` and to attribute deletion. Attributes
defined only conditionally on a class continue to use
`possibly-missing-attribute`.

### Comparison to other type checkers

Pyright distinguishes between `reportOptionalMemberAccess` (which is
specifically for unions with `None`) and `reportAttributeAccessIssue`,
which is for all other unions. mypy, meanwhile, distinguishes between
`union-attr` (which is an error code for all attribute-access errors on
union types) and `attr-defined` (which is an error code for all
attribute-access errors on non-union types). So this new split doesn't
_exactly_ align with either precedent -- but I _do_ think it's probably
the most useful split we can do, and it's the one that makes the most
sense _a priori_ in my opinion.

### Compatibility

`unresolved-attribute` remains an alias for `missing-attribute` in
configuration and inline ignores. To preserve a setting such as
`unresolved-attribute = "warn"` for both cases, also set
`missing-union-attribute = "warn"`. On an access that fails for only
some union members, replace `# ty: ignore[unresolved-attribute]` with `#
ty: ignore[missing-union-attribute]`. Tools that consume diagnostic
codes should recognize both new names.

Fixes astral-sh/ty#2839 and fixes astral-sh/ty#2471.

## Test Plan

A comparison with ty 0.0.96 on typeshed-stats, docstring-adder,
sphinx-lint, and typing_extensions found the same messages, locations,
and severities. Eight diagnostics received `missing-union-attribute`
across those four projects (two in sphinx-lint and six in
typing_extensions).

---------

Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2026-11-06 14:46:50 -05:00
Douglas Creager b4d38343dd [ty] Prioritize finite members in protocol implementation checks (#29542)
This patch has us compare members during protocol-to-protocol
comparisons using the same "structural priority" that we use during
nominal-to-protocol comparisons. This ensures that we check simple
non-recursive members first, often allowing us to determine a (failed)
comparison result before having to perform any expensive recursive
member checks.
2026-10-23 15:08:30 -04:00
Charlie Marsh 3ac6b4c312 [ty] Represent uninhabited truthiness explicitly (#28843)
## Summary

We now mark both branches of a condition with type `Never` as
unreachable:

```python
from typing import Never

def f(value: Never):
    if value:
        result = 1  # Unreachable.
    else:
        result = 2  # Also unreachable.
```

Previously, we treated `Never` as having ambiguous truthiness, allowing
either assignment to affect inference. In the reported loop, the
variables depend on each other, so including these impossible
assignments made their inferred types depend on which file we checked
first.

We add `Truthiness::Uninhabited` to represent the absence of either
boolean outcome. `Type::bool` and `Type::try_bool` return this variant
for `Never`, replacing the opt-in `bool_if_inhabited` API and making
callers handle the distinction explicitly.

Closes https://github.com/astral-sh/ty/issues/4577.
2026-10-08 20:46:56 -04:00
Shunsuke ShibayamaandCarl Meyer 160ffbf2cf [ty] Introduce Recursive types and fully support recursive implicit type aliases (#28425)
## Summary

Closes https://github.com/astral-sh/ty/issues/1738
(While this PR also resolves the error in the code provided in
https://github.com/astral-sh/ty/issues/1846, I consider the broader
issue under discussion to still be unresolved)

To fully support implicit type aliases, this PR introduces `Recursive`
types. These represent *structural recursive types*. In other words,
they are anonymous recursive types implicitly constructed through type
inference, rather than a nominally defined recursive type like PEP 695
recursive type aliases.

This is what was being discussed in
https://github.com/astral-sh/ruff/pull/24252#issuecomment-4172655350. We
are not introducing recursive types solely for implicit type aliases;
this is the first step toward completely replacing the current
`Divergent`-based approximate recursive type inference with a full
implementation.

Initially, I planned to completely remove the `Divergent` type alongside
the introduction of `Recursive` types, but because the scope of the
impact was far too large, I decided to replace it incrementally. Both
will coexist until the replacement is complete, but eventually,
everything should be replaced by `Recursive` types.

Following this PR, I plan to use the newly introduced `Recursive` types
to replace various forms of recursive type inference.

## Design

Currently, we approximate and treat recursive type structures generated
by recursive type inference as gradual types containing `Divergent`. For
example, a subscript on `x: Divergent` results in `x[n]: Divergent`.
Consequently, we get:

```
x: list[Divergent]
x[n]: Divergent
x[n][n]: Divergent
```

As a result, the type after subscripting loses the information that `x`
was a recursive list type. We solve this problem by introducing
recursive types.

`RecursiveType` is a binder for a body type that contains a recursive
variable. In other words, it roughly has the following structure:

```rust
struct RecursiveType<'db> {
    id: salsa::Id,
	body: Type<'db>,
	arguments: Option<Specialization<'db>>,
}

struct RecursiveVar<'db> {
    id: salsa::Id,
    arguments: Option<Specialization<'db>>,
}

enum Type<'db> {
    Recursive(RecursiveType<'db>),
    RecursiveVar(RecursiveVar<'db>),
    ...
}
```

For example, for the following implicit type alias:

```python
Tree = tuple[T, "Tree[list[T]] | None"]
# = μF. λT. tuple[T, F[list[T]] | None]
```

the constructed `RecursiveType` looks like this:

```rust
RecursiveType {
    id,
    body: tuple[T, RecursiveVar { id, arguments: [list[T]] } | None],
    arguments: [T]
}
```

Unlike `Divergent`, the recursive variable `RecursiveVar` has no type
properties on its own. Attempting to access this variable immediately
unfolds the recursive type, so questioning the typing properties of the
variable itself is meaningless. Therefore, if any type operation on
`RecursiveVar` appears, it should be a bug.

To infer the container subscript `x[1]` for `x: Tree[int]`, first unfold
`x`'s type. Unfolding replaces references to the recursive binder with
the recursive type itself. Writing `B[a := R]` for capture-avoiding
substitution of `R` for `a` in `B`:

```text
unfold(μa. B) = B[a := μa. B]
```

For `Tree[int]`, substitute the constructor for `F`, then apply `T :=
int`:

```text
unfold((μF. λT. tuple[T, F[list[T]] | None])[int])
= tuple[int, (μF. λT. tuple[T, F[list[T]] | None])[list[int]] | None]
(= tuple[int, Tree[list[int]] | None])
```

Tuple subscripting then selects the element at index 1: `x[1]:
Tree[list[int]] | None`.

## Test Plan

mdtest updated

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2026-09-18 04:01:49 +09:00
Alex Waygood 689a33fd36 [ty] Add rules to detect always-truthy and always-falsy conditions (#28034)
## Summary

Builds on #28032.

Add two rules for boolean tests whose truthiness is statically known:

- `redundant-condition`, enabled by default at warning severity, catches
cases such as an uncalled function or method, an unawaited coroutine, a
generator tested for emptiness, or a tuple whose length makes its
truthiness constant.
- `redundant-condition-strict`, disabled by default, covers
boolean/integer conditions, conditions whose outcome is fixed only by
short-circuit evaluation, and conditions containing walrus expressions.
These can expose real bugs, but are also much more likely to be
intentional.

The checks reuse ty's truthiness inference and apply to `if`/`elif`,
`while`, assertions, match guards, conditional expressions,
comprehension filters, and unary `not`. The operands of `and`/`or` are
checked when the expression is used as a condition. The main design
question is which statically redundant tests are likely to represent
mistakes. The implementation deliberately prefers missing some redundant
conditions to warning about common defensive or compatibility patterns.

## Avoiding false positives

### Separate likely mistakes from more opinionated checks

Types assignable to `int`, including `bool`, use the strict rule. So do
conditions whose outcome is fixed by short-circuit evaluation even
though their inferred value type has ambiguous truthiness, and tests
containing walrus expressions. Other values with fixed truthiness use
the default rule.

This keeps checks of ordinary boolean/integer flags quiet by default,
because these are extremely common and it is difficult to distinguish
deliberately constant flags from mistakes:

```python
DEBUG = False

if DEBUG:  # Only reported by redundant-condition-strict.
    print("debugging")
```

The strict rule is a sibling of the default rule: it handles a separate
set of expressions rather than reporting the same expressions more
aggressively.

### Distinguish a condition's outcome from its value type

Short-circuit evaluation can establish a fixed outcome even when the
expression's value type does not:

```python
def check(value: object):
    if value and False:  # Always false; reported by redundant-condition-strict.
        print("unreachable")
```

The condition fails whether `value` is truthy or falsy. However, saving
`value and False` to a variable and testing that variable later is
different: the saved value could be the original object, whose
`__bool__` result may change between calls. The redundant-condition
checks use short-circuit truthiness APIs introduced in
https://github.com/astral-sh/ruff/pull/28082 for direct conditions, and
value-type truthiness for tests of computed values. This also handles
chained comparisons whose intermediate results are objects with mutable
truthiness.

### Recognize explicitly constant tests

Both rules exempt direct boolean, integer and `None` literals in the
AST, such as `if False:`, `if 0:`, `while True:`, `while 1:`, and
`assert None`. These spellings are strong evidence that the constant
truthiness is deliberate. A name _inferred_ as `Literal[False]` is
different: its value may not be apparent where it is tested, so it
remains eligible for the strict rule.

### Treat walrus expressions as evidence of intentional side effects

An always-truthy operand can be useful because evaluating it assigns a
value:

```python
if should_continue() and (message := "continuing") and is_ready():
    print(message)
```

The middle operand uses the strict rule, so it does not warn by default.

We does not attempt to classify arbitrary side effects. For example, `if
items.append(value) or fallback:` can still trigger the default rule
because `append` returns `None`.

An `and`/`or` expression used to compute a value (rather than one that
is used directly in an `if` test or similar), such as `result =
items.append(value) or fallback`, is not flagged by either rule.

### Follow the origins of environment-dependent conditions

Both rules exempt compatibility checks involving `sys.version_info`,
`sys.platform`, `os.name`, and `TYPE_CHECKING`. A condition being
constant for the configured environment does not make it redundant
across the environments supported by the project.

The exemption also follows names and attributes through their
definitions, including aliases and imports across modules:

```python
import sys

IS_WINDOWS = sys.platform == "win32"
LINE_PREFIX = "\n" if IS_WINDOWS else ""

if LINE_PREFIX:  # Exempt even though its truthiness is known on this platform.
    print(LINE_PREFIX)
```

The traversal follows assignment values, augmented assignments, walrus
bindings, loop and comprehension iterables, context-manager expressions,
and the subjects of captured `match` patterns.

This lookup is deliberately not flow-sensitive: if any resolved
definition depends on one of these symbols, the use is exempt. This is
so that we err on the side of avoiding false positives.

The shared definition-resolution machinery from #28032 allows this
traversal during inference without requesting completed inference of the
same scope. Name lookups are cached per scope and name; attribute
lookups are cached per receiver type and attribute name. Definition
traversal has its own Salsa query boundary and handles cyclic aliases,
so repeated conditions can share the provenance analysis.

### Preserve defensive assertions and exhaustive dispatch

Defensive boolean/integer assertions such as `assert isinstance(value,
int)` remain useful at runtime even when `value` is annotated as `int`,
since the runtime does not enforce type annotations. These are therefore
exempt from both rules.

Non-boolean mistakes such as `assert function` (where the user meant
`assert function()`) are still reported by the default rule. A complete
assertion whose value type has ambiguous truthiness but whose outcome is
fixed by short-circuit evaluation remains eligible for the strict rule.

For `if` and `elif` statements, the strict rule also recognizes
defensive exits. An always-false condition is exempt when its body ends
in a `raise`, an assertion that might fail, a call returning `Never`, or
a return assignable to `NotImplementedType`:

```python
def increment(value: int) -> int:
    if not isinstance(value, int):
        raise TypeError("expected an int")
    return value + 1
```

For an always-true condition, the same heuristic examines an immediately
following `else` suite, or the remainder of the surrounding suite when
there are no later clauses. This accommodates exhaustive dispatch
followed by `assert_never`, and early-return dispatch followed by a
defensive exception.

These defensive-exit exemptions apply to boolean/integer conditions and
conditions requiring short-circuit analysis. An uncalled function is
still suspicious even when an otherwise unreachable branch raises.

### Check compound conditions once

In `result = flag and function`, Python does not test the truthiness of
the final operand: it simply returns that value if evaluation reaches
it. The rule therefore does not flag `function`, even though it is an
always-truthy object in a boolean expression. In `if flag and
predicate:`, the surrounding `if` does test the truthiness of
`predicate`, so the rule reports the uncalled function even though the
complete condition has ambiguous truthiness.

Unlike `and`/`or` expressions used to compute values, `not` explicitly
tests its operand's truthiness even when it used to computer a value, so
`not` expressions are checked even outside the context of
`if`/`while`/`elif`/`assert` tests.

For compound conditions, the redundant-condition checks visit operands
before deciding whether to report the complete expression. Reporting an
uncalled function in `if not function:` suppresses a second duplicate
diagnostic on the `not function` expression as a whole. Boolean/integer
operands and operands requiring short-circuit analysis are instead
suppressed so that the complete condition can be reported once. This
applies consistently to statement conditions, conditional expressions,
and comprehension filters.

Boolean tests inside value expressions, such as `if consume(not
function):`, are checked independently. Tests in nested scopes are
checked when those scopes are inferred.

Neither rule runs in stub files or files excluded from checking.
Diagnostic and provenance analysis is skipped entirely when both rules
are disabled.

## Diagnostics and fixes

Special-case diagnostics are provided to help explain some of the more
common errors we see in the ecosystem: an uncalled function or method
being always truthy, a fixed-length tuple being always truthy or always
falsy, a `TypedDict` always being truthy due to the presence of
`Required` fields, a `@final` class without `__bool__` or `__len__`
being always truthy, or an unevaluated generator being always truthy.

For simple function and method references, the diagnostic carries a call
fix. A parameterless signature gets an unsafe `()` fix; signatures with
parameters get a display-only `(...)` suggestion. Generator diagnostics
suggest `any()` and carry a display-only wrapping fix when the standard
`any` builtin is known not to be shadowed, including by a custom
`__builtins__.pyi`.

These fixes are deliberately not classified as safe: calling a function
or consuming a generator can change runtime behavior, and the intended
correction may be different.

### Default-rule diagnostics

<details>
<summary>Uncalled function</summary>

![Uncalled
function](https://github.com/user-attachments/assets/9538e4a9-b716-4fff-a4c7-24fc6deb7994)

</details>

<details>
<summary>Uncalled method</summary>

![Uncalled
method](https://github.com/user-attachments/assets/3c327ada-eec0-4bcc-9588-e5df50b3455b)

</details>

<details>
<summary>Generator: suggest <code>any()</code></summary>

![Generator: suggest
any()](https://github.com/user-attachments/assets/83d500fb-d69a-4753-94d6-4008379af8d8)

</details>

<details>
<summary>Tuple with a fixed length</summary>

![Tuple with a fixed
length](https://github.com/user-attachments/assets/e37ade52-22a6-4946-aa29-71f5328bfa79)

</details>

<details>
<summary>Tuple with a minimum length</summary>

![Tuple with a minimum
length](https://github.com/user-attachments/assets/6fc71bd5-6a52-417f-9a9b-db4e760752c3)

</details>

<details>
<summary>Empty tuple</summary>

![Empty
tuple](https://github.com/user-attachments/assets/6c8e8242-fd8b-410d-b56b-e6a49737ea0f)

</details>

<details>
<summary>Nonempty string literals</summary>

![Nonempty string
literals](https://github.com/user-attachments/assets/139f9f53-5128-4d85-8901-01937f312986)

</details>

<details>
<summary>Empty string</summary>

![Empty
string](https://github.com/user-attachments/assets/261fc16d-dc67-44b8-bf51-58f2fcb12e2f)

</details>

<details>
<summary><code>None</code></summary>


![None](https://github.com/user-attachments/assets/c04b0308-b294-4125-bba8-3603fd5bda02)

</details>

<details>
<summary><code>TypedDict</code>: required fields and their
declarations</summary>

![TypedDict: required fields and their
declarations](https://github.com/user-attachments/assets/46367c59-665f-4b40-893b-227add4ca3df)

</details>

<details>
<summary>Final class without <code>__bool__</code> or
<code>__len__</code></summary>

![Final class without __bool__ or
__len__](https://github.com/user-attachments/assets/4e8e92e7-0645-485c-ab30-4d0807943005)

</details>

<details>
<summary>Enum: implicit finality</summary>

![Enum: implicit
finality](https://github.com/user-attachments/assets/6596a69b-a122-40bb-80fc-b011c054e15d)

</details>

### Strict-rule diagnostics

These examples enable `redundant-condition-strict` explicitly.

<details>
<summary>Indexing <code>bytes</code> returns <code>int</code></summary>

![Indexing bytes returns
int](https://github.com/user-attachments/assets/e6f33bc9-c998-4be5-b1ca-36c32b073d99)

</details>

<details>
<summary>Comparing <code>str</code> with <code>bytes</code></summary>

![Comparing str with
bytes](https://github.com/user-attachments/assets/48e15215-d6ae-4779-ab44-244c21dabf55)

</details>

<details>
<summary>Length comparison: explain the fixed tuple length</summary>

![Length comparison: explain the fixed tuple
length](https://github.com/user-attachments/assets/556ef251-ac2f-4048-b7f4-b545a8b87a69)

</details>

<details>
<summary>Comparison: omit an obvious literal's type annotation</summary>

![Comparison: omit an obvious literal's type
annotation](https://github.com/user-attachments/assets/95a0e73f-38f5-457b-9726-8169016b991d)

</details>

## Implementation

The semantic index records the outermost boolean tests in each scope, so
one check owns the diagnostics for a complete condition and its
subexpressions. A shared checker separates classification, context
exemptions, and duplicate suppression from diagnostic rendering. The
main implementation is in
`types/infer/builder/redundant_conditions/mod.rs`; its `diagnostic.rs`
submodule constructs messages and fixes, and `exemptions.rs` handles
environment provenance and defensive exits.
2026-09-05 20:45:59 +01:00
Kris KaczorandCarl Meyer 3d4fd30d2f [ty] Infer yield from send/return types from the iterator returned by __iter__ (#27987)
## Summary

Fixes astral-sh/ty#4368. This is my attempt to fix the issue by using
AI.

`yield from x` delegates to `iter(x)`, so the value of the expression
(and the send type that must be compatible with the outer generator)
comes from the *iterator* returned by `x.__iter__()`, not from `x`
itself. Previously we only looked at `x`'s own MRO for
`Generator`/`Iterator`, so for a class whose `__iter__` is a generator
function we inferred `Unknown` and skipped the send-type check:

```py
class Box:
    def __iter__(self) -> Generator[str, None, int]:
        yield "hello"
        return 42

def main() -> Generator[str, None, int]:
    x = yield from Box()
    reveal_type(x)  # was: Unknown, now: int
    y: str = yield from Box()  # now: error[invalid-assignment]
    return x
```

This PR resolves the inner `GeneratorTypes` once in
`infer_yield_from_expression`: if the operand itself is a
generator/iterator we use that (unchanged behaviour, keeps the protocol
materialization handling), otherwise we call `__iter__` and take the
generator types of the returned iterator. Both the send-type validation
and the expression's type now use that. The now-unused
`generator_send_type` helper is removed.

`yield from` over a plain Iterator now consistently returns `Unknown`,
since the `Iterator` type does not specify the return type, and an
iterator implementation could raise `StopIteration` with any value,
without violating any annotated type.

## Test Plan

- New mdtests in `expression/yield_and_yield_from.md` covering the
issue's example, a send-type mismatch detected through `__iter__`, and
the `Iterator` / list case.
- `cargo test -p ty_python_semantic`, `cargo clippy -p
ty_python_semantic --all-targets -- -D warnings`, `cargo fmt --check`.
- Checked the built CLI against the repro from the issue and against a
real-world codebase relying on this pattern (effecton); the expected new
diagnostic appears and nothing else changes.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2026-08-25 22:14:05 -07:00
Charlie Marsh b0f6523709 [ty] Preserve string annotation context in lambda defaults and TypedDicts (#27914)
## Summary

Follow-up to #27882.

Previously, changing deferred lookup modes while inferring `Annotated`
metadata could discard the enclosing string-annotation context, causing
parsed expressions to be looked up in the module's semantic index and
panic:

```python
from typing_extensions import Annotated, TypedDict

value: "Annotated[int, lambda default=int: None]"
other: "Annotated[int, TypedDict('T', {}, extra_items=int)]"
```

We now preserve `InStringAnnotation` while inferring lambda defaults.
For `TypedDict.extra_items`, we pass through the existing state and rely
on annotation inference to defer stub annotations without losing their
enclosing string context. This also covers nested lambdas, dynamic
classes in defaults, and positional-only and keyword-only stub defaults.

All 838 semantic tests pass.
2026-08-19 21:59:04 -04:00
Dhruv Manilawala c876a953dd [ty] Fix variadic signature display (#27869)
## Summary

Preserve the unpacking `*` in named variadic annotations and avoid
adding a redundant bare `*` before keyword-only parameters when `*args`
is already present. This fixes malformed revealed signatures without
changing signature storage or inference.

## Test plan

- Add a directly declared mixed-signature regression.
- Update TypeVarTuple partial, lambda, and bidirectional reveal
expectations.
- Regenerate existing single-line and multiline signature-display
snapshots.
2026-08-19 10:59:10 +05:30
Lérè 93901e6478 [ty] Separate place-load resolution from type inference. (#27319)
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
  requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
This introduces a model called a `PlaceLoad`, which provides access to
place-load resolution logic (i.e., name resolution and reaching
definition analysis) independently from type inference. That logic can
then be used for things outside of type inference (e.g., for language
server refactors which require reaching definition analysis, like [file
rename support](https://github.com/astral-sh/ty/issues/1560)).

In order to clarify exactly where we've carved out place-load resolution
from type inference, this PR makes the separation directly in
`TypeInferenceBuilder`, but a [subsequent
change](https://github.com/astral-sh/ruff/pull/27320) will move that
logic to a location that is more easily shared.

I recommend reading the new `place_load.rs` module first, and then the
changes to `infer/builder.rs` that integrate it.

## Test Plan

This is primarily a refactor that relies on existing test coverage, but
I have tweaked or added test coverage in places where we have corrected
our name resolution logic.
<!-- How was it tested? -->
2026-08-11 12:07:40 -07:00
Alex Waygood 78cad66655 [ty] Add an opt-in unsound-yield lint (#27593) 2026-08-09 15:02:37 +01:00
Alex Waygood fbcaa20423 [ty] Expand type aliases for Generator when evaluating a generator function's return/send/yield type (#27577) 2026-08-07 17:43:55 +01:00
Ed Page 958efb1ff4 Vendor latest annotate-snippets (#27033)
## Summary

This is a first step towards #20411. After this, we would work to either
upstream changes or adjust things within Ruff.

Performance improvements have already been upstreamed
(https://github.com/rust-lang/annotate-snippets-rs/pull/437,
https://github.com/rust-lang/annotate-snippets-rs/pull/440)

Re-implemented the following from the fork:
- Hidden snippets
(https://github.com/rust-lang/annotate-snippets-rs/issues/448)
- Renderer-level disabling of hyperlinks
- Jupyter cell indices
(https://github.com/rust-lang/annotate-snippets-rs/issues/449)
- Fixable mark
(https://github.com/rust-lang/annotate-snippets-rs/issues/450)
- `lineno_offset` (maybe not needed if #20648 is taken care of)
- anonymized lines affects origin
(https://github.com/rust-lang/annotate-snippets-rs/issues/451)
- cut indicator (maybe not needed if we can switch to unicode renderer)

Additional hacks to minimize behavior change include
- Adjust upstream's hidden severity behavior to match Ruff's
(https://github.com/rust-lang/annotate-snippets-rs/issues/452)
- How newlines are annotated
(https://github.com/rust-lang/annotate-snippets-rs/issues/454)
- To not indent for line numbers when none will be present
(https://github.com/rust-lang/annotate-snippets-rs/issues/453)
- Enabling `simd` by default

All of the above represents work that needs to be resolved to address
#20411.

`lib.name` hack is used to simplify maintenance with the assumption that
we'll continue working towards #20411.

Behavior changes that were included
- Tweak when implicit padding lines are present
- Fix Origin's line/column is now derived from the primary annotation
- Make consistent the rendering of primary/context annotations
- Fix snippets to show up for empty source (but may degrade #24528)
- Fix handling of term width
- Fix rendering of annotations between newline and first character
(granted, maybe the caller is wrong)
- Tweak multiline annotation markers

New or upcoming features annotate-snippets features that we will be able
to leverage by moving this forward
- `AnnotationKind::Visible` to improve the quality of built-in
code-folding if Ruff were to switch to it
- `DecorStyle::Unicode`
- *(upcoming)* No-graphiscs mode for screen readers (may also be useful
for AIs)

Deferred:
- Moving related `Group`s into the same `Report`
- Using `secondary_title` for any `Group` after the first in a `Report`
- Using a `Snippet`, instead of titles, for "virtual code" like
https://github.com/astral-sh/ruff/pull/27033#discussion_r3622727236
- Switching hidden snippets to `Origin`

## Test Plan

Within cpython, I ran
```console
$ hyperfine -i "../../ruff/target-baseline/release/ruff check" "../../ruff/target-update/release/ruff check" -N -m20 --output=null
Benchmark 1: ../../ruff/target-baseline/release/ruff check
  Time (mean ± σ):      27.5 ms ±   1.1 ms    [User: 33.0 ms, System: 108.4 ms]
  Range (min … max):    25.0 ms …  31.7 ms    94 runs

  Warning: Ignoring non-zero exit code.

Benchmark 2: ../../ruff/target-update/release/ruff check
  Time (mean ± σ):      28.1 ms ±   2.0 ms    [User: 33.4 ms, System: 105.7 ms]
  Range (min … max):    24.8 ms …  38.9 ms    102 runs

  Warning: Ignoring non-zero exit code.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use t
he '--warmup' or '--prepare' options.

Summary
  ../../ruff/target-baseline/release/ruff check ran
    1.03 ± 0.08 times faster than ../../ruff/target-update/release/ruff check
```

Before:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/2d94dabf-b28d-48c1-b17c-52aac59fd57e"
/>

After:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/b388ada3-e3d7-4ba0-b90b-1330eb81f4be"
/>
2026-07-28 11:57:09 -05:00
Samuel RigaudandSamuel Rigaud 3f0ea98047 fix typos (#26462)
Another round of typos being fixed 

<img width="300" height="200" alt="image"
src="https://github.com/user-attachments/assets/acdaea05-0d28-4537-94db-9a809d76e00c"
/>

Co-authored-by: Samuel Rigaud <rigaud@gmail.com>
2026-07-01 23:23:21 +01:00
ReginaldandJelle Zijlstra dc79edf868 [ty] Add missing-type-argument lint rule (#25617)
## Summary

Add a new lint that detects generic types used without explicit type
parameters in annotations (e.g. `list` instead of `list[int]`).
This is equivalent to mypy's `--disallow-any-generics` and pyright's
`reportMissingTypeArgument`.

Can be enabled via:
```
    [rules]
    missing-type-argument = "warn"
```
It skips classes where all type parameters have PEP 696 defaults.

This should fix the following issue in ty:
- https://github.com/astral-sh/ty/issues/2442

## Test Plan

Added tests covering stdlib generics, user-defined generics, PEP 696
defaults, dotted names, function signatures, base classes, and variable annotations.

---------

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2026-06-05 13:07:10 -07:00
Charlie Marsh 889f6afc1b [ty] Synthesize precise __len__ methods for literals (#25600)
## Summary

Small follow-up to #25347.

`len(...)` already returns precise literal lengths for string and bytes
literals, but the `__len__` member lookup was returning `int`, which
prevented exact-length protocol intersections from eliminating
incompatible literal values.

We now synthesize precise `__len__` methods for string and bytes
literals so checks such as `len(value) == 1` narrow unions of literals
to the values with matching lengths:

```py
def f(x: Literal[b"", b"a"]):
    if len(x) == 1:
        reveal_type(x)  # Literal[b"a"]
```
2026-06-03 12:56:51 -04:00
David Peter c714e84952 [ty] Modernize setup of union types in mdtests (#25291)
Remember when we wrote `x = 1 if flag else None` because we couldn't
infer types from annotated parameters and/or couldn't understand union
type annotations? This PR modernizes our mdtests to simply use annotated
function parameters in those cases (I manually rejected some rewrites
when the new style was less clear, for example when the generated union
type itself was hard to spell).

The motivation for this is to set a better precedent for agents, as I've
seen them repeating that `x if flag else y` pattern, which is really
unnecessary, longer and harder to read in most cases.
2026-05-21 13:39:46 +00:00
David Peter 6c09271238 [ty] Add error context to invalid-return-type diagnostics (#24770)
## Summary

Add error context to `invalid-return-type` diagnostics as well.

## Test Plan

New snapshot test
2026-04-21 17:55:36 +02:00
Alex Waygood 65d768e4e2 [ty] Merge same-file annotations if there is only a single line separating them (#24694) 2026-04-17 17:00:07 +01:00
David Peter 08629b4954 [ty] Migrate more mdtests to inline snapshots (#24687)
## Summary

Migrate another ~20 test suites to inline snapshots (Friday batch).
2026-04-17 11:49:09 +02:00
Carl Meyer 590aab49fc [ty] stop unioning Unknown into types of un-annotated attributes (#24531) 2026-04-10 10:34:10 -07:00
Ibraheem Ahmed d444d52e2b [ty] Infer lambda expressions with Callable type context (#22633)
Infer lambda expressions eagerly as part of their parent scope, and with
type context. This allows us to infer more precise types for lambda
expressions, as well as perform check assignability against `Callable`
annotations.

Note that this does not change the inferred type of a lambda parameter
with the body of the lambda, even if it is annotated. That part is a
little more tricky, so will be addressed in a followup PR.
2026-03-26 14:14:40 -04:00
Shaygan HooshyariandDavid Peter b491f7ebed [ty] Check return type of generator functions (#24026)
## Summary

Perform return type checking for generator functions.

part of https://github.com/astral-sh/ty/issues/1718

## Test Plan

The two newly added diagnostics in ecosystem result are correct the rest
were marked as flaky. So looks good.
I updated the tests for `return_types.md`.


---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2026-03-25 13:15:05 +01:00
Charlie MarshandCarl Meyer 523fcf92b0 [ty] Support narrowing for extended walrus targets (#24129)
## Summary

Closes https://github.com/astral-sh/ty/issues/3103.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2026-03-23 18:23:13 -07:00
Shaygan HooshyariandDavid Peter f3b61a0fd2 [ty] Infer yield expression (#23796)
## Summary

Infer yield and yield from expression using function annotation.

Part of https://github.com/astral-sh/ty/issues/1718

Remaining part is checking return types, I'm planning to do that in a
separate PR.
All the remaining conformance tests are because of the return type
checking.

## Notes

Mixing Generator and Async Generator is allowed.

```py
def async_returns_generator() -> Generator[int, int, None] | AsyncGenerator[int, int]:
    x = yield 1
    reveal_type(x)
    return None
```

I was not sure if there's valid use case for this or not.
But I found [this
code](https://github.com/vippsas/zeroeventhub/blob/main/python/zeroeventhub/zeroeventhub/data_reader.py#L22)
and [mypy](https://mypy-play.net/?gist=4293848a184dac5a7b49d3a7b5ed71c8)
allows this. So I'm not doing any checks for this case. Just the normal
logic that unions the types.

[Pyright](https://pyright-play.net/?pyrightVersion=1.1.405&strict=true&code=GYJw9gtgBALgngBwJYDsDmUkQWEMogCmAboQIYA2A%2BvAoQFCiSyKoZY55QDihKhIMjFwAaKAEEAznBQBjXv0HCQ9egBNCwKGWlyqRGAFcQKSVTR8BQ3AAoAlFAC0APh6WluANqoYYn2IA5MH4AXSgAHwldeXdrEG8UX0xEkIAueihMqAAPKABeKDgkQgo1KABGDKyiUkoaREIbbLsqzINjFCgg-iA)
rejects this.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2026-03-20 21:41:57 +01:00
Amethyst Reese 09db380143 Apply ruff formatting to mdtests (#22935)
- Updated the `prek` hook to replace `blacken-docs` with ruff
- Added some formatting directives to preserve obvious intentional
formatting choices
- Made some choice edits to content for better readability with new format
- Formatted md tests and updated snapshots
2026-02-13 08:48:14 -08:00
Alex Waygood 56077ee9a9 [ty] Fix @Todo type for starred expressions (#22503) 2026-01-13 21:09:29 +00:00
Aria Desires ad3de4e488 [ty] Improve rendering of default values for function args (#22010)
## Summary

We're actually quite good at computing this but the main issue is just
that we compute it at the type-level and so wrap it in `Literal[...]`.
So just special-case the rendering of these to omit `Literal[...]` and
fallback to `...` in cases where the thing we'll show is probably
useless (i.e. `x: str = str`).

Fixes https://github.com/astral-sh/ty/issues/1882
2025-12-16 13:39:19 -05:00
Luca Chiodini 5a9d6a91ea [ty] Uniformly use "not supported" in diagnostics (#21916) 2025-12-11 15:03:55 +00:00
Alex Waygood e642874cf1 [ty] Check method definitions on subclasses for Liskov violations (#21436) 2025-11-23 18:08:15 +00:00
Carl Meyer 6b7adb0537 [ty] support PEP 613 type aliases (#21394)
Refs https://github.com/astral-sh/ty/issues/544

## Summary

Takes a more incremental approach to PEP 613 type alias support (vs
https://github.com/astral-sh/ruff/pull/20107). Instead of eagerly
inferring the RHS of a PEP 613 type alias as a type expression, infer it
as a value expression, just like we do for implicit type aliases, taking
advantage of the same support for e.g. unions and other type special
forms.

The main reason I'm following this path instead of the one in
https://github.com/astral-sh/ruff/pull/20107 is that we've realized that
people do sometimes use PEP 613 type aliases as values, not just as
types (because they are just a normal runtime assignment, unlike PEP 695
type aliases which create an opaque `TypeAliasType`).

This PR doesn't yet provide full support for recursive type aliases
(they don't panic, but they just fall back to `Unknown` at the recursion
point). This is future work.

## Test Plan

Added mdtests.

Many new ecosystem diagnostics, mostly because we
understand new types in lots of places.

Conformance suite changes are correct.

Performance regression is due to understanding lots of new
types; nothing we do in this PR is inherently expensive.
2025-11-20 17:59:35 -08:00
Alex Waygood 39f105bc4a [ty] Use "cannot" consistently over "can not" (#21255) 2025-11-03 10:38:20 -05:00
Alex Waygood bf74c824eb [ty] Delegate truthiness inference of an enum Literal type to its enum-instance supertype (#21060) 2025-10-24 14:34:16 +01:00
Alex Waygood e196c2ab37 [ty] Consider __len__ when determining the truthiness of an instance of a tuple class or a @final class (#21049) 2025-10-24 09:29:55 +00:00
Alex Waygood 1f8297cfe6 [ty] Improve error messages for unresolved attribute diagnostics (#20963)
## Summary

- Type checkers (and type-checker authors) think in terms of types, but
I think most Python users think in terms of values. Rather than saying
that a _type_ `X` "has no attribute `foo`" (which I think sounds strange
to many users), say that "an object of type `X` has no attribute `foo`"
- Special-case certain types so that the diagnostic messages read more
like normal English: rather than saying "Type `<class 'Foo'>` has no
attribute `bar`" or "Object of type `<class 'Foo'>` has no attribute
`bar`", just say "Class `Foo` has no attribute `bar`"

## Test Plan

Mdtests and snapshots updated
2025-10-19 10:58:25 +01:00
Renkai GeandAlex Waygood bf38e69870 [ty] Rename "possibly unbound" diagnostics to "possibly missing" (#20492)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-09-23 14:26:55 +00:00
Alex Waygood 33b3d44ebd [ty] Proper assignability/subtyping checks for protocols with method members (#20165) 2025-09-12 10:10:31 +00:00
Alex Waygood 0e3697a643 [ty] Minor fixes to Protocol tests (#20347) 2025-09-11 14:42:13 +00:00
David Peter 4ecf1d205a [ty] Support async/await, async with and yield from (#19595)
## Summary

- Add support for the return types of `async` functions
- Add type inference for `await` expressions
- Add support for `async with` / async context managers
- Add support for `yield from` expressions

This PR is generally lacking proper error handling in some cases (e.g.
illegal `__await__` attributes). I'm planning to work on this in a
follow-up.

part of https://github.com/astral-sh/ty/issues/151

closes https://github.com/astral-sh/ty/issues/736

## Ecosystem

There are a lot of true positives on `prefect` which look similar to:
```diff
prefect (https://github.com/PrefectHQ/prefect)
+ src/integrations/prefect-aws/tests/workers/test_ecs_worker.py:406:12: error[unresolved-attribute] Type `str` has no attribute `status_code`
```

This is due to a wrong return type annotation
[here](https://github.com/PrefectHQ/prefect/blob/e926b8c4c114e74533e7bd75d83e7f205c774645/src/integrations/prefect-aws/tests/workers/test_ecs_worker.py#L355-L391).

```diff
mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ test/mitmproxy/addons/test_clientplayback.py:18:1: error[invalid-argument-type] Argument to function `asynccontextmanager` is incorrect: Expected `(...) -> AsyncIterator[Unknown]`, found `def tcp_server(handle_conn, **server_args) -> Unknown | tuple[str, int]`
```


[This](https://github.com/mitmproxy/mitmproxy/blob/a4d794c59a27472d193a592d8037505a1cf6ae93/test/mitmproxy/addons/test_clientplayback.py#L18-L19)
is a true positive. That function should return
`AsyncIterator[Address]`, not `Address`.

I looked through almost all of the other new diagnostics and they all
look like known problems or true positives.

## Typing conformance

The typing conformance diff looks good.

## Test Plan

New Markdown tests
2025-07-30 11:51:21 +02:00
Alex Waygood cb5a9ff8dc [ty] Make tuple subclass constructors sound (#19469) 2025-07-21 21:25:11 +00:00
Alex WaygoodandBrent Westbrook c2380fa0e2 [ty] Extend tuple __len__ and __bool__ special casing to also cover tuple subclasses (#19289)
Co-authored-by: Brent Westbrook
2025-07-21 12:50:46 +00:00
Alex Waygood a50a993b9c [ty] Make tuple instantiations sound (#18987)
## Summary

Ensure that we correctly infer calls such as `tuple((1, 2))`,
`tuple(range(42))`, etc. Ensure that we emit errors on invalid calls
such as `tuple[int, str]()`.

## Test Plan

Mdtests
2025-06-27 19:37:16 +01:00
lipefree f379eb6e62 [ty] Treat lambda functions as instances of types.FunctionType (#18431) 2025-06-02 16:46:26 +01:00
Alex Waygood d1bb10a66b [ty] Understand classes that inherit from subscripted Protocol[] as generic (#17832) 2025-05-09 17:39:15 +01:00
Alex Waygood 03a4d56624 [ty] Change range of revealed-type diagnostic to be the range of the argument passed in, not the whole call (#17980) 2025-05-09 14:15:39 +01:00
Charlie Marsh a2e9a7732a Update class literal display to use <class 'Foo'> style (#17889)
## Summary

Closes https://github.com/astral-sh/ruff/issues/17238.
2025-05-06 20:11:25 -04:00
Micha Reiser b51c4f82ea Rename Red Knot (#17820) 2025-05-03 19:49:15 +02:00