Commit Graph
21 Commits
Author SHA1 Message Date
Alex Waygood bf3591bb03 [ty] Rename implicit-concatenated-string-type-annotation to implicit-string-concatenation-in-annotation; improve documentation (#30286) 2026-11-11 14:08:11 +00:00
Alex Waygood 4fc78f66fd [ty] Add Literal fixes for diagnostics flagging invalid type annotations (#28185)
## Summary

Add unsafe fixes to diagnostics that suggest `Literal`: wrap integer,
bytes, and boolean literals and strings that cannot be parsed as forward
annotations in `Literal[...]`, and replace parameterized `LiteralString`
with `Literal` when its arguments are valid literal parameters.

The fixes preserve literal spelling and reuse unshadowed import aliases.
They import from `typing` on Python 3.8+, or from `typing_extensions` on
Python 3.7 when it is a declared direct dependency and the installed
module exports `Literal`. The typing-module availability checks are
shared with the `reveal_type` and `assert_never` fixes.
2026-09-06 08:35:58 +00:00
Carl Meyer 2d942cc99a [ty] Recognize ABC metaclass methods on protocols (#27897)
Valid calls such as `collections.abc.Mapping.register(...)` currently
produce missing-attribute errors. Using `register` as a class decorator
can also hide the registered class’s members and generic parameters,
causing further false positives and negatives.

This PR gives protocol classes declared in source or ordinary `.pyi`
files the metaclass `typing._ProtocolMeta`, which derives from
`ABCMeta`. That makes ABC methods available and preserves the types of
registered classes. It also lets ty report incompatible custom
metaclasses.

Typeshed’s standard-library stubs need a narrow compatibility exception:
they sometimes list `Protocol` bases that the runtime class does not
inherit. When those bases do not select a custom metaclass, ty uses
`ABCMeta` for attribute lookup but does not let that inferred metaclass
create artificial inheritance conflicts. This applies to bundled or
configured typeshed, not to other `.pyi` files. An explicitly chosen
custom metaclass continues to constrain subclasses. Known built-ins
retain their actual `type` metaclass.

Mypy applies a similar permissive fallback more broadly; this PR limits
the exception to typeshed.

Fixes https://github.com/astral-sh/ty/issues/1204.

## Ecosystem results

The [current ecosystem
run](https://github.com/astral-sh/ruff/actions/runs/32385816515/attempts/1)
removes missing-`register` errors in pandas, ppb-vector, Jinja, Spack,
and Ibis, plus two spurious metaclass conflicts in Spack’s legacy
`typing_extensions`. All 294 Ibis changes follow from recovering
registered classes’ members and generic parameters. The added
diagnostics expose existing annotation gaps, deliberately invalid tests,
and the known [virtual ABC subclass
limitation](https://github.com/astral-sh/ty/issues/2391).

The beartype library has an [old compatibility
workaround](https://github.com/beartype/beartype/blob/d6c056225f2e5632ea72006b43ffb066bd9a2c6b/beartype/typing/_typingpep544.py#L73-L79)
for the fact that `_ProtocolMeta` used to be missing from typeshed; it
substitutes `ABCMeta` under `if TYPE_CHECKING`, so it now produces a
metaclass conflict. We accept this false positive: typeshed now declares
`_ProtocolMeta`, and beartype can remove the workaround and derive from
`type(Protocol)` directly.

## Test plan

- Cover protocol metaclasses, bundled and custom typeshed, and
registration decorators that preserve generic classes.
- Check compatible and conflicting metaclasses, built-in collections,
and inheritance through ordinary and dynamically created classes.
- Protect subclass-type bounds, class-namespace attributes, quoted
unions, and dataclass-transform inheritance.
2026-08-21 14:44:36 -07:00
Ed Page 958efb1ff4 Vendor latest annotate-snippets (#27033)
## Summary

This is a first step towards #20411. After this, we would work to either
upstream changes or adjust things within Ruff.

Performance improvements have already been upstreamed
(https://github.com/rust-lang/annotate-snippets-rs/pull/437,
https://github.com/rust-lang/annotate-snippets-rs/pull/440)

Re-implemented the following from the fork:
- Hidden snippets
(https://github.com/rust-lang/annotate-snippets-rs/issues/448)
- Renderer-level disabling of hyperlinks
- Jupyter cell indices
(https://github.com/rust-lang/annotate-snippets-rs/issues/449)
- Fixable mark
(https://github.com/rust-lang/annotate-snippets-rs/issues/450)
- `lineno_offset` (maybe not needed if #20648 is taken care of)
- anonymized lines affects origin
(https://github.com/rust-lang/annotate-snippets-rs/issues/451)
- cut indicator (maybe not needed if we can switch to unicode renderer)

Additional hacks to minimize behavior change include
- Adjust upstream's hidden severity behavior to match Ruff's
(https://github.com/rust-lang/annotate-snippets-rs/issues/452)
- How newlines are annotated
(https://github.com/rust-lang/annotate-snippets-rs/issues/454)
- To not indent for line numbers when none will be present
(https://github.com/rust-lang/annotate-snippets-rs/issues/453)
- Enabling `simd` by default

All of the above represents work that needs to be resolved to address
#20411.

`lib.name` hack is used to simplify maintenance with the assumption that
we'll continue working towards #20411.

Behavior changes that were included
- Tweak when implicit padding lines are present
- Fix Origin's line/column is now derived from the primary annotation
- Make consistent the rendering of primary/context annotations
- Fix snippets to show up for empty source (but may degrade #24528)
- Fix handling of term width
- Fix rendering of annotations between newline and first character
(granted, maybe the caller is wrong)
- Tweak multiline annotation markers

New or upcoming features annotate-snippets features that we will be able
to leverage by moving this forward
- `AnnotationKind::Visible` to improve the quality of built-in
code-folding if Ruff were to switch to it
- `DecorStyle::Unicode`
- *(upcoming)* No-graphiscs mode for screen readers (may also be useful
for AIs)

Deferred:
- Moving related `Group`s into the same `Report`
- Using `secondary_title` for any `Group` after the first in a `Report`
- Using a `Snippet`, instead of titles, for "virtual code" like
https://github.com/astral-sh/ruff/pull/27033#discussion_r3622727236
- Switching hidden snippets to `Origin`

## Test Plan

Within cpython, I ran
```console
$ hyperfine -i "../../ruff/target-baseline/release/ruff check" "../../ruff/target-update/release/ruff check" -N -m20 --output=null
Benchmark 1: ../../ruff/target-baseline/release/ruff check
  Time (mean ± σ):      27.5 ms ±   1.1 ms    [User: 33.0 ms, System: 108.4 ms]
  Range (min … max):    25.0 ms …  31.7 ms    94 runs

  Warning: Ignoring non-zero exit code.

Benchmark 2: ../../ruff/target-update/release/ruff check
  Time (mean ± σ):      28.1 ms ±   2.0 ms    [User: 33.4 ms, System: 105.7 ms]
  Range (min … max):    24.8 ms …  38.9 ms    102 runs

  Warning: Ignoring non-zero exit code.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use t
he '--warmup' or '--prepare' options.

Summary
  ../../ruff/target-baseline/release/ruff check ran
    1.03 ± 0.08 times faster than ../../ruff/target-update/release/ruff check
```

Before:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/2d94dabf-b28d-48c1-b17c-52aac59fd57e"
/>

After:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/b388ada3-e3d7-4ba0-b90b-1330eb81f4be"
/>
2026-07-28 11:57:09 -05:00
Alex WaygoodandMicha Reiser 366fe21ba3 [ty] Improve diagnostics for syntax errors in forward annotations (#25158)
## Summary

Fixes https://github.com/astral-sh/ty/issues/1627.

Here's an example diagnostic with the current release of ty:

<img width="2286" height="286" alt="image"
src="https://github.com/user-attachments/assets/46ab9154-aaf0-4451-b301-5ff12fcd8507"
/>

On this branch, this diagnostic becomes:

<img width="1464" height="408" alt="image"
src="https://github.com/user-attachments/assets/449e0b6c-58d2-4501-91df-c267db6f48ca"
/>

The exact span of the node that creates the syntax error is now retained
and highlighted in the diagnostic.

## Implementation

Propagating the range of the node inside the string annotation into the
diagnostic is trivial. However, naively implementing that quickly
revealed that this would make diagnostics like this unsuppressable:

```py
x: """list[
    yield from range(42)
]"""
```

The primary range of the diagnostic is now specifically the `yield from
range(42)` part of the string rather than the string node as a whole.
But I cannot add a `ty: ignore` comment that is either on or above the
`yield from range(42)` part of the string -- the "comment" there would
just become part of the string.

This PR also improves the consistency of our parser error messages in
general when it comes to capitalization.

## Test Plan

Mdtests extended and updated

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2026-05-28 10:51:20 +00:00
Charlie Marsh 2428fca932 [ty] Ignore and reject annotations on non-name targets (#25324)
## Summary

Like Mypy and Pyright, we now ignore annotations on non-name (i.e.,
subscript and attribute) targets, and emit a diagnostic for the invalid
annotation -- with the exception of annotated assignments on `self` and
class attributes (i.e., receivers).

Closes https://github.com/astral-sh/ty/issues/509.
2026-05-27 13:59:49 +00:00
Alex Waygood 770cca65a1 [ty] Use basic blocks for determining if a node is in an if TYPE_CHECKING block (#24394) 2026-04-06 20:05:04 +00:00
Alex Waygood b889571743 [ty] Use infer_type_expression for parsing parameter annotations and return-type annotations (#24353)
## Summary

Currently we call `infer_annotation_expression` on return annotations
and parameter annotations, and then after that we do some ad-hoc checks
to make sure that the `TypeAndQualifiers` returned doesn't actually have
any qualifiers in it. A simpler way of checking that an annotation
expression doesn't contain type qualifiers is by simply calling
`infer_type_expression` rather than `infer_annotation_expression`, since
type qualifiers are always banned in type expressions -- they're only
valid in annotation expressions.

A naive way of doing this would regress our error messages -- rather
than saying "Type qualifiers are not valid in parameter annotations", we
would end up with the vaguer and more jargon-y "Type qualifiers are not
valid in type expressions" error message. That's easily fixed, however,
by plumbing some more context through our existing `InferenceFlags`
bitflag. Doing this allows us to improve many of our existing error
messages as well.

## Test Plan

mdtests updated
2026-04-02 16:05:42 +00:00
Alex Waygood 34d54b6983 [ty] Improve consistency and quality of diagnostics relating to invalid type forms (#24325) 2026-04-02 14:16:29 +00:00
Alex Waygood ab032bf77e [ty] Avoid emitting cascading diagnostics when parsing invalid type expressions (#24326)
## Summary

In lots of places in our type-expression parsing, we continue to call
`self.infer_type_expression()` on sub-expressions in the AST even after
we've already determined that the type expression is invalid. I think
that's generally a mistake; it often leads to us emitting many
diagnostics on a single type expression when one would really be
sufficient. This PR switches many callsites from `infer_type_expression`
to `infer_expression`, to avoid this phenomenon of cascading diagnostics
in error cases.

## Test Plan

Mdtests and snapshots updated.
2026-04-01 12:09:10 +01:00
Alex Waygood 91fc7bd3f9 [ty] Fix false-positive diagnostics for PEP-604 union annotations on attribute targets on Python 3.9 when from __future__ import annotations is active (#23915) 2026-03-12 19:01:00 +00:00
Alex Waygood 2185bec02b [ty] Fixup restoration of multi-inference state in type_expression.rs (#23760)
## Summary

Two small followups to d9daf6d430:
- A minor fix for a bug I introduced in that commit. We should reset
`context.multi_inference` to the state it was previously in rather than
hardcoding `false` here. This has real-world impact on `altair` (see the
mypy_primer report)
- Also detect invalid PEP-604 unions involving `types.GenericAlias`
instances and objects that don't have `__or__` methods at runtime. These
need to be special-cased in the same way as class-literal types, because
of typeshed's very forgiving `types.GenericAlias.__or__` annotation:
https://github.com/python/typeshed/blob/07ffb67b924d5f532f8b8b72a4902f58faae6aca/stdlib/types.pyi#L700-L702

## Test Plan

mdtests extended
2026-03-06 23:15:37 +00:00
Alex Waygood d9daf6d430 [ty] Detect invalid partially stringified PEP-604 unions (#23285) 2026-03-06 11:15:43 +00:00
Amethyst Reese 09db380143 Apply ruff formatting to mdtests (#22935)
- Updated the `prek` hook to replace `blacken-docs` with ruff
- Added some formatting directives to preserve obvious intentional
formatting choices
- Made some choice edits to content for better readability with new format
- Formatted md tests and updated snapshots
2026-02-13 08:48:14 -08:00
Aria Desires 2c902c7c68 [ty] fix bug in string annotations and clean up diagnostics (#22913)
This multiplication was always using only using the larger sub-ast size
value, which didn't do anything unsound but made sub-string annotations
become exhausted on files 32x smaller than expected (16k nodes instead
of 512k nodes).

Also I decided to do a cleanup pass on the diagnostics to make them more
precise and helpful.
2026-01-28 10:55:35 -05:00
Aria Desires 45acbd0b66 [ty] Improve support for goto-type, goto-declaration, hover, and highlighting of string annotations (#22878)
I ultimately hunted down the core issue with my previous approach to "if
you give string annotation sub-AST nodes any kind of NodeIndex, a bunch
of random places in the code will start thinking it's ok to store info
about them, which is a problem because they all count up from index 0
which creates conflicts/crashes".

Rather than trying to play whackamole I decided to create a scheme to
give string annotation nodes a unique NodeIndex by shifting up the
parent node's index -- so 0xAB's sub-AST nodes look like 0xAB00...0000,
0xAB00..0001, etc. This scheme avoids any collisions for any reasonable
AST (most string annotations are like, a dozen sub-nodes, so they need
maybe 4 or 5 bits, which would require hundreds of MB of python code to
run out of bits...).

As a bonus, this admits an extremely simple implementation of recording
and fetching sub-AST types... they just are stored now, and you can just
pass in their NodeIndex and get back actual results.

* Fixes https://github.com/astral-sh/ty/issues/1640
* Fixes https://github.com/astral-sh/ty/issues/2028
2026-01-27 23:26:40 -05:00
Charlie Marsh 0a1dddbf73 [ty] Emit invalid type form for stringified annotations (#22752)
## Summary

Closes https://github.com/astral-sh/ty/issues/2553.
2026-01-20 08:37:55 +00:00
Charlie Marsh 682d29c256 [ty] Avoid enforcing standalone expression for tests in f-strings (#21967)
## Summary

Based on what we do elsewhere and my understanding of "standalone"
here...

Closes https://github.com/astral-sh/ty/issues/1865.
2025-12-15 22:31:04 -05:00
Charlie MarshandDavid Peter 5c320990f7 [ty] Avoid inferring types for invalid binary expressions in string annotations (#21911)
## Summary

Closes https://github.com/astral-sh/ty/issues/1847.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-12-11 09:40:19 +01:00
med1844 0ec2ad2fa5 [ty] Emit error for invalid binary operations in type expressions (#18991)
## Summary

This PR adds diagnostic for invalid binary operators in type
expressions. It should close https://github.com/astral-sh/ty/issues/706
if merged.

Please feel free to suggest better wordings for the diagnostic message.

## Test Plan

I modified `mdtest/annotations/invalid.md` and added a test for each
binary operator, and fixed tests that was broken by the new diagnostic.
2025-06-30 10:06:01 +02:00
Micha Reiser b51c4f82ea Rename Red Knot (#17820) 2025-05-03 19:49:15 +02:00