Files
Carl Meyer 7ae3af9e7d [ty] Preserve tuple shapes during concatenation (#28617)
Tuple concatenation currently discards element positions and tuple
length, so adding two fixed-length tuples produces a variable-length
tuple. Preserve the concatenated shape, including known prefixes and
suffixes around variable-length portions.

As with tuple slicing and iteration, this (unsoundly) assumes values
annotated as `tuple` behave like ordinary tuples, even though a subclass
could override `__add__` and/or `__radd__` incompatibly. Explicitly
known subclasses continue to use their operator methods.

Keep inference bounded by approximating large tuple combinations and
widening tuple lengths when their alternatives change between cycle
iterations. Stable tuple unions, including those supplied by collection
type context, retain their shapes. Some cycles with finite but changing
tuple lengths can still lose precision to ensure convergence.

Fixes https://github.com/astral-sh/ty/issues/3413.

## Ecosystem impact

- Removes tuple-shape false positives in Hydra-zen, Trio, Spark, and
SymPy, and two unsoundness warnings in pandas.
- Most other changes make existing diagnostics more precise. New errors
expose overly narrow tuple attributes and collection element types in
zope.interface and SymPy, including invariant dictionary key types.
- One new xarray false positive exposes existing dictionary-key
inference that incorrectly restricts keys to strings. This limitation is
left for separate work. Cycle widening also loses finite tuple
alternatives in Apprise, consolidating existing errors without changing
which operations are rejected.

[Full ecosystem
analysis](https://github.com/astral-sh/ruff/pull/28617#issuecomment-5687202176).

## Performance

The `many_tuple_assignments` microbenchmark increases from 141.1 ms to
159.7 ms in simulation and from 6.4 MB to 6.9 MB in memory. This is an
expected cost of preserving tuple shapes; the tradeoff is acceptable for
this change. See the [benchmark
report](https://github.com/astral-sh/ruff/pull/28617#issuecomment-5684963921).

## Test plan

- Mdtests cover fixed and empty tuples, variable-length prefixes and
suffixes, aliases, union operands, subclass overrides, and
`TypeVarTuple` concatenation with both typing syntaxes.
- Exercise ordinary and augmented concatenation, large unions with
custom operators, repeated doubling, and recursively growing local
variables and instance attributes.
- Preserve finite alternating tuple shapes and fixed-length tuple unions
supplied by an empty list's return context.
- Verify incompatible augmented assignment to `tuple[Any, Any]` and
instance-layout conflicts detected through concatenated `__slots__`.
- Unit tests cover tuple widening across cycle results, including equal
lengths, changing lengths, discarded alternatives, avoiding relation
queries, and `Never` elements.
2026-09-15 13:24:13 -07:00

6.5 KiB

Binary operations on tuples

Concatenation for heterogeneous tuples

Concatenating fixed-length tuples preserves the type and order of their elements. An empty tuple contributes no elements.

This inference is not strictly sound: a value annotated as tuple can be an instance of a subclass. We assume such subclasses do not override __add__ or __radd__ with behavior incompatible with ordinary tuple concatenation.

reveal_type((1, 2) + (3, 4))  # revealed: tuple[Literal[1], Literal[2], Literal[3], Literal[4]]
reveal_type(() + (1, 2))  # revealed: tuple[Literal[1], Literal[2]]
reveal_type((1, 2) + ())  # revealed: tuple[Literal[1], Literal[2]]
reveal_type(() + ())  # revealed: tuple[()]

def _(x: tuple[int, str], y: tuple[None, tuple[int]]):
    reveal_type(x + y)  # revealed: tuple[int, str, None, tuple[int]]
    reveal_type(y + x)  # revealed: tuple[None, tuple[int], int, str]

Concatenation for homogeneous tuples

Concatenating variable-length tuples combines their element types. Fixed-length operands preserve a known prefix or suffix; elements between two variable-length portions become part of the variable portion.

def _(x: tuple[int, ...], y: tuple[str, ...]):
    reveal_type(x + x)  # revealed: tuple[int, ...]
    reveal_type(x + y)  # revealed: tuple[int | str, ...]
    reveal_type((1, 2) + x)  # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...]]
    reveal_type(x + (3, 4))  # revealed: tuple[*tuple[int, ...], Literal[3], Literal[4]]
    reveal_type((1, 2) + x + (3, 4))  # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...], Literal[3], Literal[4]]
    reveal_type((1, 2) + y + (3, 4) + x)  # revealed: tuple[Literal[1], Literal[2], *tuple[int | str, ...]]

We get the same results even when we use a legacy type alias, even though this involves first inferring the tuple[...] expression as a value form. (Doing so gives a generic alias of the tuple type, but as a special case, we include the full detailed tuple element specification in specializations of tuple.)

from typing import Literal

OneTwo = tuple[Literal[1], Literal[2]]
ThreeFour = tuple[Literal[3], Literal[4]]
IntTuple = tuple[int, ...]
StrTuple = tuple[str, ...]

def _(one_two: OneTwo, x: IntTuple, y: StrTuple, three_four: ThreeFour):
    reveal_type(x + x)  # revealed: tuple[int, ...]
    reveal_type(x + y)  # revealed: tuple[int | str, ...]
    reveal_type(one_two + x)  # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...]]
    reveal_type(x + three_four)  # revealed: tuple[*tuple[int, ...], Literal[3], Literal[4]]
    reveal_type(one_two + x + three_four)  # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...], Literal[3], Literal[4]]
    reveal_type(one_two + y + three_four + x)  # revealed: tuple[Literal[1], Literal[2], *tuple[int | str, ...]]

Union operands

Concatenation preserves the possible lengths and element types of each tuple in a union.

def concatenate(left: tuple[int] | tuple[str, str], right: tuple[bytes]) -> None:
    reveal_type(left + right)  # revealed: tuple[int, bytes] | tuple[str, str, bytes]
    reveal_type(right + left)  # revealed: tuple[bytes, int] | tuple[bytes, str, str]

Tuple subclass overrides

An explicitly known tuple subclass can override addition. Its __add__ return annotation determines the result. On the right, its __radd__ can take precedence over the left operand's __add__; both results remain possible because a tuple annotation also admits subclass instances.

class Custom(tuple[int]):
    def __add__(self, other: object) -> tuple[int, int]:
        return (1, 2)

    def __radd__(self, other: object) -> str:
        return "custom"

def add_custom(custom: Custom, plain: tuple[str]) -> None:
    reveal_type(custom + plain)  # revealed: tuple[int, int]
    reveal_type(plain + custom)  # revealed: str | tuple[str | int, ...]

Many conditional concatenations

Independent conditions can produce exponentially many tuple shapes. For large combinations, concatenation approximates the result as a variable-length tuple to keep inference bounded. Other union alternatives still use their operator methods.

def flag() -> bool:
    return True

class Custom(tuple[str]):
    def __add__(self, other: object) -> "Custom":
        return self

def initial() -> Custom | tuple[()]:
    return ()

value = ()
mixed = initial()
if flag():
    value += (1,)
    mixed = mixed + (1,)
if flag():
    value += (2,)
    mixed = mixed + (2,)
if flag():
    value += (3,)
    mixed = mixed + (3,)
if flag():
    value += (4,)
    mixed = mixed + (4,)
if flag():
    value += (5,)
    mixed = mixed + (5,)
if flag():
    value += (6,)
    mixed = mixed + (6,)

reveal_type(value)  # revealed: tuple[Literal[6, 1, 2, 3, 4, 5], ...]
reveal_type(mixed)  # revealed: Custom | tuple[Literal[6, 1, 2, 3, 4, 5], ...]

Concatenating two large unions

Each operand has six tuple shapes, so concatenation can produce 36 combinations. Both ordinary and augmented assignment use a variable-length approximation when combining these alternatives.

Choice = (
    tuple[int, int, str]
    | tuple[int, str, int]
    | tuple[str, int, int]
    | tuple[int, str, str]
    | tuple[str, int, str]
    | tuple[str, str, int]
)

def concatenate(left: Choice, right: Choice) -> None:
    reveal_type(left + right)  # revealed: tuple[int | str, ...]
    result = left
    result += right
    reveal_type(result)  # revealed: tuple[int | str, ...]

Non-tuple alternatives still use their addition and reflected addition methods, even when the tuple combinations require approximation.

class Custom:
    def __add__(self, other: object) -> "Custom":
        return self

    def __radd__(self, other: object) -> bytes:
        return b"custom"

def concatenate_custom(left: Choice | Custom, right: Choice | Custom) -> None:
    reveal_type(left + right)  # revealed: bytes | tuple[int | str, ...] | Custom
    result = left
    result += right
    reveal_type(result)  # revealed: bytes | tuple[int | str, ...] | Custom

Repeated doubling

Doubling a tuple repeatedly grows its length exponentially without creating a union. Very large concatenations also use a variable-length approximation.

value = (1,)
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
reveal_type(value)  # revealed: tuple[Literal[1], ...]