Tuple concatenation currently discards element positions and tuple length, so adding two fixed-length tuples produces a variable-length tuple. Preserve the concatenated shape, including known prefixes and suffixes around variable-length portions. As with tuple slicing and iteration, this (unsoundly) assumes values annotated as `tuple` behave like ordinary tuples, even though a subclass could override `__add__` and/or `__radd__` incompatibly. Explicitly known subclasses continue to use their operator methods. Keep inference bounded by approximating large tuple combinations and widening tuple lengths when their alternatives change between cycle iterations. Stable tuple unions, including those supplied by collection type context, retain their shapes. Some cycles with finite but changing tuple lengths can still lose precision to ensure convergence. Fixes https://github.com/astral-sh/ty/issues/3413. ## Ecosystem impact - Removes tuple-shape false positives in Hydra-zen, Trio, Spark, and SymPy, and two unsoundness warnings in pandas. - Most other changes make existing diagnostics more precise. New errors expose overly narrow tuple attributes and collection element types in zope.interface and SymPy, including invariant dictionary key types. - One new xarray false positive exposes existing dictionary-key inference that incorrectly restricts keys to strings. This limitation is left for separate work. Cycle widening also loses finite tuple alternatives in Apprise, consolidating existing errors without changing which operations are rejected. [Full ecosystem analysis](https://github.com/astral-sh/ruff/pull/28617#issuecomment-5687202176). ## Performance The `many_tuple_assignments` microbenchmark increases from 141.1 ms to 159.7 ms in simulation and from 6.4 MB to 6.9 MB in memory. This is an expected cost of preserving tuple shapes; the tradeoff is acceptable for this change. See the [benchmark report](https://github.com/astral-sh/ruff/pull/28617#issuecomment-5684963921). ## Test plan - Mdtests cover fixed and empty tuples, variable-length prefixes and suffixes, aliases, union operands, subclass overrides, and `TypeVarTuple` concatenation with both typing syntaxes. - Exercise ordinary and augmented concatenation, large unions with custom operators, repeated doubling, and recursively growing local variables and instance attributes. - Preserve finite alternating tuple shapes and fixed-length tuple unions supplied by an empty list's return context. - Verify incompatible augmented assignment to `tuple[Any, Any]` and instance-layout conflicts detected through concatenated `__slots__`. - Unit tests cover tuple widening across cycle results, including equal lengths, changing lengths, discarded alternatives, avoiding relation queries, and `Never` elements.
6.5 KiB
Binary operations on tuples
Concatenation for heterogeneous tuples
Concatenating fixed-length tuples preserves the type and order of their elements. An empty tuple contributes no elements.
This inference is not strictly sound: a value annotated as tuple can be an instance of a subclass.
We assume such subclasses do not override __add__ or __radd__ with behavior incompatible with
ordinary tuple concatenation.
reveal_type((1, 2) + (3, 4)) # revealed: tuple[Literal[1], Literal[2], Literal[3], Literal[4]]
reveal_type(() + (1, 2)) # revealed: tuple[Literal[1], Literal[2]]
reveal_type((1, 2) + ()) # revealed: tuple[Literal[1], Literal[2]]
reveal_type(() + ()) # revealed: tuple[()]
def _(x: tuple[int, str], y: tuple[None, tuple[int]]):
reveal_type(x + y) # revealed: tuple[int, str, None, tuple[int]]
reveal_type(y + x) # revealed: tuple[None, tuple[int], int, str]
Concatenation for homogeneous tuples
Concatenating variable-length tuples combines their element types. Fixed-length operands preserve a known prefix or suffix; elements between two variable-length portions become part of the variable portion.
def _(x: tuple[int, ...], y: tuple[str, ...]):
reveal_type(x + x) # revealed: tuple[int, ...]
reveal_type(x + y) # revealed: tuple[int | str, ...]
reveal_type((1, 2) + x) # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...]]
reveal_type(x + (3, 4)) # revealed: tuple[*tuple[int, ...], Literal[3], Literal[4]]
reveal_type((1, 2) + x + (3, 4)) # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...], Literal[3], Literal[4]]
reveal_type((1, 2) + y + (3, 4) + x) # revealed: tuple[Literal[1], Literal[2], *tuple[int | str, ...]]
We get the same results even when we use a legacy type alias, even though this involves first
inferring the tuple[...] expression as a value form. (Doing so gives a generic alias of the
tuple type, but as a special case, we include the full detailed tuple element specification in
specializations of tuple.)
from typing import Literal
OneTwo = tuple[Literal[1], Literal[2]]
ThreeFour = tuple[Literal[3], Literal[4]]
IntTuple = tuple[int, ...]
StrTuple = tuple[str, ...]
def _(one_two: OneTwo, x: IntTuple, y: StrTuple, three_four: ThreeFour):
reveal_type(x + x) # revealed: tuple[int, ...]
reveal_type(x + y) # revealed: tuple[int | str, ...]
reveal_type(one_two + x) # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...]]
reveal_type(x + three_four) # revealed: tuple[*tuple[int, ...], Literal[3], Literal[4]]
reveal_type(one_two + x + three_four) # revealed: tuple[Literal[1], Literal[2], *tuple[int, ...], Literal[3], Literal[4]]
reveal_type(one_two + y + three_four + x) # revealed: tuple[Literal[1], Literal[2], *tuple[int | str, ...]]
Union operands
Concatenation preserves the possible lengths and element types of each tuple in a union.
def concatenate(left: tuple[int] | tuple[str, str], right: tuple[bytes]) -> None:
reveal_type(left + right) # revealed: tuple[int, bytes] | tuple[str, str, bytes]
reveal_type(right + left) # revealed: tuple[bytes, int] | tuple[bytes, str, str]
Tuple subclass overrides
An explicitly known tuple subclass can override addition. Its __add__ return annotation determines
the result. On the right, its __radd__ can take precedence over the left operand's __add__; both
results remain possible because a tuple annotation also admits subclass instances.
class Custom(tuple[int]):
def __add__(self, other: object) -> tuple[int, int]:
return (1, 2)
def __radd__(self, other: object) -> str:
return "custom"
def add_custom(custom: Custom, plain: tuple[str]) -> None:
reveal_type(custom + plain) # revealed: tuple[int, int]
reveal_type(plain + custom) # revealed: str | tuple[str | int, ...]
Many conditional concatenations
Independent conditions can produce exponentially many tuple shapes. For large combinations, concatenation approximates the result as a variable-length tuple to keep inference bounded. Other union alternatives still use their operator methods.
def flag() -> bool:
return True
class Custom(tuple[str]):
def __add__(self, other: object) -> "Custom":
return self
def initial() -> Custom | tuple[()]:
return ()
value = ()
mixed = initial()
if flag():
value += (1,)
mixed = mixed + (1,)
if flag():
value += (2,)
mixed = mixed + (2,)
if flag():
value += (3,)
mixed = mixed + (3,)
if flag():
value += (4,)
mixed = mixed + (4,)
if flag():
value += (5,)
mixed = mixed + (5,)
if flag():
value += (6,)
mixed = mixed + (6,)
reveal_type(value) # revealed: tuple[Literal[6, 1, 2, 3, 4, 5], ...]
reveal_type(mixed) # revealed: Custom | tuple[Literal[6, 1, 2, 3, 4, 5], ...]
Concatenating two large unions
Each operand has six tuple shapes, so concatenation can produce 36 combinations. Both ordinary and augmented assignment use a variable-length approximation when combining these alternatives.
Choice = (
tuple[int, int, str]
| tuple[int, str, int]
| tuple[str, int, int]
| tuple[int, str, str]
| tuple[str, int, str]
| tuple[str, str, int]
)
def concatenate(left: Choice, right: Choice) -> None:
reveal_type(left + right) # revealed: tuple[int | str, ...]
result = left
result += right
reveal_type(result) # revealed: tuple[int | str, ...]
Non-tuple alternatives still use their addition and reflected addition methods, even when the tuple combinations require approximation.
class Custom:
def __add__(self, other: object) -> "Custom":
return self
def __radd__(self, other: object) -> bytes:
return b"custom"
def concatenate_custom(left: Choice | Custom, right: Choice | Custom) -> None:
reveal_type(left + right) # revealed: bytes | tuple[int | str, ...] | Custom
result = left
result += right
reveal_type(result) # revealed: bytes | tuple[int | str, ...] | Custom
Repeated doubling
Doubling a tuple repeatedly grows its length exponentially without creating a union. Very large concatenations also use a variable-length approximation.
value = (1,)
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
value = value + value
reveal_type(value) # revealed: tuple[Literal[1], ...]