Allow variable redefinition if lifetimes don't overlap

[JukkaL]

As a follow-up to #1174 and #6197, we should perhaps allow two assignments to create independent variables whenever the lifetimes of the variables don't overlap (when using --allow-redefinitions).

Example:

if foo():
    x = 0
    print(x)
else:
    x = 'a'  # Define a new variable?
    print(x)

Another example:

for x in [1, 2]:
    print(x)
if foo():
    for x in ['a', 'b']:  # Define a new variable?
        print(x)

[ShadowLNC]

Not sure if this would need to be split into a separate issue, or if it is relevant here, but I have some code like this:

try:
    ...
except AssertionError as e:
    errors.append(e)

if condition:
    e = AssertionError("message")
    errors.append(e)

Essentially the same as Django's ValidationError pattern. mypy warns for the assignment to variable e outside the except block, and also the reading of the deleted variable, despite the fact that I've just assigned it.

[dumblob]

This is exactly what I see as a common pattern in code - quite often due to for loops or exceptions - exactly as in the above examples. So I'm also raising my hand for prioritizing this 😉.

[kshpytsya]

My case, which as far as I can tell is related to this issue, is with the following.

I am (ab)using context managers in the following pattern, which is used in some configuration building code written by end-users of my tool.

Heavily simplified:

toolmodule.py:

import contextlib
import typing

class A:
   a: int
   b: str

def get_A(name: str) -> typing.Generator[A, None, None]:
...

class B:
   c: bool

def get_B(name: str) -> typing.Generator[B, None, None]:
...

userwritten.py:

import toolmodule

with get_A("") as x:
  x.a = 1
  x.b = ""

with get_B("") as x:
  x.c = True

for i in [1]:
  with get_A("") as x: # <- error: Incompatible types in assignment (expression has type "A", variable has type "B")
    x.a = 1

(In actual code, get_X return special context managers returning accessor classes which fail in runtime once the scope of with block is exited.)

I suspected that there could be some nuances related to context managers, but the following code has the same problem:

import typing as tp


x = 1
if tp.TYPE_CHECKING:
    reveal_type(x)
print(x)


x = [1]
if tp.TYPE_CHECKING:
    reveal_type(x)
print(x)


if True:
    x = ""  # <- error: Incompatible types in assignment (expression has type "str", variable has type "List[int]")
    if tp.TYPE_CHECKING:
        reveal_type(x)
    print(x)

for i in [1]:
    x = ""  # <- error: Incompatible types in assignment (expression has type "str", variable has type "List[int]")
    if tp.TYPE_CHECKING:
        reveal_type(x)
    print(x)

Rhetorically and off-topic: I really wish Python had some scoping statement, akin to:

scoped:
  x = 1

x  # <- error: undefined

scoped for i in []:
   ...

i  # <- error: undefined

scoped with f() as y:
   ...

y  # <- error: undefined

[DevilXD]

Is there any progress on this? Having type "redefinition" errors, just because I used a variable that's named the same as the one I used in a for loop couple of lines earlier, is getting quite tiresome. The flag that allows redefinitions doesn't sound that great for my usage case either.

IMO, if someone is using a variable in a for loop, it should redefine the type of the variable, even without the "allow redefinitions" flag - consider this:

my_list: List[int]
my_other_list: List[Thing]

for x in my_list:
    reveal_type(x)  # int

reveal_type(x)  # int (still, persistent)

for x in my_other_list:  # redefines type of x
    reveal_type(x)  # Thing

x = "test"  # str (redefines type of x as well)

Perhaps the variable could be tagged as "reassignable" internally by MyPy somehow? This would also handle @ShadowLNC's case.

[arseniiv]

Can this be a related benign redefinition case too:

from pathlib import Path
for x in ['a/b']:
    reveal_type(x) # str, ok
    x = Path(x) # can’t redefine
    reveal_type(x) # str instead of Path
    reveal_type(x.stem) # error accessing, and Any because of that

With no for it works as expected, but in this manner it suddenly doesn’t (mypy 1.2, Python 3.11). I can’t see why a single iteration of a loop is different from an isolated block, as at the start of each iteration, x is reassigned a guaranteed str instance (if we know that what’s iterated over is Iterable[str])—but maybe there are pitfalls.

[filips123]

I'm not sure if this is the same issue, but I have a similar problem... I have the following code:

def get_menus(date: datetime.date) -> dict[str, Optional[dict[str, str]]]:
    # SnackMenu and LunchMenu are SQLAlchemy 2.0 models
    snack = Session.query(SnackMenu).filter(SnackMenu.date == date).first()
    lunch = Session.query(LunchMenu).filter(LunchMenu.date == date).first()

    reveal_type(snack)  # Revealed type is "Union[gimvicurnik.database.SnackMenu, None]"
    reveal_type(lunch)  # Revealed type is "Union[gimvicurnik.database.LunchMenu, None]"

    if snack:
        snack = {
            "normal": snack.normal,
            "poultry": snack.poultry,
            "vegetarian": snack.vegetarian,
            "fruitvegetable": snack.fruitvegetable,
        }

    if lunch:
        lunch = {
            "normal": lunch.normal,
            "vegetarian": lunch.vegetarian,
        }

    return {
        "snack": snack,
        "lunch": lunch,
    }

When running mypy with the following configuration (notice that allow_redefinition is enabled):

[tool.mypy]
python_version = 3.9
show_column_numbers = true
show_error_codes = true
allow_redefinition = true
disallow_untyped_decorators = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_configs = true
warn_unused_ignores = true
warn_no_return = true
warn_return_any = true
warn_unreachable = true

I get the following type errors:

gimvicurnik\blueprints\menus.py:29:25: error: Incompatible types in assignment (expression has type "Dict[str, Optional[str]]", variable has type "Optional[SnackMenu]")  [assignment]
gimvicurnik\blueprints\menus.py:37:25: error: Incompatible types in assignment (expression has type "Dict[str, Optional[str]]", variable has type "Optional[LunchMenu]")  [assignment]
gimvicurnik\blueprints\menus.py:43:17: error: Dict entry 0 has incompatible type "str": "Optional[SnackMenu]"; expected "str": "Optional[Dict[str, str]]"  [dict-item]
gimvicurnik\blueprints\menus.py:44:17: error: Dict entry 1 has incompatible type "str": "Optional[LunchMenu]"; expected "str": "Optional[Dict[str, str]]"  [dict-item]

I would expect that mypy would not reject this code, especially since allow_redefinition is enabled.