Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash on numtype numpy stubs package related to property #18764

Closed
hauntsaninja opened this issue Mar 7, 2025 · 4 comments · Fixed by #18774
Closed

Crash on numtype numpy stubs package related to property #18764

hauntsaninja opened this issue Mar 7, 2025 · 4 comments · Fixed by #18774
Labels

Comments

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Mar 7, 2025

rm -rf tmp
mkdir tmp
cd tmp
git clone https://github.com/numpy/numtype.git
mypy numtype/src --show-traceback 

numtype commit 864cda2

Seems similar to #18648 cc @ilevkivskyi

...
  File "/Users/shantanu/dev/mypy/mypy/nodes.py", line 580, in accept
    return visitor.visit_overloaded_func_def(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/shantanu/dev/mypy/mypy/checker.py", line 641, in visit_overloaded_func_def
    self._visit_overloaded_func_def(defn)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/shantanu/dev/mypy/mypy/checker.py", line 697, in _visit_overloaded_func_def
    found_method_base_classes = self.check_method_override(defn)
  File "/Users/shantanu/dev/mypy/mypy/checker.py", line 2022, in check_method_override
    result = self.check_method_or_accessor_override_for_base(
        defn, base, check_override_compatibility
    )
  File "/Users/shantanu/dev/mypy/mypy/checker.py", line 2060, in check_method_or_accessor_override_for_base
    if self.check_method_override_for_base_with_name(defn, name, base):
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/Users/shantanu/dev/mypy/mypy/checker.py", line 2194, in check_method_override_for_base_with_name
    self.check_setter_type_override(defn, base_attr, base)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shantanu/dev/mypy/mypy/checker.py", line 2084, in check_setter_type_override
    original_type, is_original_setter = get_raw_setter_type(base_node)
                                        ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/Users/shantanu/dev/mypy/mypy/checker.py", line 8919, in get_raw_setter_type
    assert var.type is not None
           ^^^^^^^^^^^^^^^^^^^^
AssertionError: 
numtype/src/numpy-stubs/ma/core.pyi:308: : note: use --pdb to drop into pdb
@hauntsaninja
Copy link
Collaborator Author

hauntsaninja commented Mar 7, 2025

Partially minimised repro:

rm -rf repro
mkdir repro
cd repro

python -m venv env
env/bin/python -m pip install numpy

mkdir numpy-stubs

echo '
from typing import Any, Generic
from typing_extensions import TypeVar

from . import matlib as matlib  # load bearing since it imports numpy

_DTypeT2_co = TypeVar("_DTypeT2_co", covariant=True)
_ShapeT2 = TypeVar("_ShapeT2", bound=tuple[int, ...])
_ShapeT2_co = TypeVar("_ShapeT2_co", bound=tuple[int, ...], covariant=True)
_ScalarT2_co = TypeVar("_ScalarT2_co", covariant=True)

class ndarray(Generic[_ShapeT2_co, _DTypeT2_co]):
    @property
    def shape(self) -> _ShapeT2_co: ...
    @shape.setter
    def shape(self: ndarray[_ShapeT2, Any], shape: _ShapeT2, /) -> None: ...
' > numpy-stubs/__init__.pyi

echo '
from typing_extensions import TypeVar

import numpy as np

_ShapeT = TypeVar("_ShapeT", bound=tuple[int, ...])
_ShapeT_co = TypeVar("_ShapeT_co", bound=tuple[int, ...], default=tuple[int, ...], covariant=True)
_DTypeT_co = TypeVar("_DTypeT_co", covariant=True)

class MaskedArray(np.ndarray[_ShapeT_co, _DTypeT_co]):
    @property
    def shape(self) -> _ShapeT_co: ...
    @shape.setter
    def shape(self: MaskedArray[_ShapeT], shape: _ShapeT, /) -> None: ...
' > numpy-stubs/core.pyi

mypy . --python-executable env/bin/python --show-traceback

@ilevkivskyi
Copy link
Member

Oh well, I was wandering why some code paths (inconsistently) expose settable property type as Overloaded(...) with a single item. Ultimately, this is just another case of the checkmember.py mega-issue. I am still not sure what to do about this particular case: another band-aid, or try cleaning this up for real. Taking into account the 1.16 is coming soon it will likely be the former, but I want to think a bit more, before I do this.

@ilevkivskyi
Copy link
Member

@hauntsaninja could you please check that #18774 fixes the original crash? (I have a very minimalist example in the tests.)

ilevkivskyi added a commit that referenced this issue Mar 10, 2025
Fixes #18764

There are two things important to understand this PR:
* First, mypy has a performance optimization - we store
decorator/overload type during semantic analysis in certain "trivial"
situations (to avoid deferrals). Otherwise, we infer the type of
decorated function or an overload variant during type checking.
* Second, for settable properties we store getter type in two places, as
a `Var.type` of getter (as a decorator), and also in overall
`OverloadedFuncDef.type`. The latter is ugly, but unfortunately it is
hard to get rid of, since some code in multiple plugins rely on this.

It turns out there are _three_ inconsistencies in how these two things
interact (first one causes the actual crash):
* For trivial settable properties (i.e. without extra decorators) when
we store the type in `semanal.py` we only store it the second way (i.e.
as `OverloadedFuncDef.type`).
* For non-trivial settable properties (where getter and/or setter are
themselves decorated), we only set the inferred type the first way (as
`Var.type`).
* When inferring setter type (unlike getter, that is handled correctly)
we actually ignore any extra decorators (this is probably quire niche,
but still inconsistent).

Essentially I simply remove these inconsistencies.
@jorenham
Copy link

jorenham commented Mar 10, 2025

I can confirm that latest mypy main (af5186e) now indeed runs without crashing on numtype (8598dc9): Thanks, @ilevkivskyi and @hauntsaninja!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants