error when (indirectly) importing mypy during package introspection

[zzzeek]

We run slotscheck on linux and it has begun intermittently core dumping on the SQLAlchemy source tree. Here's the core dump occurring on a clean checkout

[classic@zotac tmp]$ git clone https://github.com/sqlalchemy/sqlalchemy


Cloning into 'sqlalchemy'...
remote: Enumerating objects: 221912, done.
remote: Counting objects: 100% (6585/6585), done.
remote: Compressing objects: 100% (1719/1719), done.
remote: Total 221912 (delta 4306), reused 6123 (delta 4109), pack-reused 215327
Receiving objects: 100% (221912/221912), 100.43 MiB | 13.37 MiB/s, done.
Resolving deltas: 100% (166804/166804), done.
[classic@zotac tmp]$ python --version
Python 3.11.4

[classic@zotac tmp]$ python -m venv .venv
[classic@zotac tmp]$ .venv/bin/pip install slotscheck

Collecting slotscheck
  Using cached slotscheck-0.17.0-py3-none-any.whl (20 kB)
Collecting click<9.0,>=8.0
  Using cached click-8.1.6-py3-none-any.whl (97 kB)
Installing collected packages: click, slotscheck
Successfully installed click-8.1.6 slotscheck-0.17.0

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: python3.11 -m pip install --upgrade pip

[classic@zotac tmp]$ source .venv/bin/activate
(.venv) [classic@zotac sqlalchemy:main]$ pip install -e .

Obtaining file:///home/classic/tmp/sqlalchemy
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.2.0
  Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Collecting greenlet!=0.4.17
  Using cached greenlet-2.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (618 kB)
Building wheels for collected packages: SQLAlchemy
  Building editable for SQLAlchemy (pyproject.toml) ... done
  Created wheel for SQLAlchemy: filename=SQLAlchemy-2.0.20.dev0-0.editable-cp311-cp311-linux_x86_64.whl size=5469 sha256=05327af543c7001e3a33d444e161b16fef4d2b095468a448b10e60bdbc0d953a
  Stored in directory: /tmp/pip-ephem-wheel-cache-9i_xccfq/wheels/a8/7e/00/e10a0e6817e01d868585daad36d782018a2496f55f5ede1501
Successfully built SQLAlchemy
Installing collected packages: typing-extensions, greenlet, SQLAlchemy
Successfully installed SQLAlchemy-2.0.20.dev0 greenlet-2.0.2 typing-extensions-4.7.1

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip

(.venv) [classic@zotac sqlalchemy:main]$ pip install mypy
Collecting mypy
  Using cached mypy-1.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.0 MB)
Requirement already satisfied: typing-extensions>=4.1.0 in /home/classic/tmp/.venv/lib64/python3.11/site-packages (from mypy) (4.7.1)
Collecting mypy-extensions>=1.0.0
  Using cached mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Installing collected packages: mypy-extensions, mypy
Successfully installed mypy-1.5.0 mypy-extensions-1.0.0

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
All OK!
Scanned 219 module(s), 1647 class(es).
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
Segmentation fault (core dumped)

here are the steps without command output

[classic@zotac tmp]$ git clone https://github.com/sqlalchemy/sqlalchemy
[classic@zotac tmp]$ python -m venv .venv
[classic@zotac tmp]$ .venv/bin/pip install slotscheck
[classic@zotac tmp]$ source .venv/bin/activate
(.venv) [classic@zotac sqlalchemy:main]$ pip install -e .
(.venv) [classic@zotac sqlalchemy:main]$ pip install mypy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy
(.venv) [classic@zotac sqlalchemy:main]$ slotscheck -m sqlalchemy

[ariebovenberg]

Thanks for the report and reproducible steps 🙏 . This is indeed strange.

slotscheck is pure Python of course, but does lots of introspection (by necessity). My first thought is that some kind of introspection must have triggered something in particular extension modules 🤔.

I'll have a look 🔍

[zzzeek]

thanks, I looked to see if there was any debug mode for slotscheck to see perhaps what it was failing on. if you were able to reproduce locally that would be great.

[ariebovenberg]

I'm able to reproduce locally (MacOS) as well. My findings:

• The problem only occurs every so often because the introspection of modules happens in indeterminate order (discovered modules are de-duplicated using hashed collections like dict and set.)
• On my machine, running env PYTHONHASHSEED=0 slotscheck -m sqlalchemy always triggers the error case. PYTHONHASHSEED=2 gives no error.
• The problem appears to be when scanning the sqlalchemy.ext.mypy module. Skipping this module makes the error disappear
• On mypy 1.4, we get an ImportError instead of a segfault. It is likely things changed once mypy updated to 1.5. The error in 1.4: ImportError("cannot import name 'ALL_STRATEGY' from 'mypy.type_visitor' ([..]/sqlalchemy/.venv/lib/python3.11/site-packages/mypy/type_visitor.cpython-311-darwin.so)")
• A likely trigger would be an internal change to mypy's extension modules. Slotscheck probably needs to import mypy's modules because they are imported, but apparently does this in a way that crashes mypy.

I'll do some more digging later, but —for now— the more straightforward solution looks to be changing the entry in pyproject.toml:

[tool.slotscheck]
exclude-modules = '''
^sqlalchemy\.(
  testing
  |ext\.mypy  # see slotscheck/issues/178
)
'''

[ariebovenberg]

A good reason to bump priority on #116 as well. The 'troublesome' module is easily found when printing module names during introspection.

[ariebovenberg]

Below is the traceback (on mypy 1.4). The module under inspection here is sqlalchemy.ext.mypy.apply. Mypy classes themselves are not introspected. Somehow the (indirect) mypy import does fail under specific conditions. It doesn't fail when running slotscheck on only sqlalchemy.ext.mypy.apply

Traceback (most recent call last):
  File "/Users/arie/code/sqlalchemy/.venv/lib/python3.11/site-packages/slotscheck/discovery.py", line 236, in walk_classes
    module = importlib.import_module(fullname)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/arie/.pyenv/versions/3.11.4/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/arie/code/sqlalchemy/lib/sqlalchemy/ext/mypy/apply.py", line 28, in <module>
    from mypy.plugin import SemanticAnalyzerPluginInterface
  File "mypy/plugin.py", line 132, in <module>
    from mypy.messages import MessageBuilder
  File "mypy/messages.py", line 22, in <module>
    import mypy.typeops
  File "mypy/typeops.py", line 13, in <module>
    from mypy.copytype import copy_type
  File "mypy/copytype.py", line 5, in <module>
    from mypy.types import (
  File "mypy/types.py", line 2960, in <module>
    from mypy.type_visitor import (  # noqa: F811,F401
ImportError: cannot import name 'ALL_STRATEGY' from 'mypy.type_visitor' (/Users/arie/code/sqlalchemy/.venv/lib/python3.11/site-packages/mypy/type_visitor.cpython-311-darwin.so)

[zzzeek]

great idea, yeah, the mypy extension is not really a critical component for us and skipping it seems like the best approach for now

[ariebovenberg]

I've created a tracking issue for the underlying issue (see above). See there for further discussion.