Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable 'distutils imported before setuptools' check on PyPy #2868

Closed
jaraco opened this issue Nov 10, 2021 · 0 comments
Closed

Disable 'distutils imported before setuptools' check on PyPy #2868

jaraco opened this issue Nov 10, 2021 · 0 comments

Comments

@jaraco
Copy link
Member

jaraco commented Nov 10, 2021

It does look like PyPy is also, separately, importing distutils early, breaking the local tests.

In this case, it appears to be pytest-virtualenv again:

setuptools github_workflows_SETUPTOOLS_USE_DISTUTILS $ .tox/pypy/bin/python
Python 3.7.12 (44db26267d0a38e51a7e8490983ed7e7bcb84b74, Oct 28 2021, 14:16:15)
[PyPy 7.3.7 with GCC Apple LLVM 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys
>>>> import importlib_metadata as md
>>>> for ep in md.entry_points(group='pytest11'):
....   ep.load()
....   print({'distutils'} & set(sys.modules))
....   
<module 'pytest_flake8' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/pytest_flake8.py'>
set()
<module 'pytest_enabler' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/pytest_enabler/__init__.py'>
set()
<module 'pytest_checkdocs' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/pytest_checkdocs/__init__.py'>
set()
<module 'pytest_shutil.workspace' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/pytest_shutil/workspace.py'>
set()
<module 'xdist.plugin' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/xdist/plugin.py'>
set()
<module 'xdist.looponfail' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/xdist/looponfail.py'>
set()
<module 'pytest_forked' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/pytest_forked/__init__.py'>
set()
<module 'pytest_virtualenv' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/pytest_virtualenv.py'>
{'distutils'}
<module 'pytest_cov.plugin' from '/Users/jaraco/code/public/pypa/setuptools/.tox/pypy/site-packages/pytest_cov/plugin.py'>
{'distutils'}

It seems that on PyPy, from pkg_resources import working_set is causing distutils to get imported and from the stdlib, even though SETUPTOOLS_USE_DISTUTILS='local'.

Oh, right. Now I remember. PyPy imports distutils in sysconfig:

setuptools github_workflows_SETUPTOOLS_USE_DISTUTILS $ touch distutils/__init__.py
setuptools github_workflows_SETUPTOOLS_USE_DISTUTILS $ $PYTHONPATH='.' ./.tox/pypy/bin/python
Python 3.7.12 (44db26267d0a38e51a7e8490983ed7e7bcb84b74, Oct 28 2021, 14:16:15)
[PyPy 7.3.7 with GCC Apple LLVM 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import pkg_resources
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 950, in <module>
    class Environment:
  File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 954, in Environment
    self, search_path=None, platform=get_supported_platform(),
  File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 180, in get_supported_platform
    plat = get_build_platform()
  File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 385, in get_build_platform
    plat = get_platform()
  File "/usr/local/Cellar/pypy3/7.3.7/libexec/lib-python/3/sysconfig.py", line 728, in get_platform
    get_config_vars(),
  File "/usr/local/Cellar/pypy3/7.3.7/libexec/lib-python/3/sysconfig.py", line 603, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/usr/local/Cellar/pypy3/7.3.7/libexec/lib-python/3/sysconfig.py", line 467, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
  File "/usr/local/Cellar/pypy3/7.3.7/libexec/lib_pypy/_sysconfigdata.py", line 5, in <module>
    from distutils.spawn import find_executable
ModuleNotFoundError: No module named 'distutils.spawn'

So it's not possible on PyPy to import pkg_resources without importing distutils.

Maybe pytest-virtualenv can be made to work without that import as well.

In man-group/pytest-plugins@c08c4a8, I removed the import of pkg_resources and verified that with that change, one can load all of the pytest plugins without importing distutils. Unfortunately, the error still occurs, indicating that something else is probably importing pkg_resources before the test suite is loaded.

Originally posted by @jaraco in #2865 (comment)

jaraco added a commit to mkoeppe/setuptools that referenced this issue Nov 10, 2021
clrpackages pushed a commit to clearlinux-pkgs/setuptools that referenced this issue Nov 16, 2021
…on 59.0.1

Anderson Bravalheri (16):
      Add logo design and usage instructions
      Customise the sphinx theme to reflect the new logo
      Add banner to main docs page
      Rename logo files
      Add docs/README to the toctree
      Add a note about inspiration
      Use warning instead of log for distutils command
      Add news fragment for deprecated distuils commands
      Expose distutils errors
      Document setuptools replacements
      Add logo/banner to README
      Add logo/banner to README
      Add favicon and related configuration to sphinx
      Improve favicon
      Avoid using dependencies with URLs on setup.cfg
      Add news fragment

Isuru Fernando (1):
      Fix PyPy3.8 install locations

Jason R. Coombs (18):
      Prefer imperative voice.
      Use simple expression for resolution of outputdir.
      Add changelog.
      Revert ".github/workflows/main.yml: Add integration testing with setuptools"
      Get pytest-virtualenv from the pull request with distutils dependency removed.
      Fail fast again.
      Filter known deprecation warnings when stdlib distutils is used and imports sysconfig.
      Suppress 'Distutils was imported before Setuptools' warning on PyPy due to pypa/setuptools#2868.
      Move warning suppression to pytest.ini so it occurs after 'error'.
      Remove superfluous dep.
      Disable some virtualenv integration tests for PyPy.
      Avoid upgrading setuptools when creating the venv for distutils_adoption tests. Works around issue with upgrading on PyPy.
      Bring back the xfail as it appears to be needed at least on macOS.
      Skip pip@main tests for Python 3.6, no longer supported. Fixes #2874. Use pip pinned range to resolve pip version based on a repeatable pattern.
      Restore single_line as a simple, universal validator.
      Change is backward-incompatible for projects with newlines in the summary.
      Bump version: 58.5.3 → 59.0.0
      Bump version: 59.0.0 → 59.0.1

Kian-Meng, Ang (1):
      Fix typos

Matthias Koeppe (8):
      distutils/ccompiler.py (CCompiler.has_function): Do not fail if self.outputdir is set
      distutils/tests/test_unixccompiler.py (UnixCCompilerTestCase.test_has_function): New
      .github/workflows/main.yml: Add integration testing with setuptools
      tox.ini (testenv): passenv SETUPTOOLS_USE_DISTUTILS
      .github/workflows/main.yml: Test SETUPTOOLS_USE_DISTUTILS = stdlib, local
      tox.ini (testenv): Add deps = pytest
      tox.ini (testenv): Make sure to import setuptools before distutils when invoking pytest
      .github/workflows/main.yml: fail-fast = false

Sviatoslav Sydorenko (6):
      Fail on a multiline distribution package summary
      Document that pip works with editable setup.cfg
      Add a change note for PR #2870
      Add a change note for PR #2871
      Fix the word "breaks" in the change note
      Improve the mention of pip version that doesn't need `setup.py`
@jaraco jaraco closed this as completed Nov 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant