pyasn1 and pyasn1-modules pre-release package versions being installed

[katiyuki]

I have a process that uses the following packages, which has a dependency on pyasn1 and pyasn1-modules:

• PyDrive
• ldap3
• google-auth
• google-api-python-client

and on August 7th, 2022, this process worked fine. But when this process ran on August 9th, the process errored out. From the logs, it looks like it is pulling a pre-release version of pyasn1 and pyasn1-modules, for some reason - it shouldn't do that by default, right? is it because of the transition of ownership? Is there any way to fix this, other than to (possibly) explicitly set the version of these packages in my requirements.txt file? Any thoughts on this would be appreciated.

The following is the output from my jenkins build:

`+ pip3 install --upgrade pip
Collecting pip
Downloading pip-22.2.2-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.2.2
Uninstalling pip-20.2.2:
Successfully uninstalled pip-20.2.2
Successfully installed pip-22.2.2

• python3 setup.py develop
  running develop
  running egg_info
  creating archiveusers.egg-info
  writing archiveusers.egg-info/PKG-INFO
  writing dependency_links to archiveusers.egg-info/dependency_links.txt
  writing entry points to archiveusers.egg-info/entry_points.txt
  writing requirements to archiveusers.egg-info/requires.txt
  writing top-level names to archiveusers.egg-info/top_level.txt
  writing manifest file 'archiveusers.egg-info/SOURCES.txt'
  reading manifest file 'archiveusers.egg-info/SOURCES.txt'
  writing manifest file 'archiveusers.egg-info/SOURCES.txt'
  running build_ext
  Creating /usr/local/lib/python3.7/site-packages/archiveusers.egg-link (link to .)
  Adding archiveusers 1.0 to easy-install.pth file
  Installing archiveuser script to /usr/local/bin

Installed /workspace/BuildArchives
Processing dependencies for archiveusers==1.0
Searching for ldap3<2.6.0,>=2.5.1
Reading https://pypi.org/simple/ldap3/
Downloading https://files.pythonhosted.org/packages/54/93/55d4de43abf3af1b60befa648811538b49701237e5cbda456608384b25ac/ldap3-2.5.2-py2.py3-none-any.whl#sha256=dd9be8ea27773c4ffc18ede0b95c3ca1eb12513a184590b9f8ae423db3f71eb9
Best match: ldap3 2.5.2
Processing ldap3-2.5.2-py2.py3-none-any.whl
Installing ldap3-2.5.2-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding ldap3 2.5.2 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg
Searching for PyDrive==1.3.1
Reading https://pypi.org/simple/PyDrive/
Downloading https://files.pythonhosted.org/packages/52/e0/0e64788e5dd58ce2d6934549676243dc69d982f198524be9b99e9c2a4fd5/PyDrive-1.3.1.tar.gz#sha256=83890dcc2278081c6e3f6a8da1f8083e25de0bcc8eb7c91374908c5549a20787
Best match: PyDrive 1.3.1
Processing PyDrive-1.3.1.tar.gz
Writing /tmp/easy_install-nvvorrzo/PyDrive-1.3.1/setup.cfg
Running PyDrive-1.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-nvvorrzo/PyDrive-1.3.1/egg-dist-tmp-2y7_ljqu
warning: no previously-included files matching '.py[co]' found under directory ''
zip_safe flag not set; analyzing archive contents...
Moving PyDrive-1.3.1-py3.7.egg to /usr/local/lib/python3.7/site-packages
Adding PyDrive 1.3.1 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/PyDrive-1.3.1-py3.7.egg
Searching for Click<7.1,>=7.0
Reading https://pypi.org/simple/Click/
Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl#sha256=2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13
Best match: Click 7.0
Processing Click-7.0-py2.py3-none-any.whl
Installing Click-7.0-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding Click 7.0 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/Click-7.0-py3.7.egg
Searching for google-auth==1.12.0
Reading https://pypi.org/simple/google-auth/
Downloading https://files.pythonhosted.org/packages/05/b0/cc391ebf8ebf7855cdcfe0a9a4cdc8dcd90287c90e1ac22651d104ac6481/google_auth-1.12.0-py2.py3-none-any.whl#sha256=01d686448f57d3bc027726474faa1aa650ba333bedb392e06938b0add8ec8d3a
Best match: google-auth 1.12.0
Processing google_auth-1.12.0-py2.py3-none-any.whl
Installing google_auth-1.12.0-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding google-auth 1.12.0 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/google_auth-1.12.0-py3.7.egg
Searching for httplib2==0.15.0
Reading https://pypi.org/simple/httplib2/
Downloading https://files.pythonhosted.org/packages/be/83/5e006e25403871ffbbf587c7aa4650158c947d46e89f2d50dcaf018464de/httplib2-0.15.0-py3-none-any.whl#sha256=1d1f4ad7a6e55d325830ab274190f98894e069850a871fac19921caf4363259d
Best match: httplib2 0.15.0
Processing httplib2-0.15.0-py3-none-any.whl
Installing httplib2-0.15.0-py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding httplib2 0.15.0 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/httplib2-0.15.0-py3.7.egg
Searching for google-api-python-client==1.7.8
Reading https://pypi.org/simple/google-api-python-client/
Downloading https://files.pythonhosted.org/packages/55/e9/e8fb2e3a031cb69b9524b80a92b126665d9a17421700a219555e3233ab6a/google_api_python_client-1.7.8-py3-none-any.whl#sha256=937eabdc3940977f712fa648a096a5142766b6d0a0f58bc603e2ac0687397ef0
Best match: google-api-python-client 1.7.8
Processing google_api_python_client-1.7.8-py3-none-any.whl
Installing google_api_python_client-1.7.8-py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding google-api-python-client 1.7.8 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/google_api_python_client-1.7.8-py3.7.egg
Searching for pyasn1>=0.1.8
Reading https://pypi.org/simple/pyasn1/
Downloading https://files.pythonhosted.org/packages/b7/2a/3bc29f4de889beac9adcc166da5ba35c4c9bbe3e0e5ba645d868d398444d/pyasn1-0.5.0rc1-py2.py3-none-any.whl#sha256=8106f14b6a54088ac4f94faca2996fa3dfc5a0e602b3b3a17001f57446fe6a6c
Best match: pyasn1 0.5.0rc1
Processing pyasn1-0.5.0rc1-py2.py3-none-any.whl
Installing pyasn1-0.5.0rc1-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding pyasn1 0.5.0rc1 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/pyasn1-0.5.0rc1-py3.7.egg
Searching for PyYAML>=3.0
Reading https://pypi.org/simple/PyYAML/
Downloading https://files.pythonhosted.org/packages/eb/5f/6e6fe6904e1a9c67bc2ca5629a69e7a5a0b17f079da838bab98a1e548b25/PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl#sha256=231710d57adfd809ef5d34183b8ed1eeae3f76459c18fb4a0b373ad56bedcdd9
Best match: PyYAML 6.0
Processing PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Installing PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl to /usr/local/lib/python3.7/site-packages
Adding PyYAML 6.0 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/PyYAML-6.0-py3.7-linux-x86_64.egg
Searching for oauth2client>=4.0.0
Reading https://pypi.org/simple/oauth2client/
Downloading https://files.pythonhosted.org/packages/95/a9/4f25a14d23f0786b64875b91784607c2277eff25d48f915e39ff0cff505a/oauth2client-4.1.3-py2.py3-none-any.whl#sha256=b8a81cc5d60e2d364f0b1b98f958dbd472887acaf1a5b05e21c28c31a2d6d3ac
Best match: oauth2client 4.1.3
Processing oauth2client-4.1.3-py2.py3-none-any.whl
Installing oauth2client-4.1.3-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding oauth2client 4.1.3 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/oauth2client-4.1.3-py3.7.egg
Searching for six>=1.9.0
Reading https://pypi.org/simple/six/
Downloading https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl#sha256=8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254
Best match: six 1.16.0
Processing six-1.16.0-py2.py3-none-any.whl
Installing six-1.16.0-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding six 1.16.0 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/six-1.16.0-py3.7.egg
Searching for rsa<4.1,>=3.1.4
Reading https://pypi.org/simple/rsa/
Downloading https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl#sha256=14ba45700ff1ec9eeb206a2ce76b32814958a98e372006c8fb76ba820211be66
Best match: rsa 4.0
Processing rsa-4.0-py2.py3-none-any.whl
Installing rsa-4.0-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding rsa 4.0 to easy-install.pth file
Installing pyrsa-decrypt script to /usr/local/bin
Installing pyrsa-encrypt script to /usr/local/bin
Installing pyrsa-keygen script to /usr/local/bin
Installing pyrsa-priv2pub script to /usr/local/bin
Installing pyrsa-sign script to /usr/local/bin
Installing pyrsa-verify script to /usr/local/bin

Installed /usr/local/lib/python3.7/site-packages/rsa-4.0-py3.7.egg
Searching for pyasn1-modules>=0.2.1
Reading https://pypi.org/simple/pyasn1-modules/
Downloading https://files.pythonhosted.org/packages/b1/2c/490a9120015d54f50bb7c0717ac34d75f69acfb0918600350dcf087a20d3/pyasn1_modules-0.3.0rc1-py2.py3-none-any.whl#sha256=cbc7a0f2cd7bb1d54541be21410c84ec76ae4b504f5d8f6e5fc412f04981f806
Best match: pyasn1-modules 0.3.0rc1
Processing pyasn1_modules-0.3.0rc1-py2.py3-none-any.whl
Installing pyasn1_modules-0.3.0rc1-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding pyasn1-modules 0.3.0rc1 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/pyasn1_modules-0.3.0rc1-py3.7.egg
Searching for cachetools<5.0,>=2.0.0
Reading https://pypi.org/simple/cachetools/
Downloading https://files.pythonhosted.org/packages/ea/c1/4740af52db75e6dbdd57fc7e9478439815bbac549c1c05881be27d19a17d/cachetools-4.2.4-py3-none-any.whl#sha256=92971d3cb7d2a97efff7c7bb1657f21a8f5fb309a37530537c71b1774189f2d1
Best match: cachetools 4.2.4
Processing cachetools-4.2.4-py3-none-any.whl
Installing cachetools-4.2.4-py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding cachetools 4.2.4 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/cachetools-4.2.4-py3.7.egg
Searching for uritemplate<4dev,>=3.0.0
Reading https://pypi.org/simple/uritemplate/
Downloading https://files.pythonhosted.org/packages/bf/0c/60d82c077998feb631608dca3cc1fe19ac074e772bf0c24cf409b977b815/uritemplate-3.0.1-py2.py3-none-any.whl#sha256=07620c3f3f8eed1f12600845892b0e036a2420acf513c53f7de0abd911a5894f
Best match: uritemplate 3.0.1
Processing uritemplate-3.0.1-py2.py3-none-any.whl
Installing uritemplate-3.0.1-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding uritemplate 3.0.1 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/uritemplate-3.0.1-py3.7.egg
Searching for google-auth-httplib2>=0.0.3
Reading https://pypi.org/simple/google-auth-httplib2/
Downloading https://files.pythonhosted.org/packages/ba/db/721e2f3f32339080153995d16e46edc3a7657251f167ddcb9327e632783b/google_auth_httplib2-0.1.0-py2.py3-none-any.whl#sha256=31e49c36c6b5643b57e82617cb3e021e3e1d2df9da63af67252c02fa9c1f4a10
Best match: google-auth-httplib2 0.1.0
Processing google_auth_httplib2-0.1.0-py2.py3-none-any.whl
Installing google_auth_httplib2-0.1.0-py2.py3-none-any.whl to /usr/local/lib/python3.7/site-packages
Adding google-auth-httplib2 0.1.0 to easy-install.pth file

Installed /usr/local/lib/python3.7/site-packages/google_auth_httplib2-0.1.0-py3.7.egg
Searching for setuptools==49.6.0
Best match: setuptools 49.6.0
Adding setuptools 49.6.0 to easy-install.pth file
Installing easy_install script to /usr/local/bin
Installing easy_install-3.8 script to /usr/local/bin

Using /usr/local/lib/python3.7/site-packages
Finished processing dependencies for archiveusers==1.0

• archiveuser create-exports --config inmar
  Traceback (most recent call last):
  File "/usr/local/bin/archiveuser", line 33, in
  sys.exit(load_entry_point('archiveusers', 'console_scripts', 'archiveuser')())
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/init.py", line 488, in load_entry_point
  return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/init.py", line 2872, in load_entry_point
  return ep.load()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/init.py", line 2472, in load
  return self.resolve()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/init.py", line 2478, in resolve
  module = import(self.module_name, fromlist=['name'], level=0)
  File "/workspace/BuildArchives/process.py", line 8, in
  from .users import Users
  File "/workspace/BuildArchives/users.py", line 7, in
  from .ldap_manager import get_omitlist
  File "/workspace/BuildArchives/ldap_manager.py", line 9, in
  from ldap3 import Server, Connection, SUBTREE
  File "/usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg/ldap3/init.py", line 138, in
  from .core.connection import Connection
  File "/usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg/ldap3/core/connection.py", line 37, in
  from ..extend import ExtendedOperationsRoot
  File "/usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg/ldap3/extend/init.py", line 29, in
  from .microsoft.dirSync import DirSync
  File "/usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg/ldap3/extend/microsoft/dirSync.py", line 27, in
  from ...protocol.microsoft import dir_sync_control, extended_dn_control, show_deleted_control
  File "/usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg/ldap3/protocol/microsoft.py", line 32, in
  from .controls import build_control
  File "/usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg/ldap3/protocol/controls.py", line 27, in
  from ..utils.asn1 import encode
  File "/usr/local/lib/python3.7/site-packages/ldap3-2.5.2-py3.7.egg/ldap3/utils/asn1.py", line 50, in
  from pyasn1.codec.ber.encoder import tagMap, typeMap, AbstractItemEncoder
  ImportError: cannot import name 'tagMap' from 'pyasn1.codec.ber.encoder' (/usr/local/lib/python3.7/site-packages/pyasn1-0.5.0rc1-py3.7.egg/pyasn1/codec/ber/encoder.py)`

[tiran]

That sounds like a question for setuptools or pip, not for pyasn1. The pre-release is correctly marked as pre-release. Whatever "setup.py develop" or pip is doing triggers the installation of pre releases.

[katiyuki]

thanks! i wasn't sure where to start, so i thought i'd start here. I

[tiran]

It looks like the problem is setup.py develop. It ignores the pre-release status and always installs the highest version.

[katiyuki]

yeah, it looks like its a problem with setuptools use of easy_install, and it has been like that for years, and flagged as "won't fix".
for completeness, here is the link to the ticket on this subject: pypa/setuptools#855