Unable to create virtualenv on CircleCI with Pipenv

[chrisrink10]

This morning as my team started pushing changes to CircleCI after the weekend, we started experiencing failures due to an issue that appears to originate with the fix to #1561.

We have a CircleCI step (inside the image circleci/python:3.8-buster, using standard Linux builders) which does the following:

#!/bin/bash -eo pipefail
pip install -U pip~=20.0.0
sudo pip install pipenv
pipenv install --dev

The output of which is:

Requirement already up-to-date: pip~=20.0.0 in /usr/local/lib/python3.8/site-packages (20.0.2)
Requirement already satisfied: pipenv in /usr/local/lib/python3.8/site-packages (2018.11.26)
Requirement already satisfied: certifi in /usr/local/lib/python3.8/site-packages (from pipenv) (2019.11.28)
Requirement already satisfied: virtualenv in /usr/local/lib/python3.8/site-packages (from pipenv) (20.0.5)
Requirement already satisfied: virtualenv-clone>=0.2.5 in /usr/local/lib/python3.8/site-packages (from pipenv) (0.5.3)
Requirement already satisfied: pip>=9.0.1 in /usr/local/lib/python3.8/site-packages (from pipenv) (20.0.2)
Requirement already satisfied: setuptools>=36.2.1 in /usr/local/lib/python3.8/site-packages (from pipenv) (45.1.0)
Requirement already satisfied: filelock<4,>=3.0.0 in /usr/local/lib/python3.8/site-packages (from virtualenv->pipenv) (3.0.12)
Requirement already satisfied: six<2,>=1.9.0 in /usr/local/lib/python3.8/site-packages (from virtualenv->pipenv) (1.14.0)
Requirement already satisfied: appdirs<2,>=1.4.3 in /usr/local/lib/python3.8/site-packages (from virtualenv->pipenv) (1.4.3)
Requirement already satisfied: distlib<1,>=0.3.0 in /usr/local/lib/python3.8/site-packages (from virtualenv->pipenv) (0.3.0)
Creating a virtualenv for this project…
Pipfile: /home/circleci/repo/Pipfile
Using /usr/local/bin/python3.8 (3.8.1) to create virtualenv…
ImportError: cannot import name 'is_mac_os_framework' from 'virtualenv.create.via_global_ref.builtin.cpython.cpython2' (/usr/local/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/cpython/cpython2.py)


FAIL
['Traceback (most recent call last):\n', '  File "/usr/local/lib/python3.8/site-packages/pipenv/utils.py", line 501, in create_spinner\n    yield sp\n', '  File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 934, in do_create_virtualenv\n    raise exceptions.VirtualenvCreationException(\n', 'pipenv.exceptions.VirtualenvCreationException: Failed to create virtual environment.\n']
[pipenv.exceptions.VirtualenvCreationException]:   File "/usr/local/lib/python3.8/site-packages/pipenv/cli/command.py", line 235, in install
[pipenv.exceptions.VirtualenvCreationException]:       retcode = do_install(
[pipenv.exceptions.VirtualenvCreationException]:   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 1734, in do_install
[pipenv.exceptions.VirtualenvCreationException]:       ensure_project(
[pipenv.exceptions.VirtualenvCreationException]:   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 570, in ensure_project
[pipenv.exceptions.VirtualenvCreationException]:       ensure_virtualenv(
[pipenv.exceptions.VirtualenvCreationException]:   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 505, in ensure_virtualenv
[pipenv.exceptions.VirtualenvCreationException]:       do_create_virtualenv(
[pipenv.exceptions.VirtualenvCreationException]:   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 934, in do_create_virtualenv
[pipenv.exceptions.VirtualenvCreationException]:       raise exceptions.VirtualenvCreationException(
[pipenv.exceptions.VirtualenvCreationException]: 
Failed to create virtual environment.

Exited with code exit status 1

I was able to work around the issue by locking our virtualenv to version 20.0.4, which is also the same version as reported by Pipenv in the most recent successful build of the same job.

Per the reporting instructions, I don't believe I can provide you with any information inside the virtualenv since I am unable to create it.

[gaborbernat]

This looks like pip is not being installed correctly; not sure why though. Note the import failed is definitely there https://github.com/pypa/virtualenv/blob/master/src/virtualenv/create/via_global_ref/builtin/cpython/cpython2.py#L53-L55 and I've checked that it's there in sdist/wheel. You're log says: Requirement already satisfied: virtualenv in /usr/local/lib/python3.8/site-packages (from pipenv) (20.0.5) .... but feels like a broken install as that import from mac_os.py fails from cpython2. I'd like to see the install log, the original. You can try to force reinstall virtualenv as workaround.

[gaborbernat]

 docker run --rm -it --entrypoint=bash  circleci/python:3.8-buster -c 'cat /usr/local/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/cpython/cpython2.py' | grep is_mac_os                                                                                                                                    611ms  Mon 24 Feb 16:47:03 2020
def is_mac_os_framework(interpreter):
        return is_mac_os_framework(interpreter) is False and super(CPython2Posix, cls).can_describe(interpreter)

Seems it's there, please put together a docker file that reproduces the issue.

[chrisrink10]

I tried pulling our dependencies again and it seems to have resolved. Must've been corrupted as you said. Thanks.