Large project directories take a long time to install.

[Peilonrayz]

Environment

• pip version: pip 19.0.3
• Python version: Python 3.7.2
• OS: Windows

I tried updating pip, but it seems my system's a little broken. I don't think it'll make a difference however.

Description

I'm using a Tox like software that creates and installs a lot of virtualenvs to test against. This means I have a folder with a lot of files and data in it. When running pip install . pip creates a temporary directory and seems to copy the entire directory into that location.

import 'pip._vendor.lockfile.mkdirlockfile' # <_frozen_importlib_external.SourceFileLoader object at 0x000002A35A3A70B8>
Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
Created temporary directory: <withheld>
Created temporary directory: <withheld>
Created requirements tracker <withheld>
Created temporary directory: <withheld>
Processing <withheld - location of my project>
  Created temporary directory: <withheld>

Navigating to the location of the final temporary directory I can see all the files being copied over.

Expected behavior

For the installation to not take over 30 minutes because it's copying files it has no need for.

How to Reproduce

Copy any large directory into a Python project and use pip install ..

Describe the solution you'd like

Maybe using something similar to building an sdist would work? This is as the following installs fine and in under a minute:

$ python setup.py sdist
$ pip install ./dist/<name>.tar.gz

It should be noted that using the -e option seems to bypass the requirement to make a temporary directory, and works fine.

[chrahunt]

Hello! This is a known issue tracked by #2195. For .tox and .nox specifically there is a fix to ignore these directories in #6770 which will be in the next release. The possible generic solutions, as described in #2195, would be:

1. something like pipignore to allow specifying ignored directories
2. building via sdist as you describe

[pradyunsg]

Thanks for filing this @Peilonrayz! I'm gonna close this as a duplicate of #2195.

@chrahunt the route we wanna go down, is building the sdist in-tree.

[Peilonrayz]

Sorry for the duplicate guys. Thanks for the fast reply. 👍