pip downloads many versions of extra'ed dependency while the exact version is pinned via a constraint

[sbidoul]

Description

Pip looks at many version of an extra'ed dependency, even if the version is pinned with a constraint.

Expected behavior

Since there is a constraint that specifies an exact version for google-api-core, could the resolver use that information to avoid looking at many versions? If the [grpc] extra is not specified it seems pip immediately selects the correct version.

pip version

21.2.1

Python version

3.8

OS

Linux, probably not relevant

How to Reproduce

Assume the following setup.py, with a dependency on google-api-core with grpc extra:

from setuptools import setup

setup(
    name="pkga",
    install_requires=[
        "google-api-core[grpc]",
    ],
)

and requirements.txt which pins google-api-core:

google-api-core==1.14.3
grpcio==1.39.0

Running pip install -c requirements.txt -e . shows that pip is trying several versions of google-api-core:

Collecting google-api-core[grpc]
  Using cached google_api_core-1.31.1-py2.py3-none-any.whl (93 kB)
  Using cached google_api_core-1.31.0-py2.py3-none-any.whl (93 kB)
  Using cached google_api_core-1.30.0-py2.py3-none-any.whl (93 kB)
  Using cached google_api_core-1.29.0-py2.py3-none-any.whl (93 kB)
  Using cached google_api_core-1.28.0-py2.py3-none-any.whl (92 kB)
  Using cached google_api_core-1.27.0-py2.py3-none-any.whl (93 kB)
  Using cached google_api_core-1.26.3-py2.py3-none-any.whl (93 kB)
  Using cached google_api_core-1.26.2-py2.py3-none-any.whl (93 kB)
  Using cached google_api_core-1.26.1-py2.py3-none-any.whl (92 kB)
  Using cached google_api_core-1.26.0-py2.py3-none-any.whl (92 kB)
  Using cached google_api_core-1.25.1-py2.py3-none-any.whl (92 kB)
  Using cached google_api_core-1.25.0-py2.py3-none-any.whl (92 kB)
  Using cached google_api_core-1.24.1-py2.py3-none-any.whl (92 kB)
  Using cached google_api_core-1.24.0-py2.py3-none-any.whl (91 kB)
  Using cached google_api_core-1.23.0-py2.py3-none-any.whl (91 kB)
  Using cached google_api_core-1.22.4-py2.py3-none-any.whl (91 kB)
  Using cached google_api_core-1.22.3-py2.py3-none-any.whl (91 kB)
  Using cached google_api_core-1.22.2-py2.py3-none-any.whl (91 kB)
  Using cached google_api_core-1.22.1-py2.py3-none-any.whl (91 kB)
  Using cached google_api_core-1.22.0-py2.py3-none-any.whl (91 kB)
  Using cached google_api_core-1.21.0-py2.py3-none-any.whl (90 kB)
  Using cached google_api_core-1.20.1-py2.py3-none-any.whl (90 kB)
  Using cached google_api_core-1.19.1-py2.py3-none-any.whl (90 kB)
  Using cached google_api_core-1.19.0-py2.py3-none-any.whl (90 kB)
  Using cached google_api_core-1.18.0-py2.py3-none-any.whl (90 kB)
  Using cached google_api_core-1.17.0-py2.py3-none-any.whl (70 kB)
  Using cached google_api_core-1.16.0-py2.py3-none-any.whl (70 kB)
  Using cached google_api_core-1.15.0-py2.py3-none-any.whl (69 kB)
  Using cached google_api_core-1.14.3-py2.py3-none-any.whl (68 kB)
Collecting protobuf>=3.4.0
  Using cached protobuf-3.17.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
  Using cached googleapis_common_protos-1.53.0-py2.py3-none-any.whl (198 kB)
Collecting google-auth<2.0dev,>=0.4.0
  Using cached google_auth-1.34.0-py2.py3-none-any.whl (152 kB)
Collecting six>=1.10.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting requests<3.0.0dev,>=2.18.0
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Requirement already satisfied: setuptools>=34.0.0 in /home/sbi-local/.virtualenvs/tempenv-12e7210736546/lib/python3.8/site-packages (from google-api-core[grpc]->pkga==0.0.0) (57.0.0)
Collecting pytz
  Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)
Collecting grpcio<2.0dev,>=1.8.2
  Using cached grpcio-1.39.0-cp38-cp38-manylinux2014_x86_64.whl (4.3 MB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting cachetools<5.0,>=2.0.0
  Using cached cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.3-py3-none-any.whl (35 kB)
Installing collected packages: six, pyasn1, urllib3, rsa, pyasn1-modules, protobuf, idna, charset-normalizer, certifi, cachetools, requests, pytz, googleapis-common-protos, google-auth, grpcio, google-api-core, pkga
  Running setup.py develop for pkga
Successfully installed cachetools-4.2.2 certifi-2021.5.30 charset-normalizer-2.0.3 google-api-core-1.14.3 google-auth-1.34.0 googleapis-common-protos-1.53.0 grpcio-1.39.0 idna-3.2 pkga-0.0.0 protobuf-3.17.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 pytz-2021.1 requests-2.26.0 rsa-4.7.2 six-1.16.0 urllib3-1.26.6

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

This is likely because the resolver treats google-api-core and google-api-core[grpc] as different candidates, with an implied constraint that they must both have the same version. But the explicit version constraint is on google-api-core and the resolver is "driving" the search from google-api-core[grpc], which therefore doesn't see that.

Extras are hard. And yucky.

It should be possible to "push" the explicit constraint down to the candidate with the extra, but it might be tricky to implement.

[uranusjr]

PR up in #10239.