error: error in setup.cfg: command 'bdist_wheel' has no such option 'azure_namespace_package'

[kappa]

Describe the bug

I am trying to build the wheel for azure-cli and the build fails with the message
"error: error in setup.cfg: command 'bdist_wheel' has no such option 'azure_namespace_package'" in several dependant packages which I am trying to list here:

azure-mgmt-billing
azure-mgmt-applicationinsights
azure-mgmt-consumption
azure-mgmt-datalake-analytics
azure-mgmt-relay

To Reproduce

$ pip wheel --no-binary :all: azure-cli

Expected behavior

Environment summary

Building the wheel with "pip wheel" on Fedora 31 with python 3.7.7.

Additional context

I believe the issue is similar to #10232.

[kappa]

I see that this issue may be more suitable for https://github.com/Azure/azure-sdk-for-python. I copied the issue there: https://github.com/Azure/azure-sdk-for-python/issues/12606

[yungezz]

thanks @kappa . @arrownj could you pls see if the issue could be resolved by PEP 420? thanks.

[arrownj]

Hi @kappa , which repo are you building ? Is it azure-cli ? Azure CLI has a customized azure_bdist_wheel file, and it is only valid for azure cli code base. So if you are building other SDKs with copied setup.py from CLI, it is not recommended.

[kappa]

@arrownj Did not copy anything. To reproduce (without cloning any repositories):

$ pip wheel --no-binary :all: azure-cli

It breaks on 5 packages that are listed in the task.

[kappa]

There are new details from @lmazuel in Azure/azure-sdk-for-python#12612.

[arrownj]

Hi @kappa , do you mean you are building azure-cli wheel ? I can build an azure-cli wheel successfully with below commands:

cd azure-cli/src/azure-cli
python setup.py bdist_wheel

I'm still not sure how to reproduce your problems.

[kappa]

@arrownj Does building with "pip"work for you? Try running "pip wheel --no-binary :all: azure-cli".

The main differences of building with "pip wheel" versus what you propose are:

1. "pip wheel" command also builds all the dependencies.
2. "pip wheel" will download the sources from pypi.org.

Some of azure-cli dependencies on pypi.org (parts of azure-sdk) are very old and that's why the whole build fails. An example: https://pypi.org/project/azure-mgmt-billing/#files

[arrownj]

Hi @kappa , I run pip wheel --no-binary :all: azure-cliand it succeed to build all the whl files in my local (I used a python3.8 virtual env).

I'm still wondering why you need to use pip wheel to build all azure-cli dependencies. All the dependencies are already on pypi and actually they are not controlled by Azure CLI.

I will close this issue first as this is not a build scenario of azure cli. If you have other building problems, feel free to reopen or submit a new issue. Thanks

[kappa]

@arrownj Trying now as well.

$  uname -v
#1 SMP Debian 5.7.10-1~bpo10+1 (2020-07-30)
$ python3 -V
Python 3.7.3
$ pip3 wheel --no-binary :all: azure-cli
...
<many successful wheels built>

Failed to build azure-mgmt-applicationinsights azure-mgmt-billing azure-mgmt-consumption azure-mgmt-datalake-analytics azure-mgmt-relay
ERROR: Failed to build one or more wheels 

So, the same 5 packages are failing. I will try on 3.8 as well later and post here.

@arrownj We need reproducible builds, cannot use binary pre-built packages.

[yungezz]

duplicate with Azure/azure-sdk-for-python#12612

[yungezz]

from the error, the failed built pacakges are sdk packages

 azure-mgmt-applicationinsights azure-mgmt-billing azure-mgmt-consumption azure-mgmt-datalake-analytics azure-mgmt-relay

hi @arrownj why cli need build sdk wheel package, shouldn't them from pypi directly?

[kappa]

Same 5 packages are failing on Fedora+Python 3.8:

Failed to build azure-mgmt-applicationinsights azure-mgmt-billing azure-mgmt-consumption azure-mgmt-datalake-analytics azure-mgmt-relay

[arrownj]

Hi @kappa ,

Just as discussed in Azure/azure-sdk-for-python#12612, this is a build problem of the 5 SDK packages. Below are the status of these SDKs.

SDK	Latest Version	Latest Version Build Success	CLI Used Version	CLI Used Version Build Success
azure-mgmt-applicationinsights	0.3.0	✅	0.1.1	❌
azure-mgmt-billing	0.2.0	❌	0.2.0	❌
azure-mgmt-consumption	3.0.0	❌	2.0.0	❌
azure-mgmt-datalake-analytics	0.6.0	❌	0.2.1	❌
azure-mgmt-relay	0.2.0	❌	0.1.0	❌

To fix this, a new version for billing/consumption/datalake-analytics/relay should be released first. Then CLI will update its depend version to the latest one.

For applicationinsights, CLI will try to use 0.3.0 version in our coming release (We need to confirm with service team, as this SDK is referenced by an extension which is owned by service team).

For your case, I think a workaround solution is to use wheel 0.30.0 version. Try to run pip install wheel==0.30.0 in your virtual env and all these 5 SDKs should be able to build successfully.

[yungezz]

hi @kappa does above workaround from @arrownj make sense to you? Could you pls give it a try then?

[yungezz]

hi @kappa pls let us know if the workaround ok.

[charmoniumQ]

To recap this and other threads:

1. azure-cli requires old versions of azure-mgmt-{consumption,datalake-analytics,relay}.
2. Those versions cannot be built from source because they depend on an old version of wheel.
   
   details

   1. setup.py tries to import azure_bdist_wheel.
   2. azure_bdist_wheel tries to import wheel.pep425tags.
   3. wheel.pep425tags was removed from wheel.
   4. The user sees Wheel is not available, disabling bdist_wheel hook.
   5. setup.py cannot use azure_bdist_wheel's custom cmdclass
   6. The default cmdclass does not know how to interpret azure_namespace_package in setup.cfg.
   7. The user sees error in setup.cfg: command 'bdist_wheel' has no such option 'azure_namespace_package'.
3. New versions of those packages no longer use azure_bdist_wheel.py, so they will close these issues and say azure-cli should use newer versions.
4. The workaround @arrownj suggested is to downgrade wheel to its 2017 version, which has the necessary deprecated submodule.

It's not obvious how to downgrade to this 2017 version of wheel from within a poetry2nix build environment for just these dependencies. Some responses say to just use the wheels (I can't find them right now). But building from source is sometimes a requirement for security and reproducibility reasons.

Is building Azure CLI/SDK from source officially supported by Microsoft? Or is it just temporarily broken, but eventually going to be fixed?