Cannot install azure cli on macOS via script - virtualenv error

[alfeilex]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command
curl -L https://aka.ms/InstallAzureCli | bash

Describe the bug
To install Azure CLI on MacOS, I would like to use a bash script and not a package manager like homebrew as recommend in the Azure CLI Docs. I have used the following instructions to setup Azure CLI via script.
Installation with Script

During the installation of azure cli with the previous related command and error occured that is related with virtualenv variable of Python.

To Reproduce
Start the installation with related command on macOS and make sure that the requirements are met before installation.

Expected behavior
The installation process should run without errors.

Environment summary
Interactive script executed by Terminal on MacOS High Sierra | az --version > 2.36.0

Additional context
The only way to fix the error was to install virtualenv with conda.
conda install virtualenv

Here is the error message.

===> In what directory would you like to place the install? (leave blank to use '/Users/root1/lib/azure-cli'): 
-- We will install at '/Users/root1/lib/azure-cli'.

===> In what directory would you like to place the 'az' executable? (leave blank to use '/Users/root1/bin'): 
-- The executable will be in '/Users/root1/bin'.
-- Downloading virtualenv package from https://pypi.python.org/packages/source/v/virtualenv/virtualenv-16.7.11.tar.gz.

-- Downloaded virtualenv package to /var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/tmpoeiop18y/virtualenv-16.7.11.tar.gz.
-- Checksum of /var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/tmpoeiop18y/virtualenv-16.7.11.tar.gz OK.
-- Extracting '/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/tmpoeiop18y/virtualenv-16.7.11.tar.gz' to '/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/tmpoeiop18y'.
-- Executing: ['/Library/Frameworks/Python.framework/Versions/3.10/bin/python3', 'virtualenv.py', '--python', '/Library/Frameworks/Python.framework/Versions/3.10/bin/python3', '/Users/root1/lib/azure-cli']
/private/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/tmpoeiop18y/virtualenv-16.7.11/virtualenv.py:24: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  import distutils.spawn
/private/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/tmpoeiop18y/virtualenv-16.7.11/virtualenv.py:25: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  import distutils.sysconfig
Already using interpreter /Library/Frameworks/Python.framework/Versions/3.10/bin/python3
Using base prefix '/Library/Frameworks/Python.framework/Versions/3.10'
New python executable in /Users/root1/lib/azure-cli/bin/python3
Also creating executable in /Users/root1/lib/azure-cli/bin/python
ERROR: The executable /Users/root1/lib/azure-cli/bin/python3 is not functioning
ERROR: It thinks sys.prefix is '/Library/Frameworks/Python.framework/Versions/3.10' (should be '/Users/root1/lib/azure-cli')
ERROR: virtualenv is not compatible with this system or executable
Traceback (most recent call last):
  File "/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/azure_cli_install_tmp_XXXXXX.9JI5cENc", line 415, in <module>
    main()
  File "/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/azure_cli_install_tmp_XXXXXX.9JI5cENc", line 399, in main
    create_virtualenv(tmp_dir, install_dir)
  File "/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/azure_cli_install_tmp_XXXXXX.9JI5cENc", line 144, in create_virtualenv
    exec_command(cmd, cwd=working_dir)
  File "/var/folders/9w/sj6bkvbd7p3dzj9fxtvx9vqm0000gn/T/azure_cli_install_tmp_XXXXXX.9JI5cENc", line 105, in exec_command
    subprocess.check_call(command_list, cwd=cwd, env=env)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.10/bin/python3', 'virtualenv.py', '--python', '/Library/Frameworks/Python.framework/Versions/3.10/bin/python3', '/Users/root1/lib/azure-cli']' returned non-zero exit status 100.

[yonzhan]

@jiasli for awareness

[bebound]

I can reproduce this, seems like a compatibility problem of virtualenv in macOS. Failed to create virtual env using system python /Library/Developer/CommandLineTools/usr/bin/python3 and brew Python /opt/homebrew/opt/[email protected]/bin/python3.9.

Can we replace virtualenv with venv? @jiasli

[bebound]

@alfeilex This script is outdated, you can find the latest pip installation guide here: #20476.

[alfeilex]

@bebound Is there currently a way to install Azure CLI via bash script without package manager like pip or homebrew

[bebound]

I don't think there is a simpler way to do so on macOS.

[alfeilex]

I don't think there is a simpler way to do so on macOS.

Would it be possible to make an update so that via a bash script you can install Azure CLI directly on macOS without package manager.

[bebound]

python3 -m venv azure-cli-env
azure-cli-env/bin/python -m pip install --upgrade pip
azure-cli-env/bin/python -m pip install wheel
azure-cli-env/bin/python -m pip install azure-cli

This script is modified from the guide above. az command can be called like this: azure-cli-env/bin/az --version.
Does this meet your request?

[alfeilex]

Thanks for the help. Your procedure is a local installation. Is there also the possibility to install Azure globally on Mac like on Windows?

If not, the ticket can be closed. I can work with your suggestion.

[bebound]

Unfortunately, brew package is the only way to install CLI globally on Mac.

[alfeilex]

Counter question... is there a way to install Azure CLI for Windows locally without Python? Are there installation urls to ZIP files?

[Sehnbarn]

As root, we will install Azure CLI (system-wide) in /usr/local rather than in a non-privileged user context (~/)

yourhost:~ yourlogin$ sudo -s -H

Enter your session password to get a session in the root user's context. Check your environment, e.g. on macOS Ventura (13.0) :

bash-3.2# uname -v
Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64

If needed, install Xcode Command Line Tools (or Xcode from the App Store) :
bash-3.2# xcode-select --install
xcode-select: note: install requested for command line developer tools

A window "Install Command Line Developer Tools" will pop up; click "Install" and "Agree" the license agreement. This will trigger the download for the required software and install the command line tools in /Library/Developer/CommandLineTools.

Install/Update Python package installer (pip) :
bash-3.2# python3 -m pip install --upgrade pip
Requirement already satisfied: pip in /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages (21.2.4)
Collecting pip
Downloading pip-22.3-py3-none-any.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 2.3 MB/s
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 21.2.4
WARNING: Value for bin_prefix does not match. Please report this to pypa/pip#10151
distutils: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin
sysconfig: /usr/local/bin
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Uninstalling pip-21.2.4:
Successfully uninstalled pip-21.2.4
Successfully installed pip-22.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Install Python virtual environment manager (virtualenv) :
bash-3.2# python3 -m pip install virtualenv
Collecting virtualenv
Downloading virtualenv-20.16.6-py3-none-any.whl (8.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 4.2 MB/s eta 0:00:00
Collecting platformdirs<3,>=2.4
Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB)
Collecting distlib<1,>=0.3.6
Downloading distlib-0.3.6-py2.py3-none-any.whl (468 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.5/468.5 kB 3.7 MB/s eta 0:00:00
Collecting filelock<4,>=3.4.1
Downloading filelock-3.8.0-py3-none-any.whl (10 kB)
Installing collected packages: distlib, platformdirs, filelock, virtualenv
Successfully installed distlib-0.3.6 filelock-3.8.0 platformdirs-2.5.2 virtualenv-20.16.6
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Check your version of virtualenv :
bash-3.2# virtualenv --version
virtualenv 20.16.6 from /Library/Python/3.9/site-packages/virtualenv/init.py

Prepare temporary environment and get Azure CLI installation script :
bash-3.2# mkdir -p /tmp/virtualenv-20.16.6
bash-3.2# cd /tmp/virtualenv-20.16.6
bash-3.2# ln -s /usr/local/bin/virtualenv

Download the Azure CLI installation script :
bash-3.2# curl -O https://azurecliprod.blob.core.windows.net/install.py

Edit the install.py script to prevent the installation of a problematic virtualenv version that the script wants - see attached file install.py.zip.
The lines concerning the virtual environment (beginning with VIRTUALENV_) have been commented out.
The default installation has ben modified to favor the /usr/local/* directories(lines 49 and 50).
The function called create_virtualenv(...) has been modified to use the current (/tmp/virtualenv-20.16.6) directory and the virtualenv executable (/usr/local/bin/virtualenv) with the following lines (near line #144) :

#working_dir = os.path.join(tmp_dir, virtualenv_dir_name)
working_dir = os.path.join('/tmp', virtualenv_dir_name)
#cmd = [sys.executable, 'virtualenv.py', '--python', sys.executable, install_dir]
cmd = [sys.executable, 'virtualenv', '--python', sys.executable, install_dir]

Launch the installation with :
bash-3.2# python3 install.py

You will be asked the two following questions :
===> In what directory would you like to place the install? (leave blank to use '/usr/local/lib/azure-cli'):
-- We will install at '/usr/local/lib/azure-cli'.

===> In what directory would you like to place the 'az' executable? (leave blank to use '/usr/local/bin'):
-- The executable will be in '/usr/local/bin'.

The script will then download and install all the Azure CLI modules/packages :

[...]

Installing collected packages: urllib3-secure-extra, jsondiff, invoke, chardet, azure-nspkg, azure-common, argcomplete, applicationinsights, antlr4-python3-runtime, xmltodict, wrapt, websocket-client, urllib3, typing-extensions, tabulate, six, semver, pyyaml, PySocks, pyparsing, PyMySQL, PyJWT, pygments, pycparser, psutil, portalocker, pkginfo, oauthlib, jmespath, idna, humanfriendly, colorama, charset-normalizer, certifi, bcrypt, azure-mgmt-nspkg, requests, python-dateutil, pathlib2, packaging, knack, javaproperties, isodate, deprecated, cffi, azure-mgmt-datalake-nspkg, azure-cli-telemetry, requests-oauthlib, PyNaCl, cryptography, azure-cosmos, azure-core, pyopenssl, PyGithub, paramiko, msrest, azure-storage-common, azure-mgmt-core, adal, sshtunnel, scp, msrestazure, msal, fabric, azure-synapse-spark, azure-synapse-managedprivateendpoints, azure-synapse-artifacts, azure-synapse-accesscontrol, azure-multiapi-storage, azure-mgmt-web, azure-mgmt-trafficmanager, azure-mgmt-synapse, azure-mgmt-storage, azure-mgmt-sqlvirtualmachine, azure-mgmt-sql, azure-mgmt-signalr, azure-mgmt-servicelinker, azure-mgmt-servicefabricmanagedclusters, azure-mgmt-servicefabric, azure-mgmt-servicebus, azure-mgmt-security, azure-mgmt-search, azure-mgmt-resource, azure-mgmt-redis, azure-mgmt-redhatopenshift, azure-mgmt-recoveryservicesbackup, azure-mgmt-recoveryservices, azure-mgmt-rdbms, azure-mgmt-privatedns, azure-mgmt-policyinsights, azure-mgmt-network, azure-mgmt-netapp, azure-mgmt-msi, azure-mgmt-monitor, azure-mgmt-media, azure-mgmt-marketplaceordering, azure-mgmt-maps, azure-mgmt-managementgroups, azure-mgmt-loganalytics, azure-mgmt-keyvault, azure-mgmt-iothubprovisioningservices, azure-mgmt-iothub, azure-mgmt-iotcentral, azure-mgmt-imagebuilder, azure-mgmt-hdinsight, azure-mgmt-extendedlocation, azure-mgmt-eventhub, azure-mgmt-eventgrid, azure-mgmt-dns, azure-mgmt-datamigration, azure-mgmt-databoxedge, azure-mgmt-cosmosdb, azure-mgmt-containerservice, azure-mgmt-containerregistry, azure-mgmt-containerinstance, azure-mgmt-compute, azure-mgmt-cognitiveservices, azure-mgmt-cdn, azure-mgmt-botservice, azure-mgmt-billing, azure-mgmt-batchai, azure-mgmt-batch, azure-mgmt-applicationinsights, azure-mgmt-appconfiguration, azure-mgmt-apimanagement, azure-mgmt-advisor, azure-loganalytics, azure-keyvault-keys, azure-keyvault-administration, azure-datalake-store, azure-data-tables, azure-appconfiguration, msal-extensions, azure-mgmt-relay, azure-mgmt-managedservices, azure-mgmt-kusto, azure-mgmt-devtestlabs, azure-mgmt-deploymentmanager, azure-mgmt-datalake-store, azure-mgmt-datalake-analytics, azure-mgmt-consumption, azure-mgmt-authorization, azure-keyvault, azure-graphrbac, azure-batch, azure-cli-core, azure-cli
Successfully installed PyGithub-1.56 PyJWT-2.6.0 PyMySQL-1.0.2 PyNaCl-1.5.0 PySocks-1.7.1 adal-1.2.7 antlr4-python3-runtime-4.9.3 applicationinsights-0.11.10 argcomplete-1.12.3 azure-appconfiguration-1.1.1 azure-batch-12.0.0 azure-cli-2.42.0 azure-cli-core-2.42.0 azure-cli-telemetry-1.0.8 azure-common-1.1.28 azure-core-1.26.0 azure-cosmos-3.2.0 azure-data-tables-12.4.0 azure-datalake-store-0.0.52 azure-graphrbac-0.60.0 azure-keyvault-1.1.0 azure-keyvault-administration-4.0.0b3 azure-keyvault-keys-4.5.1 azure-loganalytics-0.1.1 azure-mgmt-advisor-9.0.0 azure-mgmt-apimanagement-3.0.0 azure-mgmt-appconfiguration-2.2.0 azure-mgmt-applicationinsights-1.0.0 azure-mgmt-authorization-0.61.0 azure-mgmt-batch-16.2.0 azure-mgmt-batchai-7.0.0b1 azure-mgmt-billing-6.0.0 azure-mgmt-botservice-2.0.0b3 azure-mgmt-cdn-12.0.0 azure-mgmt-cognitiveservices-13.3.0 azure-mgmt-compute-29.0.0 azure-mgmt-consumption-2.0.0 azure-mgmt-containerinstance-9.1.0 azure-mgmt-containerregistry-10.0.0 azure-mgmt-containerservice-20.6.0 azure-mgmt-core-1.3.2 azure-mgmt-cosmosdb-8.0.0 azure-mgmt-databoxedge-1.0.0 azure-mgmt-datalake-analytics-0.2.1 azure-mgmt-datalake-nspkg-3.0.1 azure-mgmt-datalake-store-0.5.0 azure-mgmt-datamigration-10.0.0 azure-mgmt-deploymentmanager-0.2.0 azure-mgmt-devtestlabs-4.0.0 azure-mgmt-dns-8.0.0 azure-mgmt-eventgrid-10.2.0b2 azure-mgmt-eventhub-10.1.0 azure-mgmt-extendedlocation-1.0.0b2 azure-mgmt-hdinsight-9.0.0 azure-mgmt-imagebuilder-1.1.0 azure-mgmt-iotcentral-10.0.0b1 azure-mgmt-iothub-2.2.0 azure-mgmt-iothubprovisioningservices-1.1.0 azure-mgmt-keyvault-10.1.0 azure-mgmt-kusto-0.3.0 azure-mgmt-loganalytics-13.0.0b4 azure-mgmt-managedservices-1.0.0 azure-mgmt-managementgroups-1.0.0 azure-mgmt-maps-2.0.0 azure-mgmt-marketplaceordering-1.1.0 azure-mgmt-media-9.0.0 azure-mgmt-monitor-5.0.1 azure-mgmt-msi-6.1.0 azure-mgmt-netapp-9.0.1 azure-mgmt-network-21.0.1 azure-mgmt-nspkg-3.0.2 azure-mgmt-policyinsights-1.1.0b3 azure-mgmt-privatedns-1.0.0 azure-mgmt-rdbms-10.2.0b4 azure-mgmt-recoveryservices-2.1.0 azure-mgmt-recoveryservicesbackup-5.1.0b2 azure-mgmt-redhatopenshift-1.1.0 azure-mgmt-redis-13.1.0 azure-mgmt-relay-0.1.0 azure-mgmt-resource-21.1.0b1 azure-mgmt-search-8.0.0 azure-mgmt-security-2.0.0b1 azure-mgmt-servicebus-8.1.0 azure-mgmt-servicefabric-1.0.0 azure-mgmt-servicefabricmanagedclusters-1.0.0 azure-mgmt-servicelinker-1.0.0 azure-mgmt-signalr-1.1.0 azure-mgmt-sql-4.0.0b4 azure-mgmt-sqlvirtualmachine-1.0.0b3 azure-mgmt-storage-20.1.0 azure-mgmt-synapse-2.1.0b5 azure-mgmt-trafficmanager-1.0.0 azure-mgmt-web-7.0.0 azure-multiapi-storage-0.10.0 azure-nspkg-3.0.2 azure-storage-common-1.4.2 azure-synapse-accesscontrol-0.5.0 azure-synapse-artifacts-0.14.0 azure-synapse-managedprivateendpoints-0.3.0 azure-synapse-spark-0.2.0 bcrypt-4.0.1 certifi-2022.9.24 cffi-1.15.1 chardet-3.0.4 charset-normalizer-2.1.1 colorama-0.4.6 cryptography-38.0.3 deprecated-1.2.13 fabric-2.7.1 humanfriendly-10.0 idna-3.4 invoke-1.7.3 isodate-0.6.1 javaproperties-0.5.2 jmespath-1.0.1 jsondiff-2.0.0 knack-0.10.0 msal-1.20.0 msal-extensions-1.0.0 msrest-0.7.1 msrestazure-0.6.4 oauthlib-3.2.2 packaging-21.3 paramiko-2.11.0 pathlib2-2.3.7.post1 pkginfo-1.8.3 portalocker-2.6.0 psutil-5.9.3 pycparser-2.21 pygments-2.13.0 pyopenssl-22.1.0 pyparsing-3.0.9 python-dateutil-2.8.2 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 scp-0.13.6 semver-2.13.0 six-1.16.0 sshtunnel-0.1.5 tabulate-0.9.0 typing-extensions-4.4.0 urllib3-1.26.12 urllib3-secure-extra-0.1.0 websocket-client-1.3.3 wrapt-1.14.1 xmltodict-0.13.0
-- The executable is available at '/usr/local/bin/az'.
-- Created tab completion file at '/usr/local/lib/azure-cli/az.completion'

At the end, you will be asked if you want to update your $PATH environment variable :
===> Modify profile to update your $PATH and enable shell/tab completion now? (Y/n): Y

===> Could not automatically find a suitable file to use. Create /var/root/.bash_profile now? (Y/n): Y

The installation is now done. The final step will consist in the modification of your .bash_profile to add these two lines (accordingly to your previous choices concerning the installation paths), e.g. :

export PATH=$PATH:/usr/local/bin
source '/usr/local/lib/azure-cli/az.completion'

Exit the root session and test an az command, for instance :

yourhost:~ yourlogin$ which az
/usr/local/bin/az

or

yourhost:~ yourlogin$ az version
{
"azure-cli": "2.42.0",
"azure-cli-core": "2.42.0",
"azure-cli-telemetry": "1.0.8",
"extensions": {}
}

Clean up the temporary installation directory :
yourhost:~ yourlogin$ sudo rm -Rf /tmp/virtualenv-20.16.6

Enjoy !