We improve performance by concurrently processing files and leveraging a multi-storage model. This model allows for various limitations in the current approaches to be improved.
Tested with Python 2.7.6
Note: Installing packages may require sudo
First install a package manager. I recommend using pip which can be installed following these instructions
Required packages:
- Google Storage API
$ pip install google-api-python-client - Dropbox Core API
$ pip install dropbox - skimage (version 0.10.1)
$ pip install scikit-image==0.10.1 - Pillow (version 2.4.0)
$ pip install Pillow==2.4.0 - MoviePy (version 2.2.11)
$ pip install moviepy
num_providers: The number of instances to create with each storage provider--uploador--download: Whether to upload or download the file--million,--split,--available,--scale,--low: User policies
Example command: python driver.py 2 --upload --split