Tensorflow-io lacks a linux-arm64

[leah-jo]

I tried to download tensorflow-io and got the error code as follows.

pip3 install tensorflow-io==0.16.0
ERROR: Could not find a version that satisfies the requirement tensorflow-io==0.16.0 (from versions: none)
ERROR: No matching distribution found for tensorflow-io==0.16.0

I'm using NVIDIA's Jetson Nano board which supports ubuntu 18.04 and arm64. There was no wheel file equivalent to linux-arm64 on pypi.org. Could you add a wheel file for linux-arm64 at tensorflow-io 0.16.0 or tensorflow-io 0.15.0 ? I need either tensorflow-io 0.15.0 or tensorflow-io 0.16.0 because I’m using tensorflow 2.3.0.

[yongtang]

@leah-jo Which tensorflow arm64 package are you using? When I look at the https://pypi.org/project/tensorflow/#files I don't see a arm64 package so I am wondering about the package you have.

Assuming we can get access to tensorflow arm64 package, it could still be a little challenging as we will need to have access to a linux arm64 machine as part of the CI. As far as I can see GitHub Actions does not have native linux arm64 support yet. (see actions/runner-images#2552)

The only viable option at the moment, is to build the arm64 binary inside a docker container with QEMU.

The link actions/runner-images#2552 referenced several QEMU's so it might be possible.

[leah-jo]

Thanks for your reply! I installed tensorflow through the installing tensorflow for Jetson platform guide provided by NVIDIA (https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html), so you may need a Jetson device to get an arm64 package.
I'm going to build an arm64 binary inside the docker container with QEMU.

[hrw]

As TensorFlow 2.6-rc2 now requires tensorflow-io-gcs-filesystem I am working on building it for AArch64 as part of Linaro TensorFlow builds.

[hrw]

https://snapshots.linaro.org/ldcg/python-cache/ has aarch64 tensorflow-io packages if someone needs.

versioned as "0.20.0" but they are git HEAD ones.

[hrw]

https://git.linaro.org/ci/job/configs.git/tree/ldcg-python-manylinux-tensorflow/ansible has playbook/roles I use

[yongtang]

@hrw This is nice! Would you mind adding a small section in README.md so that more people can try out the package for arm64?

[hrw]

Feel free to add. This cache is public. No warranty, may eat your kitten etc.

One day CI systems will get aarch64 builders and user will just fetch from official places.

[cfRod]

Hi @hrw,

I installed the tensorflow-io-gcs package as shown in the screenshot but I still have issues when installing TensorFlow. I have shown the Python version and OS that I am working with.

[hrw]

Started Ubuntu:20.04 container:

root@jagular:/# apt update;apt install -y python3-venv
root@jagular:~# python3 -mvenv venv-tf
root@jagular:~# . venv-tf/bin/activate
(venv-tf) root@jagular:~# export PIP_EXTRA_INDEX_URL=https://snapshots.linaro.org/ldcg/python-cache/
Looking in indexes: https://pypi.org/simple, https://snapshots.linaro.org/ldcg/python-cache/
Collecting tensorflow-aarch64==2.7.0
  Downloading https://snapshots.linaro.org/ldcg/python/tensorflow-manylinux-nightly/20210912-94/tensorflow-aarch64/tensorflow_aarch64-2.7.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (147.2 MB)
     |████████████████████████████████| 147.2 MB 128 kB/s 
Collecting absl-py~=0.10
  Downloading absl_py-0.13.0-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 1.7 MB/s 
Collecting libclang~=11.1.0
  Downloading libclang-11.1.0-py2.py3-none-manylinux2014_aarch64.whl (13.3 MB)
     |████████████████████████████████| 13.3 MB 7.0 MB/s 
Collecting tensorflow-io-gcs-filesystem>=0.20.0
  Downloading https://snapshots.linaro.org/ldcg/python-cache/tensorflow-io-gcs-filesystem/tensorflow_io_gcs_filesystem-0.20.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.8 MB)
     |████████████████████████████████| 2.8 MB 710 kB/s 
Collecting opt-einsum~=3.3.0
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     |████████████████████████████████| 65 kB 7.7 MB/s 
Collecting termcolor~=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting grpcio<2.0,>=1.37.0
  Downloading https://snapshots.linaro.org/ldcg/python-cache/grpcio/grpcio-1.40.0-cp38-cp38-linux_aarch64.whl (50.4 MB)
     |████████████████████████████████| 50.4 MB 54 kB/s 
Collecting keras~=2.6
  Downloading keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 39.5 MB/s 
Collecting typing-extensions~=3.7.4
  Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting keras-preprocessing~=1.1.2
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
     |████████████████████████████████| 42 kB 2.7 MB/s 
Collecting tensorflow-estimator~=2.6
  Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
     |████████████████████████████████| 462 kB 40.1 MB/s 
Collecting astunparse~=1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting gast==0.4.0
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting protobuf>=3.9.2
  Downloading protobuf-3.17.3-cp38-cp38-manylinux2014_aarch64.whl (923 kB)
     |████████████████████████████████| 923 kB 38.4 MB/s 
Collecting six~=1.15.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting google-pasta~=0.2
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 9.8 MB/s 
Collecting wheel~=0.35
  Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting flatbuffers~=2.0
  Downloading flatbuffers-2.0-py2.py3-none-any.whl (26 kB)
Collecting tensorboard~=2.6
  Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
     |████████████████████████████████| 5.6 MB 38.9 MB/s 
Collecting wrapt~=1.12.1
  Downloading https://snapshots.linaro.org/ldcg/python-cache/wrapt/wrapt-1.12.1-cp38-cp38-manylinux2014_aarch64.whl (81 kB)
     |████████████████████████████████| 81 kB 589 kB/s 
Collecting h5py~=3.1.0
  Downloading https://snapshots.linaro.org/ldcg/python-cache/h5py/h5py-3.1.0-cp38-cp38-linux_aarch64.whl (7.3 MB)
     |████████████████████████████████| 7.3 MB 2.3 MB/s 
Collecting numpy~=1.19.2
  Downloading numpy-1.19.5-cp38-cp38-manylinux2014_aarch64.whl (12.4 MB)
     |████████████████████████████████| 12.4 MB 35.0 MB/s 
Requirement already satisfied: setuptools>=41.0.0 in ./venv-tf/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow-aarch64==2.7.0) (44.0.0)
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
     |████████████████████████████████| 781 kB 29.3 MB/s 
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-2.0.1-py3-none-any.whl (288 kB)
     |████████████████████████████████| 288 kB 40.5 MB/s 
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
     |████████████████████████████████| 152 kB 37.3 MB/s 
Collecting markdown>=2.6.8
  Downloading Markdown-3.3.4-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 5.8 MB/s 
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Downloading tensorboard_data_server-0.6.1-py3-none-any.whl (2.4 kB)
Collecting requests<3,>=2.21.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 503 kB/s 
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 39.4 MB/s 
Collecting rsa<5,>=3.1.4; python_version >= "3.6"
  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 40.0 MB/s 
Collecting charset-normalizer~=2.0.0; python_version >= "3"
  Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 37.6 MB/s 
Collecting idna<4,>=2.5; python_version >= "3"
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 4.9 MB/s 
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 5.9 MB/s 
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
     |████████████████████████████████| 146 kB 39.7 MB/s 
Building wheels for collected packages: termcolor
  Building wheel for termcolor (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /root/venv-tf/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-le0w22cv/termcolor/setup.py'"'"'; __file__='"'"'/tmp/pip-install-le0w22cv/termcolor/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_nk51hvn
       cwd: /tmp/pip-install-le0w22cv/termcolor/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help
  
  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for termcolor
  Running setup.py clean for termcolor
Failed to build termcolor
Installing collected packages: six, absl-py, libclang, tensorflow-io-gcs-filesystem, numpy, opt-einsum, termcolor, grpcio, keras, typing-extensions, keras-preprocessing, tensorflow-estimator, wheel, astunparse, gast, protobuf, google-pasta, flatbuffers, tensorboard-plugin-wit, werkzeug, cachetools, pyasn1, pyasn1-modules, rsa, google-auth, markdown, tensorboard-data-server, urllib3, charset-normalizer, certifi, idna, requests, oauthlib, requests-oauthlib, google-auth-oauthlib, tensorboard, wrapt, h5py, tensorflow-aarch64
    Running setup.py install for termcolor ... done
Successfully installed absl-py-0.13.0 astunparse-1.6.3 cachetools-4.2.2 certifi-2021.5.30 charset-normalizer-2.0.4 flatbuffers-2.0 gast-0.4.0 google-auth-1.35.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.40.0 h5py-3.1.0 idna-3.2 keras-2.6.0 keras-preprocessing-1.1.2 libclang-11.1.0 markdown-3.3.4 numpy-1.19.5 oauthlib-3.1.1 opt-einsum-3.3.0 protobuf-3.17.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.26.0 requests-oauthlib-1.3.0 rsa-4.7.2 six-1.15.0 tensorboard-2.6.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 tensorflow-aarch64-2.7.0 tensorflow-estimator-2.6.0 tensorflow-io-gcs-filesystem-0.20.0 termcolor-1.1.0 typing-extensions-3.7.4.3 urllib3-1.26.6 werkzeug-2.0.1 wheel-0.37.0 wrapt-1.12.1

Installed fine with our Tensorflow build.

[cfRod]

export PIP_EXTRA_INDEX_URL=https://snapshots.linaro.org/ldcg/python-cache/ was the step needed. Thanks.

[cfRod]

related tensorflow/tensorflow#51976 ?

[hrw]

Nope. #1528 is related. And packages are in Linaro cache already.

[hrw]

@yongtang can building of aarch64 packages be added to the project?

What is needed to make it happen?

[yongtang]

@hrw That will be welcomed!

At the moment, the challenge is that for every commit we cannot build aarch64 with GitHub Actions as aarch64 is not supported.

If a community aarch64 build environment is available that can at least compile and build the commit, and makes pip package available for download, then we can push the aarch64 pip package when releasing.

Otherwise we may have to wait for GitHub Actions to provide aarch64 support first I guess.

[elfringham]

Hi @yongtang if Linaro enabled nightly builds of tensorflow-io for aarch64, what testing would be needed for you to take the generated wheels after a release and upload them to your pypi repository? Is this a workable scenario?

[elfringham]

The PR #1613 would also be very helpful for this.

[elfringham]

tensorflow-io releases since 0.24.0 have included wheels for manylinux2014_aarch64 so this issue can probably be closed now.

[yongtang]

Close as this issue is resolved. Thanks all for the help and contribution!