Create Linux wheels

[bovee]

It would be really nice if you could pip install taxonomy on a Linux machine the same way you can on a Mac.

[bovee]

I tried to do this three different ways, but I can't get anything usable. A very long description follows below.

The first attempt was to just use a pre-built Python image and install Rust into it. This works and generates a binary wheel, but the wheel can't be uploaded to pypi because it doesn't meet the manylinux specification.

docker pull python:3.7.3-slim-stretch
docker run -it -v ${PWD}:/mnt/ python:3.7.3-slim-stretch /mnt/python_wheel_linux.sh
# python_wheel_linux.sh
apt update -y
apt install build-essentials curl -y
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly -y
source $HOME/.cargo/env
cd /mnt
python setup.py bdist_wheel

So I tried to use the manylinux Docker image to do the build:

docker pull quay.io/pypa/manylinux2010_x86_64
docker run -v ${PWD}:/mnt python:3.7.3-slim-stretch /mnt/python_wheel_linux.sh
# python_wheel_linux.sh
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly -y
source $HOME/.cargo/env
cd /mnt
/opt/python/cp37-cp37m/bin/pip install setuptools-rust
/opt/python/cp37-cp37m/bin/python setup.py bdist_wheel
# or /opt/python/cp36-cp36m/bin/pip and /opt/python/cp36-cp36m/bin/python 

This fails because the version of glibc in the manylinux container is super old and not compatible with whatever rust is using (this version is part of the manylinux spec though so it's not like we can change it :/ ). Error message below:

Caused by:
  process didn't exit successfully: `/mnt/target/release/build/pyo3-120a7db499bffb31/build-script-build` (exit code: 1)
--- stderr
/mnt/target/release/build/pyo3-120a7db499bffb31/build-script-build: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /mnt/target/release/build/pyo3-120a7db499bffb31/build-script-build)
/mnt/target/release/build/pyo3-120a7db499bffb31/build-script-build: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by /mnt/target/release/build/pyo3-120a7db499bffb31/build-script-build)
/mnt/target/release/build/pyo3-120a7db499bffb31/build-script-build: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /mnt/target/release/build/pyo3-120a7db499bffb31/build-script-build)

error: cargo failed with code: 101

Finally, I stumbled across a Docker image put together the guy who does pyo3 (see issue here: PyO3/maturin#47). This image is built off of the manylinux spec, but supposedly works with the pyo3 build process. I was unable to get it to build though; the error message was the same as above for the raw manylinux container.

docker pull konstin2/pyo3-pack
docker run --rm --entrypoint /io/python_wheel_linux.sh -v $(pwd):/io konstin2/pyo3-pack
# python_wheel_linux.sh
rustup default nightly
/opt/python/cp37-cp37m/bin/pip install setuptools-rust
/opt/python/cp37-cp37m/bin/python setup.py bdist_wheel

So this is really tricky. I think the best course of action here may be to try to use pyo3-pack which should do this whole build process for us (but pyo3-pack has the notable disadvantage of not being able to build python bindings that are behind a feature flag such as we have).

[Keats]

@bovee have you tried pypa/manylinux#118 (comment) ?

[luizirber]

I don't know if this is useful, but in sourmash-bio/sourmash#424 I set up Rust in Travis and use our current wheel jobs (using cibuildwheel) to create a valid manylinux wheel. I don't see the GLIBC error, but I'm using stable (and milksnake), not nightly and pyO3.

The generated wheels are available on the GitHub releases for my fork, and installing with pip using the appropriate wheel URL works (it will eventually be in pypi, once the PR is merged in the main repo)

[bovee]

@Keats Thanks for finding that. I haven't seen that before. The issue I'm having is more with the linker not properly finding GLIBC than with the compiler not working. Good for reference though.

@luizirber Thanks too! I think part of the issue is that setuptools-rust isn't quite as developed at handling the old manylinux bindings as milksnake is (or pyo3-pack). We should definitely be setting up a CI deploy at some point though!

[bovee]

So I ended up getting this working by making a small patch to pyo3-pack to support the feature gating that we're doing, building/installing my new version of that in the konstin2/pyo3-pack image and then building in there. This is still fairly hacky, but I'm submitting an issue upstream to hopefully improve support for our use-case.