ModuleNotFoundError in musllinux

[veikman]

In this FOSS project I use the ghcr.io/pyo3/maturin Docker container to build for both manylinux and musllinux. Wheels built for musllinux can be installed on musl systems, but cannot be used.

Here is a Dockerfile I’ve used to test musllinux:

FROM python:3.11-alpine
COPY dist/ .
RUN pip3 install punwrap-0.2.3-cp311-cp311-musllinux_1_2_x86_64.whl 

Here is the output I get running python -c 'import punwrap' in that container:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/punwrap/__init__.py", line 1, in <module>
    from .punwrap import *
ModuleNotFoundError: No module named 'punwrap.punwrap'

In the Docker container, /usr/local/lib/python3.11/site-packages/punwrap/ does have the compiled punwrap module (extension) as expected.

__init__.py
__pycache__
punwrap.cpython-311-x86_64-linux-gnu.so

For context, wheels I build for manylinux have similar contents and work as intended on libc systems. I had musllinux builds working with an earlier version of maturin and Python, though not in Docker.

I notice that the built extension (.so) for musslinux has a GNU suffix, while Alpine is not tagged GNU/Linux. Could that be why the interpreter in the Alpine Linux environment ignores the extension?

[messense]

How did you build musllinux wheels exactly?

[veikman]

Thank you for the advice. I tried adding that option just now. Rust compilation now breaks, to the point that Cargo can’t seem to find an appropriate std.

error: could not compile `memchr` (lib) due to 988 previous errors
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `CARGO_ENCODED_RUSTFLAGS="-C\u{1f}target-feature=-crt-static" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.8-64bit" PYO3_PYTHON="/opt/python/cp38-cp38/bin/python3.8" PYTHON_SYS_EXECUTABLE="/opt/python/cp38-cp38/bin/python3.8" "cargo" "rustc" "--target" "x86_64-unknown-linux-musl" "--message-format" "json-render-diagnostics" "--manifest-path" "/io/Cargo.toml" "--release" "--lib"`

[messense]

It should break because you don't have a musl Rust std and a working musl gcc.

[veikman]

Yes, that part makes sense. It is more surprising to me that, without such requirements in place, maturin still accepts the musllinux compatibility option, providing no warnings about being unable to build for musl.

From my perspective as a user, I would like for the musl requirements to be containerized with maturin in some easily discoverable way, but I know that’s asking a lot.

Do you recommend that I replicate something like this example for my project?

[messense]

Do you recommend that I replicate something like this example for my project?

No, that one is a binary crate. For extension module, you can replicate https://github.com/messense/py-dissimilar/blob/main/.github/workflows/CI.yml

[veikman]

Thank you very much for your help and your work on maturin! I’ve got my project building in CI now.

On the subject of maturin in CI, I assume that maturin-action does not support trusted publishing. Please correct me if I am wrong.

[messense]

Yes and no, while maturin-action doesn't support trusted publishing directly maturin does, so it should just work with maturin-action.