pip's error message for wheel-build failure on PEP517 packages isn't very friendly

[Julian]

Environment

• pip version: 19.0.2
• Python version:

Python 2.7.13 (ab0b9caf307db6592905a80b8faffd69b39005b8, Sep 30 2018, 13:49:36)
[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.2)]

• OS: macOS

Description
When pip fails to build a wheel, a fairly cryptic message appears for someone not "intimately" aware of the Python packaging ecosystem:

(The below is with cryptography, which does not publish macOS wheels for PyPy, but should fail the same way for any similar failure).

⊙  pypy ~/Development/virtualenv/virtualenv.py venv && venv/bin/pip install --quiet cryptography                                                                                                                                       jberman@USNYHJBERMANMB2
New pypy executable in /Users/jberman/Desktop/venv/bin/pypy
Installing setuptools, pip, wheel...done.
DEPRECATION: A future version of pip will drop support for Python 2.7.
/Users/jberman/Desktop/venv/site-packages/pip/_vendor/msgpack/fallback.py:222: PendingDeprecationWarning: encoding is deprecated, Use raw=False instead.
  PendingDeprecationWarning)
  Failed building wheel for cryptography
Could not build wheels for cryptography which use PEP 517 and cannot be installed directly

I wouldn't expect a random Python developer to know what PEP517 is (today?) in isolation. The error message appears to just be a way of saying "building the wheel failed, for reasons like e.g. not being able to find OpenSSL in this specific case, but more generically the compiler or some other thing in the build toolchain exiting unsuccessfully".

These appear to be the relevant pip source lines.

Expected behavior
Ideally an error message that could be understood in isolation would be nice -- I can't tell immediately the specific set of things that need to happen to trigger that branch in the code, but my best guess at the minute would be something like `Building a wheel failed. See above for details of the specific error. Refusing to attempt an sdist build because uses features present only in PEP517. See for details and <foo's setup.py presumably, ideally with some specific info on what feature it uses?> for specifics."

FWIW the old version of this error was not any more friendly. On pip 18.x:

Command "/Users/jberman/Desktop/venv/bin/pypy -u -c "import setuptools, tokenize;__file__='/private/var/folders/46/1d5dl2sn5n9b0f3pd8lbvz9hhr5mnp/T/pip-install-96YFMf/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/46/1d5dl2sn5n9b0f3pd8lbvz9hhr5mnp/T/pip-record-kUXGwh/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jberman/Desktop/venv/include/site/python2.7/cryptography" failed with error code 1 in /private/var/folders/46/1d5dl2sn5n9b0f3pd8lbvz9hhr5mnp/T/pip-install-96YFMf/cryptography/

which is close to gibberish, but I guess I was conditioned to tune out that line and just focus on the one above it that says building the wheel failed -- but ideally if the error message is changing, would be nice to show something that could help the person seeing it to know what to do.

How to Reproduce / Output
(See above shell line)

[cjerdonek]

I agree. It would be nice to have a way to talk about this without having to refer to PEP 517 / requiring people to know about it. Referencing pyproject.toml on the other hand could be okay IMO as it's something more public-facing.

[pfmoore]

Agreed, this should be reported better. However, the message here is only intended as supplemental, the details of what went wrong should have been reported by the build system itself, and in this case it doesn't appear to be doing so, making pip's generic message even less useful :-(

[cjerdonek]

@pfmoore I noticed that, too. But from looking at the code, I'm pretty sure the details are normally included. Notice that --quiet was included in the original report. I'm guessing the reporter did that to focus on the relevant part of the message. @Julian, can you confirm?

[Julian]

Yes, exactly!

…

On Tue, Feb 12, 2019, 15:26 Chris Jerdonek ***@***.*** wrote: @pfmoore <https://github.com/pfmoore> I noticed that, too. But from looking at the code, I'm *pretty* sure the details are normally included. Notice that --quiet was included in the original report. I'm guessing the reporter did that to focus on the relevant part of the message. @Julian <https://github.com/Julian>, can you confirm? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6256 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUIXgbx2EX_gVmkH1fSDd8culD7QNrMks5vMyN6gaJpZM4a1B-C> .

[callegar]

The problem is not really with that message. The problem is that what is provided before that message is most of the time totally irrelevant.

For instance, if on my system I try pip install --force-reinstall --no-binary :all: numpy, then that fails with the infamous ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly. Before that, I see that the install fails precisely at the Building wheel for numpy (PEP 517) phase.

Pip tries to provide Complete output. That on my system is a good 369 lines. Not a single one containing the word "error" or an indication that something is not found. No indication whatsoever of what the building of the wheel implied (calling external programs? invoking the linker?).

Any hint at how to know what pip (or possibly setuptools) is trying to achieve before erroring out?

[Julian]

My experience is that what's before it generally is relevant, and is what needs looking at, though is certainly cryptic and error specific -- but that those are facts more relevant to the state of compilers than of anything Python related. What you get is the huge dump of errors coming from a C compiler or worse, and yeah, reading them is not easy.

Given the opposite experience though perhaps we can discuss specific examples (perhaps your specific build output?), and maybe in a separate ticket?

I assume you don't disagree with:

• pip should show the output of what it was trying to do untouched
• pip when failing should not reference PEPs, and should instead describe what it was running when referencing that output in simpler terms that are as actionable as possible

and that ideally

• pip should provide hints when it knows specific areas in the output that should be looked at, based on some interior knowledge of what was being tried

(So personally I think the issue of this ticket is precisely that message and not beyond it)