Generated requirements.nix leads to failing psycopg2 shell.

[ldelelis]

(Assumption in the title, proceed with caution)

Generating an expression from a requirements.txt that includes psycopg2 (amongst other things) with the following command:

pypi2nix -v -V 3.6 -r requirements.txt -E postgresql -E pkgconfig -E zlib -E libjpeg -E openjpeg -E libtiff -E freetype -E lcms2 -E libwebp -E tcl

Correctly builds and generates a nix file. However, after trying to use said fail with either nix-shell, nix-build, or nix commands leads to a build error:

[...]
building
installing
/build/olefile-0.46/dist /build/olefile-0.46
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/__init__.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.6/psycopg2
running build_ext

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source.  Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.

For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
[...]

System information:

 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.66, NixOS, 19.09pre189585.1412af4b2cf (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2.2`
 - channels(root): `"nixos-19.09pre189585.1412af4b2cf"`
 - channels(blanc): `"home-manager, nixos-19.09pre189585.1412af4b2cf, nixpkgs-19.09pre190003.ed07403382c"`
 - nixpkgs: `/home/blanc/.nix-defexpr/channels/nixpkgs`

[seppeljordan]

Hi, please install the current master version of pypi2nix and try again. The following command will install it:

nix-env -if https://github.com/nix-community/pypi2nix/tarball/master

I gave it try on my machine and it seemed to work. I would love to hear back from you if this also solved your problem.

This is my go at it:

[nix-shell:~/src/pypi2nix/testing]$ result/bin/pypi2nix -V 3.6 -e psycopg2 -E postgresql
INFO: pypi2nix v2.0.0
INFO:  running ...
INFO: Stage1: Downloading wheels and creating wheelhouse ...
INFO: Stage2: Extracting metadata from pypi.python.org ...
INFO: Stage3: Generating Nix expressions ...
INFO: 
INFO: Nix expressions generated successfully.
INFO: 
INFO: To start development run:
INFO:     nix-shell requirements.nix -A interpreter
INFO: 
INFO: More information you can find at
INFO:     https://github.com/nix-community/pypi2nix

[nix-shell:~/src/pypi2nix/testing]$ nix build -f requirements.nix interpreter
[3 built, 2 copied (5.8 MiB), 1.0 MiB DL]

[nix-shell:~/src/pypi2nix/testing]$ echo $?
0

[ldelelis]

@seppeljordan thanks for the quick response! Building from master works correctly.

What I just noticed is that nixpkgs refers to the latest release (v1.8.1), which was almost 2 years ago. Would it be ok to generate a new release as to up its nixpkgs version?

[seppeljordan]

Hi, I definitely see how we should update pypi2nix on nixpkgs. But there are several issues that make me uncomfortable pushing this now. The biggest issue is a problem with pip. This application currently distributes its own version of pip from a personal fork. The corresponding pull request for pip is unfortunately stuck. And without this patched version pypi2nix simply would not work for a lot of basic packages, e.g. flake8. This makes me very hesitant to publish.

[ldelelis]

Makes sense. Anything I can do to help?

[ldelelis]

Also closing this as we've verified this doesn't happen in upstream

[seppeljordan]

First of all, thanks for asking.

pypa/pip#6606

This is a PR that addresses the issue with pip. If you know somebody who can review this or can do it yourself that would be awesome.