Installation/Usage unclear

[easherma]

Hi there. Running in terminal on a CentOS7 box. I installed libpostal and can use its utilities, and followed instructions to install python bindings (sudo pip install postal), but I get this error when trying to use code examples:

from postal.expand import expand_address ...
cannot open shared object file: No such file or directory...

dir(postal) = ['builtins', 'doc', 'file', 'name', 'package', 'path']

[albarrentine]

That sounds import path related. Did you just run pip install, or did you clone the repo first? If you cloned the repo, ran pip install, then try to import from the same checkout directory, Python's going to try to import first from the local package rather than the installed version in site-packages (and the local package hasn't compiled the C libs, so it will error). If that's what happened, you can still import from your local checkout by running python setup.py build_ext --inplace.

If that's not it, it may be related to the use of sudo. I haven't used CentOS in a few years, so my information is slightly out of date, but as I recall CentOS has been slow to adopt newer Python versions, which often created conflicts between system python and the version used by applications. If that's still the case, you might try creating a virtualenv, activating and then doing a sudo-less pip install postal.

[easherma]

pip install didn't work for me initially, so I tried cloning the repo. In the end, I've followed both instructions (pip install as well as building the extensions in place in my cloned directory), and both operations worked, but I still am having the issue.

If I wanted to chuck this and start over, is it as simple as deleting my locally installed repo or do I need to 'uninstall' it somehow?

[albarrentine]

pip uninstall postal/pip install postal should work. Might want to do the uninstall with and without sudo as well. Could be a permissions issue.

What happened with the initial pip install?

[easherma]

Should that work irregardless of the pypostal clone + python setup.py build_ext --inplace I did? I tried the uninstall/install and have the same problem as before.

Or does 'python setup.py build_ext --inplace' need to be run in the pypostal clone or in my libpostal directory? I see a setup.py in the scripts folder of libpostal but the docs seemed to indicate I should leave that folder alone.

Looking back at my log, I believe this was my initial attempt to install postal:

sudo pip install postal
Collecting postal
Downloading postal-0.3.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/site-packages (from postal)
Installing collected packages: postal
Running setup.py install for postal ... �[?25l-� �error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-8gi9dn/postal/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-spCWhV-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/postal
copying postal/init.py -> build/lib.linux-x86_64-2.7/postal
copying postal/expand.py -> build/lib.linux-x86_64-2.7/postal
copying postal/parser.py -> build/lib.linux-x86_64-2.7/postal
creating build/lib.linux-x86_64-2.7/postal/tests
copying postal/tests/init.py -> build/lib.linux-x86_64-2.7/postal/tests
copying postal/tests/test_expand.py -> build/lib.linux-x86_64-2.7/postal/tests
copying postal/tests/test_parser.py -> build/lib.linux-x86_64-2.7/postal/tests
creating build/lib.linux-x86_64-2.7/postal/utils
copying postal/utils/init.py -> build/lib.linux-x86_64-2.7/postal/utils
copying postal/utils/encoding.py -> build/lib.linux-x86_64-2.7/postal/utils
copying postal/utils/enum.py -> build/lib.linux-x86_64-2.7/postal/utils
copying postal/utils/omitted.py -> build/lib.linux-x86_64-2.7/postal/utils
running build_ext
building 'postal._expand' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/postal
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/include -I/usr/include/python2.7 -c postal/pyexpand.c -o build/temp.linux-x86_64-2.7/postal/pyexpand.o -std=c99 -Wno-unused-function
postal/pyexpand.c:1:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
�>
Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-8gi9dn/postal/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-spCWhV-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-8gi9dn/postal/�

Seemed to be dependency issue, I think I was missing one of the tools you listed as required.

[albarrentine]

Ah, ok. Python.h can be obtained using yum install python-devel I believe (or python27-devel, something like that).

python setup.py build_ext --inplace would be run from the pypostal directory (only really necessary if running a shell from that checkout directory for import path reasons).

[easherma]

Yeah, I figured that out which was able to resolve the pip install issue. Just put in some edits to the Readme to that effect to.

So I'm not sure where to go from here, then. I can uninstall/install postal successfully and I have libpostal installed.

ImportError                               Traceback (most recent call last)
<ipython-input-1-9c40c7ef0098> in <module>()
----> 1 from postal.expand import expand_address

/home/esherman/pypostal/postal/expand.py in <module>()
      3
      4 from __future__ import unicode_literals
----> 5 from postal import _expand
      6 from postal.utils.encoding import safe_decode
      7

ImportError: libpostal.so.0: cannot open shared object file: No such file or directory

If I'm trying from another directory:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-11-9c40c7ef0098> in <module>()
----> 1 from postal.expand import expand_address

/usr/lib64/python2.7/site-packages/postal/expand.py in <module>()
      3
      4 from __future__ import unicode_literals
----> 5 from postal import _expand
      6 from postal.utils.encoding import safe_decode
      7

ImportError: libpostal.so.0: cannot open shared object file: No such file or directory

The path that pip installed to:

Requirement already satisfied (use --upgrade to upgrade): postal in /usr/lib64/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/site-packages (from postal)

[albarrentine]

Ok. I don't think it's a CentOS thing if the other libpostal utilities are working. Somewhere there's

Is what you posted a full trace? What's the message after "cannot open shared object file: No such file or directory"?

Also, can you do import postal from a shell, and check that the location of postal.__file__ is where you expect it to be? When you ls that directory, you should see the files "_expand.so" and "_parser.so" which are the shared object files in question.

[easherma]

Maybe it is something with my libpostal install? Only problem I might see if I was missing a dependency when I installed that, and/or I also confirmed the data directory to be in a totally different place than the libpostal repo clone directory. But that all seemed to work fine and libpostal seems to work fine...I'm pretty stumped.

[albarrentine]

Ah, that's a linker issue, nothing wrong with install. Just make sure /usr/local/lib (or wherever you installed libpostal) is on LD_LIBRARY_PATH. Add export LD_LIBRARY_PATH=/usr/local/lib to your bashrc and open a new ssh terminal to make it persistent across sessions, probably a good idea anyway since many C libraries install there.

[easherma]

Phew, that seems to have done it!

[easherma]

I'll close this as it seems to be a pretty individual problem.

[tbonza]

+1 Thanks @easherma!

[KamilMroczek]

Had the same issue and this soled it for me. thanks guys!

[sumdan]

On CentOS, I needed to explicitly tell ldconfig to look in /usr/local/lib

cat /etc/ld.so.conf.d/$USER.conf 
/usr/local/lib

If I did that and subsequently ran ldconfig, /usr/local/lib/libpostal.so.1 was found
Note: filename must match patterns listed in /etc/ld.so.conf

[Rkuro]

For anyone that still has issues with this, I had to do export LD_LIBRARY_PATH=/usr/local/lib64/ in order for the linker to find the correct file. This was using centOS on AWS EC2 AMI. Hope this helps!

[stephenlacy]

Not trying to re-start a 1+ year old issue, but can we get #8 (comment) added to the readme? I could send a pr.
I needed to google the error to find this issue, and the comment that solved it.

[barseghyanartur]

Ah, that's a linker issue, nothing wrong with install. Just make sure /usr/local/lib (or wherever you installed libpostal) is on LD_LIBRARY_PATH. Add export LD_LIBRARY_PATH=/usr/local/lib to your bashrc and open a new ssh terminal to make it persistent across sessions, probably a good idea anyway since many C libraries install there.

Thank you! This should find place in the official documentation.