RPATH support in Ubuntu

[diegoferigo]

This is really interesting, if today on Ubuntu 18.04 I execute the same command of #147 (comment), I get:

dferigo@iiticublap110:~> objdump -x /iit/local/lib/libYARP_math.so | grep -2 PATH
  NEEDED               libc.so.6
  SONAME               libYARP_math.so.3
  RUNPATH              $ORIGIN/:$ORIGIN/../lib:$ORIGIN/../
  INIT                 0x0000000000008298
  FINI                 0x00000000000ba4e4

Note that RPATH is no longer there. I warmly recommend reading this comment that applies to recent Ubuntu distros. Can you please check the behaviour on your system?

Related to #147.

cc @traversaro @drdanz @claudiofantacci @francesco-romano

[drdanz]

The RPATH name was changed to RUNPATH at some point, i have no idea when and why this happened

[traversaro]

[diegoferigo]

The RPATH name was changed to RUNPATH at some point

What?!? Do you have any link? I just found the RPATH was deprecated (other resource ld.so.8.html).

So at this point there's no way to set directories with higher precedence wrt LD_LIBRARY_PATH?

[claudiofantacci]

So at this point there's no way to set directories with higher precedence wrt LD_LIBRARY_PATH?

You can set the linker flag --disable-new-dtags to use RPATH in place of RUNPATH.

Just to understand the problem better, why do you need an higher precedence than LD_LIBRARY_PATH? Do you feel you might have some conflicts? This should not happen anyway.

[diegoferigo]

You can set the linker flag --disable-new-dtags to use RPATH in place of RUNPATH.

Should this be done by the add_install_rpath_support macro of YCM?

My case in quite simple. I have an installed project that is bugged. As a developer, I fix the problem in the build tree of the repository (without installing it). I run the unit tests through ctest but the libraries loaded by the dynamic linker come from the installed tree (still bugged) since its lib directory is inside LD_LIBRARY_PATH and has more preference than the build tree. Hence, the tests keep failing unless I install the project.

[claudiofantacci]

since its lib directory is inside LD_LIBRARY_PATH and has more preference than the build tree

I don't know why this should be happening. I guess you are using plugins, am I right?

[claudiofantacci]

Should this be done by the add_install_rpath_support macro of YCM?

I don't feel like this is clean at all. I would never set a linker flag from within a script. @drdanz what's your opinion about this issue?

[diegoferigo]

I guess you are using plugins, am I right?

No I don't. The libraries that are wrongly used by the dynamic linker are regular shared libraries. If I have understood well, the problem is that LD_LIBRARY_PATH has the precedence on RUNPATH. The use case of RPATH was to provide support to get over LD_LIBRARY_PATH, but this behaviour created problems and it was deprecated.

[claudiofantacci]

No I don't. The libraries that are wrongly used by the dynamic linker are regular shared libraries. If I have understood well, the problem is that LD_LIBRARY_PATH has the precedence on RUNPATH. The use case of RPATH was to provide support to get over LD_LIBRARY_PATH, but this behaviour created problems and it was deprecated.

Yes, and infact under macOS this has never be the case beacuse DYLD_LIBRARY_PATH has higher precedence than RPATH.

Sorry for insisting about this: why are you setting LD_LIBRARY_PATH? Is it really necessary? I always try to avoid that. Since you are working from withing the build folder, the binaries should link the libraires inside the build folder as well. Did you try compiling disabling RPAHT?

[traversaro]

Yes, I also suspect as well that the use of LD_LIBRARY_PATH is the fundamental issue here.

See the related issues:

• Add <prefix>/lib/rtf to LD_LIBRARY_PATH ? robotology-superbuild#95
• If a user is configuring its enviroments following superbuild documentation, test executables will link to installed libraries instead of build ones  robotology-superbuild#97

[diegoferigo]

Actually the issue I had is described in robotology/robotology-superbuild#97.

Sorry for insisting about this: why are you setting LD_LIBRARY_PATH? Is it really necessary?

If you have any plugins in <installprefix>/lib, this folder must be present in LD_LIBRARY_PATH. I check and in my case I do not have any plugin there, so I can safely remove this folder from LD_LIBRARY_PATH.

At this point we should remember that is a good practice to install plugins in their own folder, and avoid with all means to install them in the regular ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}.

Unfortunately there are cases where we cannot avoid it. I do not use python bindings but I remember that in robotology/robotology-superbuild#82 we were exactly in the previous situation. I don't know if this is still necessary or if we should find a workaround.

[traversaro]

I do not use python bindings but I remember that in robotology/robotology-superbuild#82 we were exactly in the previous situation. I don't know if this is still necessary or if we should find a workaround.

To be honest I do not remember the exact issue as well. LD_LIBRARY_PATH was probably a workaround also in that case.

[traversaro]

At this point we should remember that is a good practice to install plugins in their own folder, and avoid with all means to install them in the regular ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}.

I am not sure about that. We have always installed Gazebo plugins in ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}, without any major problem (but in that case the plugins can be found using GAZEBO_PLUGIN_PATH).

[diegoferigo]

Unfortunately due to robotology/robotology-superbuild#82 we cannot solve robotology/robotology-superbuild#97, and hence the problem discussed in this issue since it is related to the install path of the superbuild.

I am not sure about that. We have always installed Gazebo plugins in ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}, without any major problem (but in that case the plugins can be found using GAZEBO_PLUGIN_PATH).

Yes probably we can think of it as a safe expection since it does not require setting LD_LIBRARY_PATH.

I think we can close this issue here in YCM since we cleared yet another time the RPATH situation, we will discuss and fix existing problems based on the specific case. Feel free to reopen it and add further details if needed.