Undefined reference error in boost when compiling the latest

[sammy17]

I am getting the following error when I try to compile latest

/share/pkg.7/gcc/9.3.0/install/bin/g++ -L. -Wl,--export-dynamic -L/usr/lib64  -Wl,-rpath,/projectnb2/risc-v/riscv-isa-sim/build/lib  -o spike-dasm spike-dasm.o  libspike_dasm.a  libriscv.a  libdisasm.a -lpthread -ldl -lboost_regex -lboost_system -lpthread  -lboost_system-mt -lboost_regex-mt
libriscv.a(interactive.o): In function `boost::cpp_regex_traits<char>::transform[abi:cxx11](char const*, char const*) const':
/usr/include/boost/regex/v4/cpp_regex_traits.hpp:919: undefined reference to `boost::re_detail::cpp_regex_traits_implementation<char>::transform[abi:cxx11](char const*, char const*) const'
libriscv.a(interactive.o): In function `boost::cpp_regex_traits<char>::transform_primary[abi:cxx11](char const*, char const*) const':
/usr/include/boost/regex/v4/cpp_regex_traits.hpp:923: undefined reference to `boost::re_detail::cpp_regex_traits_implementation<char>::transform_primary[abi:cxx11](char const*, char const*) const'

As shown, the error is coming when spike-dasm is being linked. Even if I spike spike-dasm, same error comes when linking spike-main.

Please let me know if there is any way to identify the root cause and resolve the issue.

[aswaterman]

As a workaround, try using the --without-boost argument to configure.

[aswaterman]

Actually, --without-boost --without-boost-asio --without-boost-regex

[aswaterman]

cc @emelcher — it appears the autoconf checks for Boost aren’t sufficient. (This is part of the reason I was leery of incorporating Boost into Spike to begin with…)

[sammy17]

It compiled successfully when --without-boost --without-boost-asio --without-boost-regex options were used to configure. Thanks!

[scottj97]

Please leave this bug open until the issue is fixed without any workaround needed. This has come up many times before (here
and here and #820).

[scottj97]

I have observed some strange behavior myself. Without any boost installation, my config.log shows:

configure:5354: We could not detect the boost libraries (version 1.53 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation

But config.status is still setting a (nonexistent) include path:

S["BOOST_REGEX_LIB"]=""
S["BOOST_ASIO_LIB"]=""
S["BOOST_LDFLAGS"]="-L/lib64"
S["BOOST_CPPFLAGS"]="-I/include/boost-0"

Even more strange: if I set $BOOST_ROOT to point to my installation of Boost 1.67, configure still sets all the same things. It can't find the library but still sets this nonexistent include path.

[emelcher]

I found out how to install and use, from home office, virtual machines in the university computer.
Now I will work at creating several VMs so I can reproduce each one of the problems reported.
Each time I come up with a solution, I will test it in all the different configurations, until I come up with a set of solutions that works in all of them, including the three physical computers I have been using for testing.
I will report back to You before making any PR, by the end of this week or sooner.

[emelcher]

The problem #834 (comment) has been reproduced and solved.
In order to reproduce #820, #829, and #74, a MacOS VM was created and spike compiled successfully without boost. Next is installing boost.
For this issue #834, found monero-project/monero#5449 which relates to two installations of boost on the same computer. A CentOS8 VM has been created with boost compiled from source to a completely non-standard location and spike was compiled successfully --with-boost=. Next is compiling to /usr/local and installing another boost using yum, to try reproduce the issue.
Will report on progress next Wednesday.

[aswaterman]

Thanks for following up!

[emelcher]

Installed boost in MacOS and encountered a problem different from #820, #829, and #71.
(In my last comment I erroneously refered to #74 instead of #71)
Undefined symbols for architecture x86_64:
"_fdt_header_size", referenced from:
_fdt_check_header in libfdt.a(fdt.o)
(maybe you meant: fdt_header_size)

Confirmed that different versions of boost, i.e. 1.53 installed in /usr/local and 1.74 in /usr, lead to linker error. @sammy17 could You please check if there are multiple installations of boost on Your computer ?

[emelcher]

Found out that in MacOS there is a /usr/local/lib/libfdt.a installed. In Linux, this library is installed as libfdt.so.1
Building riscv-isa-sim creates another libfdt.a
When the library path /usr/local/lib is needed for linking, the wrong libfdt.a gets linked in MacOS.

[aswaterman]

Interesting. So I guess we need to change our Makefiles to link our locally built libraries explicitly: e.g., ./libfdt.a instead of -lfdt?

[emelcher]

I agree. However, in the failing command line I see libftd.a instead of -lfdt, so changing libftd.a to ./libfdt.a should help. Unfortunately, doing a copy-and-paste of the failing command line and making this substitution does not solve the problem. Looking further I found /usr/local/include/fdt.hand /usr/local/include/libfdt.h which might get mixed up with riscv-isa-sim/fdt/fdt.h and riscv-isa-sim/fdt/libfdt.h because riscv-isa-sim/fdt/fdt.c references #include <fdt.h> and #include <libfdt.h> instead of #include "fdt.h" and #include "libfdt.h".
So I renamed all these conflicting files under /usr/local and then successfully built spike with boost.

Generally speaking, one might consider doing regression tests. A computer that is not being used at night could start a sequence of different VMs with different O.S.s and configuration variants and automatically pull and compile spike in each one, with different configure options, run it using the --debug-file option and compare the output to a pre-recorded golden output. Any difference resulting from this comparison would be mailed to someone for further investigation.
This should make bugs appear before a user detects them and also tends to avoid accumulation of bugs.
If You find this idea interesting I could start to work on it.

Indendent from regression test, still need to reproduce #820, #829, and #71 and find a clean solution for the fdt issue.

[aswaterman]

I’m absolutely on board with doing better regression testing. I set up a CI flow a couple years ago, but then the CI provider (travis) stopped offering their free service. I haven’t taken the time to migrate the flow to a new CI provider. I’ll look into that soon.

Unfortunately, it’s substantially more difficult to do multi-platform regression testing, as we’d need to do to catch build issues like this one.

It sounds like you’re zeroing in on a fix to the FDT+boost problem, though. That’s good!

[timsifive]

Since then github has added a feature called "github actions." I'm using it in OpenOCD to verify that builds run, and in a few spec repos to check the PDF build. It works better than travis did IMO.

[aswaterman]

Thanks, @timsifive; that's probably what we'll do.

[emelcher]

I noticed that my MacOS Catalina was outdated because issue reports mention Big Sur,
so I updated, but landed directly in Monterey.

#820 has been reproduced and found to be related to a macro definition in riscv/decode.h:177
The name of this macro is the single letter P. This is probably not an ideal solution,
considering that there are no namespaces for macros, so conflict with another macro name,
or, as in this case, a template type placeholder name, is not unlikely.
The said macro defines P as (*p) and then uses it as P.some_processor_t_property.
Perhaps using p-> instead of P. would be better and make the definition of that macro
unnecessary?

#829 apparently is that FDT problem which I have already encountered.

#71 apparently is the same as #820.

So, all issues seem to be accounted for and reproduced, so I can start fixing, being able
to test if the fix is effective.

Maybe I should start with the FDT issue, then the P issue and last the config.status issue ?

[aswaterman]

I'll handle the P issue, because I'd like to find a less disruptive fix (if one exists). The rest of your plan sounds good.

[aswaterman]

#820 fixed again in #854

[aswaterman]

Since #856 has also been merged, can we call this issue resolved?

[emelcher]

As in #834 (comment), I suspect that this issue is due to two different versions of boost installed in the user's computer, one installed under /usr/local and another under /usr, but we seem to have no confirmation from @sammy17 if this is the case or not.
Judging from the error message initially reported for this issue, the FDT fix probably does not resolve this issue.

[sequencer]

I also encountered this issue today:

configure:5866: result: no
configure:5946: checking for library containing dlopen
configure:5977: clang++ -o conftest -g -O2   -Wl,--export-dynamic conftest.cpp -lpthread  >&5
/nix/store/cdm6zywd51mbabxhklsixwcskv4n70s3-binutils-2.35.2/bin/ld: /run/user/1000/conftest-25f9fb.o: in function `main':
/home/user/playground/out/spike/compile/dest/conftest.cpp:41: undefined reference to `dlopen'

I was trying to compile under nix environment, however with -L/lib64 command added by BOOST_LDFLAGS, configure directly query the LSB from linux.
adding

 --without-boost --without-boost-asio --without-boost-regex

works for me.

[Leon924]

my linux version is RHEL6.10

I encounter boostlib problem like this. The system already has the version of boostlib 1.41. But I need the version of 1.53 then install it in my homedir. So how should I setup environment path or something else to make 1.53 version used prior to 1.41?