Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various CMake + ENABLE_CUDA Fixes #415

Merged
merged 2 commits into from
Aug 21, 2023
Merged

Various CMake + ENABLE_CUDA Fixes #415

merged 2 commits into from
Aug 21, 2023

Conversation

wavefunction91
Copy link

Fix various CMake bugs stemming from ENABLE_CUDA

  1. LibRETT used to fail with parallel build + GNU Makefiles (ala the Umpire build bug and fix). This fix follows the prescription used in Fix parallel compilation on Umpire #340.
  2. Unsure how this ever worked, but for some reason (at least for CMake 3.25+), you need to explicitly link to the CUDA::cudart TARGET, because apparently it doesn't get propagated via other dependencies... Unclear why CUDA::cublas doesn't propagate it, but who knows. Before this fix, I get the following on my system (CUDA 11.17 CMake 3.25) when building and linking ta_test
[ 88%] Linking CXX executable ta_test
cd /home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/tests && /home/dbwy/Software/BuildSystems/CMake/3.25.0/install/bin/cmake -E cmake_link_script CMakeFiles/ta_test.dir/link.txt --verbose=1
/usr/bin/c++ -g -Wl,-rpath -Wl,/home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/objdir/../install/lib -Wl,--enable-new-dtags CMakeFiles/ta_test.dir/ta_test.cpp.o CMakeFiles/ta_test.dir/range1.cpp.o CMakeFiles/ta_test.dir/range.cpp.o CMakeFiles/ta_test.dir/type_traits.cpp.o CMakeFiles/ta_test.dir/tensor.cpp.o CMakeFiles/ta_test.dir/tensor_of_tensor.cpp.o CMakeFiles/ta_test.dir/tensor_tensor_view.cpp.o CMakeFiles/ta_test.dir/tensor_shift_wrapper.cpp.o CMakeFiles/ta_test.dir/tiled_range1.cpp.o CMakeFiles/ta_test.dir/tiled_range.cpp.o CMakeFiles/ta_test.dir/blocked_pmap.cpp.o CMakeFiles/ta_test.dir/round_robin_pmap.cpp.o CMakeFiles/ta_test.dir/hash_pmap.cpp.o CMakeFiles/ta_test.dir/cyclic_pmap.cpp.o CMakeFiles/ta_test.dir/replicated_pmap.cpp.o CMakeFiles/ta_test.dir/dense_shape.cpp.o CMakeFiles/ta_test.dir/sparse_shape.cpp.o CMakeFiles/ta_test.dir/distributed_storage.cpp.o CMakeFiles/ta_test.dir/tensor_impl.cpp.o CMakeFiles/ta_test.dir/array_impl.cpp.o CMakeFiles/ta_test.dir/index_list.cpp.o CMakeFiles/ta_test.dir/bipartite_index_list.cpp.o CMakeFiles/ta_test.dir/dist_array.cpp.o CMakeFiles/ta_test.dir/conversions.cpp.o CMakeFiles/ta_test.dir/eigen.cpp.o CMakeFiles/ta_test.dir/dist_op_dist_cache.cpp.o CMakeFiles/ta_test.dir/dist_op_group.cpp.o CMakeFiles/ta_test.dir/dist_op_communicator.cpp.o CMakeFiles/ta_test.dir/tile_op_noop.cpp.o CMakeFiles/ta_test.dir/tile_op_scal.cpp.o CMakeFiles/ta_test.dir/dist_eval_array_eval.cpp.o CMakeFiles/ta_test.dir/dist_eval_unary_eval.cpp.o CMakeFiles/ta_test.dir/tile_op_add.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_add.cpp.o CMakeFiles/ta_test.dir/tile_op_subt.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_subt.cpp.o CMakeFiles/ta_test.dir/dist_eval_binary_eval.cpp.o CMakeFiles/ta_test.dir/tile_op_mult.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_mult.cpp.o CMakeFiles/ta_test.dir/tile_op_contract_reduce.cpp.o CMakeFiles/ta_test.dir/reduce_task.cpp.o CMakeFiles/ta_test.dir/proc_grid.cpp.o CMakeFiles/ta_test.dir/dist_eval_contraction_eval.cpp.o CMakeFiles/ta_test.dir/expressions.cpp.o CMakeFiles/ta_test.dir/expressions_sparse.cpp.o CMakeFiles/ta_test.dir/expressions_complex.cpp.o CMakeFiles/ta_test.dir/expressions_btas.cpp.o CMakeFiles/ta_test.dir/expressions_mixed.cpp.o CMakeFiles/ta_test.dir/foreach.cpp.o CMakeFiles/ta_test.dir/solvers.cpp.o CMakeFiles/ta_test.dir/initializer_list.cpp.o CMakeFiles/ta_test.dir/diagonal_array.cpp.o CMakeFiles/ta_test.dir/retile.cpp.o CMakeFiles/ta_test.dir/tot_dist_array_part1.cpp.o CMakeFiles/ta_test.dir/tot_dist_array_part2.cpp.o CMakeFiles/ta_test.dir/random.cpp.o CMakeFiles/ta_test.dir/trace.cpp.o CMakeFiles/ta_test.dir/tot_expressions.cpp.o CMakeFiles/ta_test.dir/annotation.cpp.o CMakeFiles/ta_test.dir/contraction_helpers.cpp.o CMakeFiles/ta_test.dir/s_t_t_contract_.cpp.o CMakeFiles/ta_test.dir/t_t_t_contract_.cpp.o CMakeFiles/ta_test.dir/t_s_t_contract_.cpp.o CMakeFiles/ta_test.dir/einsum.cpp.o CMakeFiles/ta_test.dir/linalg.cpp.o CMakeFiles/ta_test.dir/cp.cpp.o CMakeFiles/ta_test.dir/librett.cpp.o CMakeFiles/ta_test.dir/expressions_cuda_um.cpp.o CMakeFiles/ta_test.dir/tensor_um.cpp.o -o ta_test   -L/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/umpire-build/lib  -L/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/librett-build/src  -Wl,-rpath,/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/src:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/madness-build/src/madness/world:/home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/lapackpp-build:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/blaspp-build:/home/dbwy/Software/Boost/gcc/9.4.0/1.80.0/install/lib:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/umpire-build/lib:/usr/local/cuda-11.7/lib64:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/librett-build/src:/home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-libflame/lib/LP64:/home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-blis/lib/LP64 ../src/libtiledarray.so ../_deps/madness-build/src/madness/world/libMADworld.so /home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib/libmpicxx.so /home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib/libmpi.so -lpthread ../_deps/lapackpp-build/liblapackpp.so ../_deps/blaspp-build/libblaspp.so /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so /usr/lib/x86_64-linux-gnu/libpthread.so /home/dbwy/Software/Boost/gcc/9.4.0/1.80.0/install/lib/libboost_serialization.so.1.80.0 -lumpire /usr/local/cuda-11.7/lib64/libcublas.so /usr/local/cuda-11.7/lib64/libcublasLt.so /usr/local/cuda-11.7/lib64/libculibos.a /usr/local/cuda-11.7/lib64/libnvToolsExt.so -lrett /home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-libflame/lib/LP64/libflame.so /home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-blis/lib/LP64/libblis.so -lm 
/usr/bin/ld: CMakeFiles/ta_test.dir/dist_array.cpp.o: in function `array_suite::serialization_by_tile::test_method()':
/home/dbwy/Development/NWChemEx/nwx_update_ta/tiledarray/tests/dist_array.cpp:652: warning: the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp'
/usr/bin/ld: CMakeFiles/ta_test.dir/expressions_cuda_um.cpp.o: undefined reference to symbol 'cudaStreamSynchronize@@libcudart.so.11.0'

@wavefunction91 wavefunction91 requested a review from evaleev August 18, 2023 16:27
@evaleev evaleev merged commit efba9d1 into master Aug 21, 2023
@evaleev evaleev deleted the dbwy/bugfix/cmake_cuda branch August 21, 2023 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants