Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Visualizer and Collision detection glue_to_surface mode causes segmentation faults #4663

Closed
itischler opened this issue Feb 6, 2023 · 3 comments · Fixed by #4686
Closed

Comments

@itischler
Copy link
Contributor

itischler commented Feb 6, 2023

The collision detection mode "glue_to_surface" leads the visualizer to throw segmetation fault and other memory errors. This occurs random even if there is no collision happening.

from espressomd.visualization import openGLLive
from espressomd.interactions import HarmonicBond
import espressomd.collision_detection
from threading import Thread

system = espressomd.System(box_l = [50,50,50])
system.time_step = 0.01

bond = HarmonicBond(k=10.0, r_cut=2.0, r_0=1.0)
system.part.add(pos=[25,25,25], type=1)
system.bonded_inter.add(bond)
system.thermostat.set_langevin(kT=1.0, seed=1, gamma=1.0)

system.collision_detection.set_params(mode="glue_to_surface",
                                      distance=1.0,
                                      bond_centers=bond,
                                      bond_vs=bond,
                                      part_type_to_be_glued=1,
                                      part_type_to_attach_vs_to=1,
                                      part_type_vs=1,
                                      part_type_after_glueing=1,
                                      distance_glued_particle_to_vs=0.5)

visualizer = openGLLive(system)

def integrate():
  for i in range(10000):
    system.integrator.run(100)
    visualizer.update()

t=Thread(target=integrate)
t.daemon=True
t.start()
visualizer.start()
@RudolfWeeber
Copy link
Contributor

I'd guess (without having checked) that the visualizer doesn't take the change in particle number (or type) gracefully.
The gluie_to_surface scheme places a virtual site on a particle and changes the type of a second one.

@jngrad
Copy link
Member

jngrad commented Mar 8, 2023

Most likely undefined behavior: the segmentation fault doesn't happen when running the script inside of GDB, or without the visualizer, or with ./pypresso -m trace -t mwe.py. Seems to only happen when we broadcast an empty vector.

@jngrad
Copy link
Member

jngrad commented Mar 8, 2023

Spoke too soon: it also happens on non-empty vectors.

The bug boils down to:

#include <boost/mpi/collectives.hpp>
#include <boost/serialization/vector.hpp>
#include <vector>

int main() {
  boost::mpi::environment env;
  boost::mpi::communicator world;
  for (long i = 0; i < 1000000; ++i) {
    std::vector<int> test = {1};
    boost::mpi::broadcast(world, test, 0); // fatal error
  }
  return 0;
}

It is however not reproducible without the ESPResSo visualizer. Running the MWE doesn't trigger the bug:

mpicxx mwe.cpp -lboost_mpi -lboost_serialization
mpiexec -n 1 ./a.out

To make the investigation simpler, inject the MWE in ESPResSo with this patch:

diff --git a/src/core/collision.cpp b/src/core/collision.cpp
index 41f47f6d7..e3564dba1 100644
--- a/src/core/collision.cpp
+++ b/src/core/collision.cpp
@@ -397,8 +397,8 @@ void glue_to_surface_bind_part_to_vs(const Particle *const p1,
 
 std::vector<CollisionPair> gather_global_collision_queue() {
   std::vector<CollisionPair> res = local_collision_queue;
-  Utils::Mpi::gather_buffer(res, comm_cart);
-  boost::mpi::broadcast(comm_cart, res, 0);
+  std::vector<int> test = {1};
+  boost::mpi::broadcast(comm_cart, test, 0);
 
   return res;
 }

Several different fatal exception traces from an ASAN+UBSAN build using the injected MWE are reproduced below.

Since the issue happens inside the Boost MPI code but isn't reproducible outside of ESPResSo, the Boost team cannot act on it. As a workaround, we can skip the fragile broadcast when the MPI world has size 1, which is the most common use case when running the visualizer.

stack traces and sanitizer reports (click to unroll)
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1784320==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x1538f7acb9e1 bp 0x153861be3dd0 sp 0x153861be3b10 T68)
==1784320==The signal is caused by a READ memory access.
==1784320==Hint: address points to the zero page.
AddressSanitizer:DEADLYSIGNAL
    #0 0x1538f7acb9e1 in mca_mpool_base_tree_item_get (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7e9e1) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12)
    #1 0x1538f7acbb08 in mca_mpool_base_alloc (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7eb08) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12)
    #2 0x1538f2b167c3 in PMPI_Alloc_mem (/lib/x86_64-linux-gnu/libmpi.so.40+0x687c3) (BuildId: 686243cd1f842a09ffad60eb3af133aa3fa0f847)
    #3 0x1538f52d6164 in boost::mpi::allocator<char>::allocate(unsigned long, void const*) /usr/include/boost/mpi/allocator.hpp:139:5
    #4 0x1538f52d6164 in std::allocator_traits<boost::mpi::allocator<char> >::allocate(boost::mpi::allocator<char>&, unsigned long) /usr/include/c++/10/bits/alloc_traits.h:314:20
    #5 0x1538f52d50a7 in std::_Vector_base<char, boost::mpi::allocator<char> >::_M_allocate(unsigned long) /usr/include/c++/10/bits/stl_vector.h:346:20
    #6 0x1538f52d50a7 in void std::vector<char, boost::mpi::allocator<char> >::_M_range_insert<char const*>(__gnu_cxx::__normal_iterator<char*, std::vector<char, boost::mpi::allocator<char> > >, char const*, char const*, std::forward_iterator_tag) /usr/include/c++/10/bits/vector.tcc:769:29
    #7 0x1538f52d4e10 in void std::vector<char, boost::mpi::allocator<char> >::_M_insert_dispatch<char const*>(__gnu_cxx::__normal_iterator<char*, std::vector<char, boost::mpi::allocator<char> > >, char const*, char const*, std::__false_type) /usr/include/c++/10/bits/stl_vector.h:1665:4
    #8 0x1538f52d4e10 in __gnu_cxx::__normal_iterator<char*, std::vector<char, boost::mpi::allocator<char> > > std::vector<char, boost::mpi::allocator<char> >::insert<char const*, void>(__gnu_cxx::__normal_iterator<char const*, std::vector<char, boost::mpi::allocator<char> > >, char const*, char const*) /usr/include/c++/10/bits/stl_vector.h:1383:4
    #9 0x1538f52d4c32 in boost::mpi::binary_buffer_oprimitive::save_impl(void const*, int) /usr/include/boost/mpi/detail/binary_buffer_oprimitive.hpp:100:15
    #10 0x1538f52d8a1e in void boost::mpi::binary_buffer_oprimitive::save<int>(int const&) /usr/include/boost/mpi/detail/binary_buffer_oprimitive.hpp:84:7
    #11 0x1538f52d8a1e in void boost::mpi::packed_oarchive::save_override<int>(int const&, mpl_::bool_<true>) /usr/include/boost/mpi/packed_oarchive.hpp:104:17
    #12 0x1538f52d8a1e in void boost::mpi::packed_oarchive::save_override<int>(int const&) /usr/include/boost/mpi/packed_oarchive.hpp:112:5
    #13 0x1538f52d3d87 in boost::mpi::packed_oarchive& boost::archive::detail::interface_oarchive<boost::mpi::packed_oarchive>::operator<<<int>(int const&) /usr/include/boost/archive/detail/interface_oarchive.hpp:70:23
    #14 0x1538f349ca82 in auto void Communication::MpiCallbacks::call<int&, int&>(int, int&, int&) const::'lambda'(auto&&)::operator()<int&>(auto&&) const /work/jgrad/espresso/src/core/MpiCallbacks.hpp:550:42
    #15 0x1538f349ca82 in void Utils::detail::for_each_impl<std::tuple<int&, int&>, void Communication::MpiCallbacks::call<int&, int&>(int, int&, int&) const::'lambda'(auto&&), 0ul, 1ul>(void Communication::MpiCallbacks::call<int&, int&>(int, int&, int&) const::'lambda'(auto&&)&&, auto, std::integer_sequence<unsigned long, 0ul, 1ul>) /work/jgrad/espresso/src/utils/include/utils/tuple.hpp:39:35
    #16 0x1538f349c96f in void Utils::for_each<void Communication::MpiCallbacks::call<int&, int&>(int, int&, int&) const::'lambda'(auto&&), std::tuple<int&, int&> >(auto&&, std::tuple<int&, int&>&&) /work/jgrad/espresso/src/utils/include/utils/tuple.hpp:53:3
    #17 0x1538f349c648 in void Communication::MpiCallbacks::call<int&, int&>(int, int&, int&) const /work/jgrad/espresso/src/core/MpiCallbacks.hpp:550:5
    #18 0x1538f349c275 in std::remove_reference<decltype(fp0(std::declval<int>(), std::declval<int>()))>::type Communication::MpiCallbacks::call<std::plus<int>, int, int, int>(Communication::Result::Reduction, std::plus<int>, int (*)(int, int), int, int) const /work/jgrad/espresso/src/core/MpiCallbacks.hpp:607:5
    #19 0x1538f3486fb1 in auto mpi_call<Communication::Result::Reduction, std::plus<int>, int, int, int, int&, int&>(Communication::Result::Reduction, std::plus<int>&&, int (*)(int, int), int&, int&) /work/jgrad/espresso/src/core/communication.hpp:128:40
    #20 0x1538f34856d8 in mpi_integrate(int, int) /work/jgrad/espresso/src/core/integrate.cpp:500:10
    #21 0x1538f34856d8 in python_integrate(int, bool, bool) /work/jgrad/espresso/src/core/integrate.cpp:462:9
    #22 0x1537d4ad4999 in __pyx_f_10espressomd_9integrate__integrate(int, bool, int) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:3833:19
    #23 0x1537d4ad4999 in __pyx_pf_10espressomd_9integrate_10Integrator_run(_object*, _object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:3089:10
    #24 0x1537d4ad16f6 in __pyx_pw_10espressomd_9integrate_10Integrator_1run(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2811:13
    #25 0x1537a35c7854 in __Pyx_CyFunction_CallMethod(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21365:16
    #26 0x1537a35c4b82 in __Pyx_CyFunction_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21410:12
    #27 0x1537a35c4b82 in __Pyx_CyFunction_CallAsMethod(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21431:18
    #28 0x55f0123d27da in _PyObject_MakeTpCall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:215:18
    #29 0x55f0123ea7bf in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:112:16
    #30 0x55f0123ea7bf in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:99:1
    #31 0x55f0123ea7bf in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:83:18
    #32 0x1537d4ac7bb6 in __Pyx_PyObject_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:5689:14
    #33 0x1537d4acc442 in __pyx_pf_10espressomd_9integrate_16IntegratorHandle_4run(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2073:15
    #34 0x1537d4acbab4 in __pyx_pw_10espressomd_9integrate_16IntegratorHandle_5run(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2038:13
    #35 0x1537a35c7854 in __Pyx_CyFunction_CallMethod(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21365:16
    #36 0x1537a35c4b82 in __Pyx_CyFunction_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21410:12
    #37 0x1537a35c4b82 in __Pyx_CyFunction_CallAsMethod(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21431:18
    #38 0x55f0123d27da in _PyObject_MakeTpCall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:215:18
    #39 0x55f0123ea66a in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:112:16
    #40 0x55f0123ea66a in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:99:1
    #41 0x55f0123ea66a in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:53:18
    #42 0x55f0123caa71 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #43 0x55f0123caa71 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #44 0x55f0123caa71 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #45 0x55f0123caa71 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4181:23
    #46 0x55f0123dc3ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #47 0x55f0123dc3ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #48 0x55f0123dc3ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #49 0x55f0123c73af in do_call_core /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5943:12
    #50 0x55f0123c73af in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4277:22
    #51 0x55f0123dc3ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #52 0x55f0123dc3ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #53 0x55f0123dc3ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #54 0x55f0123c5149 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #55 0x55f0123c5149 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #56 0x55f0123c5149 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #57 0x55f0123c5149 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4198:23
    #58 0x55f0123dc3ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #59 0x55f0123dc3ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #60 0x55f0123dc3ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #61 0x55f0123c5149 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #62 0x55f0123c5149 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #63 0x55f0123c5149 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #64 0x55f0123c5149 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4198:23
    #65 0x55f0123ea5f0 in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #66 0x55f0123ea5f0 in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #67 0x55f0123ea5f0 in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #68 0x55f0123ea5f0 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #69 0x55f0123ea5f0 in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:61:20
    #70 0x55f012517aa9 in thread_run /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Modules/_threadmodule.c:1090:21
    #71 0x55f01250dd37 in pythread_wrapper /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/thread_pthread.h:253:5
    #72 0x1538f9494b42 in start_thread nptl/./nptl/pthread_create.c:442:8
    #73 0x1538f95269ff  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7e9e1) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12) in mca_mpool_base_tree_item_get
Thread T68 created by T0 here:
    #0 0x1538f98b6c8c in __interceptor_pthread_create (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xb6c8c) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x55f01250d82a in PyThread_start_new_thread /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/thread_pthread.h:301:14

==1784320==ABORTING
=================================================================
AddressSanitizer:DEADLYSIGNAL
==1783770==ERROR: AddressSanitizer: heap-use-after-free on address 0x60200007baf0 at pc 0x14e6f4ed5f98 bp 0x14e6517e3050 sp 0x14e6517e3048
READ of size 1 at 0x60200007baf0 thread T68
    #0 0x14e6f4ed5f97 in boost::mpi::allocator<char>::construct(char*, char const&) /usr/include/boost/mpi/allocator.hpp:170:23
    #1 0x14e6f4ed5f97 in std::enable_if<__and_<std::allocator_traits<boost::mpi::allocator<char> >::__construct_helper<char, char>::type>::value, void>::type std::allocator_traits<boost::mpi::allocator<char> >::_S_construct<char, char>(boost::mpi::allocator<char>&, char*, char&&) /usr/include/c++/10/bits/alloc_traits.h:247:8
    #2 0x14e6f4ed5f97 in decltype(_S_construct(fp, fp0, std::forward<char>(fp1))) std::allocator_traits<boost::mpi::allocator<char> >::construct<char, char>(boost::mpi::allocator<char>&, char*, char&&) /usr/include/c++/10/bits/alloc_traits.h:360:4
    #3 0x14e6f4ed5d07 in char* std::__uninitialized_copy_a<std::move_iterator<char*>, char*, boost::mpi::allocator<char> >(std::move_iterator<char*>, std::move_iterator<char*>, char*, boost::mpi::allocator<char>&) /usr/include/c++/10/bits/stl_uninitialized.h:311:6
    #4 0x14e6f4ed54c1 in char* std::__uninitialized_move_if_noexcept_a<char*, char*, boost::mpi::allocator<char> >(char*, char*, char*, boost::mpi::allocator<char>&) /usr/include/c++/10/bits/stl_uninitialized.h:346:14
    #5 0x14e6f4ed54c1 in void std::vector<char, boost::mpi::allocator<char> >::_M_range_insert<char const*>(__gnu_cxx::__normal_iterator<char*, std::vector<char, boost::mpi::allocator<char> > >, char const*, char const*, std::forward_iterator_tag) /usr/include/c++/10/bits/vector.tcc:774:11
    #6 0x14e6f4ed4e10 in void std::vector<char, boost::mpi::allocator<char> >::_M_insert_dispatch<char const*>(__gnu_cxx::__normal_iterator<char*, std::vector<char, boost::mpi::allocator<char> > >, char const*, char const*, std::__false_type) /usr/include/c++/10/bits/stl_vector.h:1665:4
    #7 0x14e6f4ed4e10 in __gnu_cxx::__normal_iterator<char*, std::vector<char, boost::mpi::allocator<char> > > std::vector<char, boost::mpi::allocator<char> >::insert<char const*, void>(__gnu_cxx::__normal_iterator<char const*, std::vector<char, boost::mpi::allocator<char> > >, char const*, char const*) /usr/include/c++/10/bits/stl_vector.h:1383:4
    #8 0x14e6f4ed4c32 in boost::mpi::binary_buffer_oprimitive::save_impl(void const*, int) /usr/include/boost/mpi/detail/binary_buffer_oprimitive.hpp:100:15
    #9 0x14e6f4ee72a3 in void boost::mpi::binary_buffer_oprimitive::save_array<int const>(boost::serialization::array_wrapper<int const> const&, unsigned int) /usr/include/boost/mpi/detail/binary_buffer_oprimitive.hpp:68:9
    #10 0x14e6f4ee7254 in void boost::serialization::array_wrapper<int const>::save<boost::mpi::packed_oarchive>(boost::mpi::packed_oarchive&, unsigned int) const /usr/include/boost/serialization/array_wrapper.hpp:73:10
    #11 0x14e6f4ee7254 in void boost::serialization::access::member_save<boost::mpi::packed_oarchive, boost::serialization::array_wrapper<int const> const>(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const> const&, unsigned int) /usr/include/boost/serialization/access.hpp:91:11
    #12 0x14e6f4ee7231 in boost::serialization::detail::member_saver<boost::mpi::packed_oarchive, boost::serialization::array_wrapper<int const> >::invoke(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const> const&, unsigned int) /usr/include/boost/serialization/split_member.hpp:43:13
    #13 0x14e6f4ee7231 in void boost::serialization::split_member<boost::mpi::packed_oarchive, boost::serialization::array_wrapper<int const> >(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const>&, unsigned int) /usr/include/boost/serialization/split_member.hpp:69:5
    #14 0x14e6f4ee7231 in void boost::serialization::array_wrapper<int const>::serialize_optimized<boost::mpi::packed_oarchive>(boost::mpi::packed_oarchive&, unsigned int, mpl_::bool_<true>) /usr/include/boost/serialization/array_wrapper.hpp:66:7
    #15 0x14e6f4ee71f7 in void boost::serialization::array_wrapper<int const>::serialize<boost::mpi::packed_oarchive>(boost::mpi::packed_oarchive&, unsigned int) /usr/include/boost/serialization/array_wrapper.hpp:91:7
    #16 0x14e6f4ee71f7 in void boost::serialization::access::serialize<boost::mpi::packed_oarchive, boost::serialization::array_wrapper<int const> >(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const>&, unsigned int) /usr/include/boost/serialization/access.hpp:116:11
    #17 0x14e6f4ee71b2 in void boost::serialization::serialize<boost::mpi::packed_oarchive, boost::serialization::array_wrapper<int const> >(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const>&, unsigned int) /usr/include/boost/serialization/serialization.hpp:68:5
    #18 0x14e6f4ee71b2 in void boost::serialization::serialize_adl<boost::mpi::packed_oarchive, boost::serialization::array_wrapper<int const> >(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const>&, unsigned int) /usr/include/boost/serialization/serialization.hpp:118:5
    #19 0x14e6f4ee71b2 in void boost::archive::detail::save_non_pointer_type<boost::mpi::packed_oarchive>::save_only::invoke<boost::serialization::array_wrapper<int const> >(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const> const&) /usr/include/boost/archive/detail/oserializer.hpp:246:13
    #20 0x14e6f4ee7186 in void boost::archive::detail::save_non_pointer_type<boost::mpi::packed_oarchive>::invoke<boost::serialization::array_wrapper<int const> >(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const> const&) /usr/include/boost/archive/detail/oserializer.hpp:315:9
    #21 0x14e6f4ee7186 in void boost::archive::save<boost::mpi::packed_oarchive, boost::serialization::array_wrapper<int const> const>(boost::mpi::packed_oarchive&, boost::serialization::array_wrapper<int const> const&) /usr/include/boost/archive/detail/oserializer.hpp:539:5
    #22 0x14e6f4ee7186 in void boost::archive::detail::common_oarchive<boost::mpi::packed_oarchive>::save_override<boost::serialization::array_wrapper<int const> const>(boost::serialization::array_wrapper<int const> const&) /usr/include/boost/archive/detail/common_oarchive.hpp:71:9
    #23 0x14e6f4ee7106 in void boost::mpi::packed_oarchive::save_override<boost::serialization::array_wrapper<int const> >(boost::serialization::array_wrapper<int const> const&, mpl_::bool_<false>) /usr/include/boost/mpi/packed_oarchive.hpp:97:56
    #24 0x14e6f4ee7075 in void boost::mpi::packed_oarchive::save_override<boost::serialization::array_wrapper<int const> >(boost::serialization::array_wrapper<int const> const&) /usr/include/boost/mpi/packed_oarchive.hpp:112:5
    #25 0x14e6f4ee6ce7 in boost::mpi::packed_oarchive& boost::archive::detail::interface_oarchive<boost::mpi::packed_oarchive>::operator<<<boost::serialization::array_wrapper<int const> >(boost::serialization::array_wrapper<int const> const&) /usr/include/boost/archive/detail/interface_oarchive.hpp:70:23
    #26 0x14e6f4ee69cd in void boost::serialization::save<boost::mpi::packed_oarchive, int, std::allocator<int> >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> > const&, unsigned int, mpl_::bool_<true>) /usr/include/boost/serialization/vector.hpp:105:12
    #27 0x14e6f4ee67f7 in void boost::serialization::save<boost::mpi::packed_oarchive, int, std::allocator<int> >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> > const&, unsigned int) /usr/include/boost/serialization/vector.hpp:145:5
    #28 0x14e6f4ee67f7 in boost::serialization::free_saver<boost::mpi::packed_oarchive, std::vector<int, std::allocator<int> > >::invoke(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> > const&, unsigned int) /usr/include/boost/serialization/split_free.hpp:45:9
    #29 0x14e6f4ee658f in void boost::serialization::split_free<boost::mpi::packed_oarchive, std::vector<int, std::allocator<int> > >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> >&, unsigned int) /usr/include/boost/serialization/split_free.hpp:74:5
    #30 0x14e6f4ee658f in void boost::serialization::serialize<boost::mpi::packed_oarchive, int, std::allocator<int> >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> >&, unsigned int) /usr/include/boost/serialization/vector.hpp:176:5
    #31 0x14e6f4ee658f in void boost::serialization::serialize_adl<boost::mpi::packed_oarchive, std::vector<int, std::allocator<int> > >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> >&, unsigned int) /usr/include/boost/serialization/serialization.hpp:118:5
    #32 0x14e6f4ee658f in boost::archive::detail::oserializer<boost::mpi::packed_oarchive, std::vector<int, std::allocator<int> > >::save_object_data(boost::archive::detail::basic_oarchive&, void const*) const /usr/include/boost/archive/detail/oserializer.hpp:153:5
    #33 0x14e6f4ee3686 in void boost::archive::detail::save_non_pointer_type<boost::mpi::packed_oarchive>::save_standard::invoke<std::vector<int, std::allocator<int> > >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> > const&) /usr/include/boost/archive/detail/oserializer.hpp:258:16
    #34 0x14e6f4ee35f6 in void boost::archive::detail::save_non_pointer_type<boost::mpi::packed_oarchive>::save_conditional::invoke<std::vector<int, std::allocator<int> > >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> > const&) /usr/include/boost/archive/detail/oserializer.hpp:275:17
    #35 0x14e6f4ee35f6 in void boost::archive::detail::save_non_pointer_type<boost::mpi::packed_oarchive>::invoke<std::vector<int, std::allocator<int> > >(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> > const&) /usr/include/boost/archive/detail/oserializer.hpp:315:9
    #36 0x14e6f4ee35f6 in void boost::archive::save<boost::mpi::packed_oarchive, std::vector<int, std::allocator<int> > const>(boost::mpi::packed_oarchive&, std::vector<int, std::allocator<int> > const&) /usr/include/boost/archive/detail/oserializer.hpp:539:5
    #37 0x14e6f4ee35f6 in void boost::archive::detail::common_oarchive<boost::mpi::packed_oarchive>::save_override<std::vector<int, std::allocator<int> > const>(std::vector<int, std::allocator<int> > const&) /usr/include/boost/archive/detail/common_oarchive.hpp:71:9
    #38 0x14e6f4ee3576 in void boost::mpi::packed_oarchive::save_override<std::vector<int, std::allocator<int> > >(std::vector<int, std::allocator<int> > const&, mpl_::bool_<false>) /usr/include/boost/mpi/packed_oarchive.hpp:97:56
    #39 0x14e6f4ee34e5 in void boost::mpi::packed_oarchive::save_override<std::vector<int, std::allocator<int> > >(std::vector<int, std::allocator<int> > const&) /usr/include/boost/mpi/packed_oarchive.hpp:112:5
    #40 0x14e6f4ee3377 in boost::mpi::packed_oarchive& boost::archive::detail::interface_oarchive<boost::mpi::packed_oarchive>::operator<<<std::vector<int, std::allocator<int> > >(std::vector<int, std::allocator<int> > const&) /usr/include/boost/archive/detail/interface_oarchive.hpp:70:23
    #41 0x14e6f2fc2aeb in void boost::mpi::detail::broadcast_impl<std::vector<int, std::allocator<int> > >(boost::mpi::communicator const&, std::vector<int, std::allocator<int> >*, int, int, mpl_::bool_<false>) /usr/include/boost/mpi/collectives/broadcast.hpp:113:12
    #42 0x14e6f2fb66c2 in void boost::mpi::broadcast<std::vector<int, std::allocator<int> > >(boost::mpi::communicator const&, std::vector<int, std::allocator<int> >&, int) /usr/include/boost/mpi/collectives/broadcast.hpp:141:3
    #43 0x14e6f2fb66c2 in gather_global_collision_queue() /work/jgrad/espresso/src/core/collision.cpp:401:3
    #44 0x14e6f2fb7640 in handle_collisions() /work/jgrad/espresso/src/core/collision.cpp:495:27
    #45 0x14e6f30847b7 in integrate(int, int) /work/jgrad/espresso/src/core/integrate.cpp:380:7
    #46 0x14e6f3085dc0 in mpi_integrate_local(int, int) /work/jgrad/espresso/src/core/integrate.cpp:492:3
    #47 0x14e6f309c2fd in std::remove_reference<decltype(fp0(std::declval<int>(), std::declval<int>()))>::type Communication::MpiCallbacks::call<std::plus<int>, int, int, int>(Communication::Result::Reduction, std::plus<int>, int (*)(int, int), int, int) const /work/jgrad/espresso/src/core/MpiCallbacks.hpp:612:32
    #48 0x14e6f3086fb1 in auto mpi_call<Communication::Result::Reduction, std::plus<int>, int, int, int, int&, int&>(Communication::Result::Reduction, std::plus<int>&&, int (*)(int, int), int&, int&) /work/jgrad/espresso/src/core/communication.hpp:128:40
    #49 0x14e6f30856d8 in mpi_integrate(int, int) /work/jgrad/espresso/src/core/integrate.cpp:500:10
    #50 0x14e6f30856d8 in python_integrate(int, bool, bool) /work/jgrad/espresso/src/core/integrate.cpp:462:9
    #51 0x14e5d22b0999 in __pyx_f_10espressomd_9integrate__integrate(int, bool, int) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:3833:19
    #52 0x14e5d22b0999 in __pyx_pf_10espressomd_9integrate_10Integrator_run(_object*, _object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:3089:10
    #53 0x14e5d22ad6f6 in __pyx_pw_10espressomd_9integrate_10Integrator_1run(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2811:13
    #54 0x14e5a32aa854 in __Pyx_CyFunction_CallMethod(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21365:16
    #55 0x14e5a32a7b82 in __Pyx_CyFunction_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21410:12
    #56 0x14e5a32a7b82 in __Pyx_CyFunction_CallAsMethod(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21431:18
    #57 0x556c7e1757da in _PyObject_MakeTpCall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:215:18
    #58 0x556c7e18d7bf in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:112:16
    #59 0x556c7e18d7bf in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:99:1
    #60 0x556c7e18d7bf in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:83:18
    #61 0x14e5d22a3bb6 in __Pyx_PyObject_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:5689:14
    #62 0x14e5d22a8442 in __pyx_pf_10espressomd_9integrate_16IntegratorHandle_4run(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2073:15
    #63 0x14e5d22a7ab4 in __pyx_pw_10espressomd_9integrate_16IntegratorHandle_5run(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2038:13
    #64 0x14e5a32aa854 in __Pyx_CyFunction_CallMethod(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21365:16
    #65 0x14e5a32a7b82 in __Pyx_CyFunction_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21410:12
    #66 0x14e5a32a7b82 in __Pyx_CyFunction_CallAsMethod(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21431:18
    #67 0x556c7e1757da in _PyObject_MakeTpCall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:215:18
    #68 0x556c7e18d66a in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:112:16
    #69 0x556c7e18d66a in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:99:1
    #70 0x556c7e18d66a in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:53:18
    #71 0x556c7e16da71 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #72 0x556c7e16da71 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #73 0x556c7e16da71 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #74 0x556c7e16da71 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4181:23
    #75 0x556c7e17f3ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #76 0x556c7e17f3ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #77 0x556c7e17f3ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #78 0x556c7e16a3af in do_call_core /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5943:12
    #79 0x556c7e16a3af in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4277:22
    #80 0x556c7e17f3ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #81 0x556c7e17f3ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #82 0x556c7e17f3ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #83 0x556c7e168149 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #84 0x556c7e168149 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #85 0x556c7e168149 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #86 0x556c7e168149 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4198:23
    #87 0x556c7e17f3ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #88 0x556c7e17f3ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #89 0x556c7e17f3ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #90 0x556c7e168149 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #91 0x556c7e168149 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #92 0x556c7e168149 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #93 0x556c7e168149 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4198:23
    #94 0x556c7e18d5f0 in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #95 0x556c7e18d5f0 in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #96 0x556c7e18d5f0 in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #97 0x556c7e18d5f0 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #98 0x556c7e18d5f0 in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:61:20
    #99 0x556c7e2baaa9 in thread_run /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Modules/_threadmodule.c:1090:21
    #100 0x556c7e2b0d37 in pythread_wrapper /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/thread_pthread.h:253:5
    #101 0x14e6f9094b42 in start_thread nptl/./nptl/pthread_create.c:442:8
    #102 0x14e6f91269ff  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

0x60200007baf0 is located 0 bytes inside of 8-byte region [0x60200007baf0,0x60200007baf8)
freed by thread T0 here:
    #0 0x14e6f94cd542 in free (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xcd542) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x14e6f80cb97c in mca_mpool_base_free (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7e97c) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12)

previously allocated by thread T68 here:
    #0 0x14e6f94cd7ee in __interceptor_malloc (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xcd7ee) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x14e6f80cbb3f in mca_mpool_base_alloc (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7eb3f) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12)

Thread T68 created by T0 here:
    #0 0x14e6f94b6c8c in __interceptor_pthread_create (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xb6c8c) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x556c7e2b082a in PyThread_start_new_thread /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/thread_pthread.h:301:14

SUMMARY: AddressSanitizer: heap-use-after-free /usr/include/boost/mpi/allocator.hpp:170:23 in boost::mpi::allocator<char>::construct(char*, char const&)
Shadow bytes around the buggy address:
  0x0c0480007700: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x0c0480007710: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x0c0480007720: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x0c0480007730: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x0c0480007740: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
=>0x0c0480007750: fa fa fd fa fa fa fd fd fa fa 04 fa fa fa[fd]fa
  0x0c0480007760: fa fa 00 00 fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480007770: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480007780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480007790: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c04800077a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1783770==ABORTING
=================================================================
==1783101==ERROR: AddressSanitizer: attempting double-free on 0x6020003da0f0 in thread T68:
    #0 0x154dd46cd542 in free (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xcd542) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x154dd28cb99a in mca_mpool_base_free (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7e99a) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12)
    #2 0x154dcd923221 in PMPI_Free_mem (/lib/x86_64-linux-gnu/libmpi.so.40+0x75221) (BuildId: 686243cd1f842a09ffad60eb3af133aa3fa0f847)
    #3 0x154dd00d6e28 in boost::mpi::allocator<char>::deallocate(char*, unsigned long) /usr/include/boost/mpi/allocator.hpp:155:5
    #4 0x154dd00d6e28 in std::allocator_traits<boost::mpi::allocator<char> >::deallocate(boost::mpi::allocator<char>&, char*, unsigned long) /usr/include/c++/10/bits/alloc_traits.h:341:13
    #5 0x154dd00d965d in std::_Vector_base<char, boost::mpi::allocator<char> >::_M_deallocate(char*, unsigned long) /usr/include/c++/10/bits/stl_vector.h:354:4
    #6 0x154dd00d965d in std::_Vector_base<char, boost::mpi::allocator<char> >::~_Vector_base() /usr/include/c++/10/bits/stl_vector.h:335:2
    #7 0x154dd00d95f4 in std::vector<char, boost::mpi::allocator<char> >::~vector() /usr/include/c++/10/bits/stl_vector.h:683:7
    #8 0x154dce1c2db7 in boost::mpi::packed_oarchive::~packed_oarchive() /usr/include/boost/mpi/packed_oarchive.hpp:48:22
    #9 0x154dce1c2db7 in void boost::mpi::detail::broadcast_impl<std::vector<int, std::allocator<int> > >(boost::mpi::communicator const&, std::vector<int, std::allocator<int> >*, int, int, mpl_::bool_<false>) /usr/include/boost/mpi/collectives/broadcast.hpp:122:5
    #10 0x154dce1b66c2 in void boost::mpi::broadcast<std::vector<int, std::allocator<int> > >(boost::mpi::communicator const&, std::vector<int, std::allocator<int> >&, int) /usr/include/boost/mpi/collectives/broadcast.hpp:141:3
    #11 0x154dce1b66c2 in gather_global_collision_queue() /work/jgrad/espresso/src/core/collision.cpp:401:3
    #12 0x154dce1b7640 in handle_collisions() /work/jgrad/espresso/src/core/collision.cpp:495:27
    #13 0x154dce2847b7 in integrate(int, int) /work/jgrad/espresso/src/core/integrate.cpp:380:7
    #14 0x154dce285dc0 in mpi_integrate_local(int, int) /work/jgrad/espresso/src/core/integrate.cpp:492:3
    #15 0x154dce29c2fd in std::remove_reference<decltype(fp0(std::declval<int>(), std::declval<int>()))>::type Communication::MpiCallbacks::call<std::plus<int>, int, int, int>(Communication::Result::Reduction, std::plus<int>, int (*)(int, int), int, int) const /work/jgrad/espresso/src/core/MpiCallbacks.hpp:612:32
    #16 0x154dce286fb1 in auto mpi_call<Communication::Result::Reduction, std::plus<int>, int, int, int, int&, int&>(Communication::Result::Reduction, std::plus<int>&&, int (*)(int, int), int&, int&) /work/jgrad/espresso/src/core/communication.hpp:128:40
    #17 0x154dce2856d8 in mpi_integrate(int, int) /work/jgrad/espresso/src/core/integrate.cpp:500:10
    #18 0x154dce2856d8 in python_integrate(int, bool, bool) /work/jgrad/espresso/src/core/integrate.cpp:462:9
    #19 0x154ca97b0999 in __pyx_f_10espressomd_9integrate__integrate(int, bool, int) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:3833:19
    #20 0x154ca97b0999 in __pyx_pf_10espressomd_9integrate_10Integrator_run(_object*, _object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:3089:10
    #21 0x154ca97ad6f6 in __pyx_pw_10espressomd_9integrate_10Integrator_1run(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2811:13
    #22 0x154c7e38f854 in __Pyx_CyFunction_CallMethod(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21365:16
    #23 0x154c7e38cb82 in __Pyx_CyFunction_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21410:12
    #24 0x154c7e38cb82 in __Pyx_CyFunction_CallAsMethod(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21431:18
    #25 0x55bbd11bb7da in _PyObject_MakeTpCall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:215:18
    #26 0x55bbd11d37bf in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:112:16
    #27 0x55bbd11d37bf in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:99:1
    #28 0x55bbd11d37bf in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:83:18
    #29 0x154ca97a3bb6 in __Pyx_PyObject_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:5689:14
    #30 0x154ca97a8442 in __pyx_pf_10espressomd_9integrate_16IntegratorHandle_4run(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2073:15
    #31 0x154ca97a7ab4 in __pyx_pw_10espressomd_9integrate_16IntegratorHandle_5run(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/integrate.cpp:2038:13
    #32 0x154c7e38f854 in __Pyx_CyFunction_CallMethod(_object*, _object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21365:16
    #33 0x154c7e38cb82 in __Pyx_CyFunction_Call(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21410:12
    #34 0x154c7e38cb82 in __Pyx_CyFunction_CallAsMethod(_object*, _object*, _object*) /work/jgrad/espresso/build/src/python/espressomd/script_interface.cpp:21431:18
    #35 0x55bbd11bb7da in _PyObject_MakeTpCall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:215:18
    #36 0x55bbd11d366a in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:112:16
    #37 0x55bbd11d366a in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:99:1
    #38 0x55bbd11d366a in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:53:18
    #39 0x55bbd11b3a71 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #40 0x55bbd11b3a71 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #41 0x55bbd11b3a71 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #42 0x55bbd11b3a71 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4181:23
    #43 0x55bbd11c53ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #44 0x55bbd11c53ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #45 0x55bbd11c53ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #46 0x55bbd11b03af in do_call_core /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5943:12
    #47 0x55bbd11b03af in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4277:22
    #48 0x55bbd11c53ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #49 0x55bbd11c53ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #50 0x55bbd11c53ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #51 0x55bbd11ae149 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #52 0x55bbd11ae149 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #53 0x55bbd11ae149 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #54 0x55bbd11ae149 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4198:23
    #55 0x55bbd11c53ab in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #56 0x55bbd11c53ab in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #57 0x55bbd11c53ab in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #58 0x55bbd11ae149 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #59 0x55bbd11ae149 in PyObject_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:123:12
    #60 0x55bbd11ae149 in call_function /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5891:13
    #61 0x55bbd11ae149 in _PyEval_EvalFrameDefault /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:4198:23
    #62 0x55bbd11d35f0 in _PyEval_EvalFrame /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/internal/pycore_ceval.h:46:12
    #63 0x55bbd11d35f0 in _PyEval_Vector /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/ceval.c:5065:24
    #64 0x55bbd11d35f0 in _PyFunction_Vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/call.c:342:16
    #65 0x55bbd11d35f0 in _PyObject_VectorcallTstate /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Include/cpython/abstract.h:114:11
    #66 0x55bbd11d35f0 in method_vectorcall /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Objects/classobject.c:61:20
    #67 0x55bbd1300aa9 in thread_run /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Modules/_threadmodule.c:1090:21
    #68 0x55bbd12f6d37 in pythread_wrapper /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/thread_pthread.h:253:5
    #69 0x154dd4294b42 in start_thread nptl/./nptl/pthread_create.c:442:8
    #70 0x154dd43269ff  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

0x6020003da0f0 is located 0 bytes inside of 16-byte region [0x6020003da0f0,0x6020003da100)
freed by thread T0 here:
    #0 0x154dd46cd542 in free (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xcd542) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x154dd28cb97c in mca_mpool_base_free (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7e97c) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12)

previously allocated by thread T68 here:
    #0 0x154dd46cd7ee in __interceptor_malloc (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xcd7ee) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x154dd28cbb3f in mca_mpool_base_alloc (/lib/x86_64-linux-gnu/libopen-pal.so.40+0x7eb3f) (BuildId: 38fa7808858b4015181810be8d43eeefa9adeb12)

Thread T68 created by T0 here:
    #0 0x154dd46b6c8c in __interceptor_pthread_create (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xb6c8c) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443)
    #1 0x55bbd12f682a in PyThread_start_new_thread /build/python3.10-MgDKGo/python3.10-3.10.6/build-static/../Python/thread_pthread.h:301:14

SUMMARY: AddressSanitizer: double-free (/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/libclang_rt.asan-x86_64.so+0xcd542) (BuildId: 0fc20d2022c0d572c45850b6d559d9ccfabd5443) in free
==1783101==ABORTING
terminate called after throwing an instance of 'boost::wrapexcept<boost::mpi::exception>'
  what():  MPI_Free_mem: MPI_ERR_NO_MEM: out of memory
[lama:1782811] *** Process received signal ***
[lama:1782811] Signal: Aborted (6)
[lama:1782811] Signal code:  (-6)
[lama:1782811] [ 0] /lib/x86_64-linux-gnu/libopen-pal.so.40(+0x6ab79)[0x1508350b7b79]
[lama:1782811] [ 1] /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x150836042520]
[lama:1782811] [ 2] /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x150836096a7c]
[lama:1782811] [ 3] /lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x150836042476]
[lama:1782811] [ 4] /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x1508360287f3]
[lama:1782811] [ 5] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bbe)[0x150835ca2bbe]
[lama:1782811] [ 6] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae24c)[0x150835cae24c]
[lama:1782811] [ 7] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2b7)[0x150835cae2b7]
[lama:1782811] [ 8] /work/jgrad/espresso/build/src/script_interface/espresso_script_interface.so(+0x16574ab)[0x150831e234ab]
[lama:1782811] [ 9] /work/jgrad/espresso/build/src/script_interface/espresso_script_interface.so(+0x170d682)[0x150831ed9682]
[lama:1782811] [10] /work/jgrad/espresso/build/src/script_interface/espresso_script_interface.so(std::vector<char, boost::mpi::allocator<char> >::~vector()+0x45)[0x150831ed95f5]
[lama:1782811] [11] /work/jgrad/espresso/build/src/core/espresso_core.so(void boost::mpi::detail::broadcast_impl<std::vector<int, std::allocator<int> > >(boost::mpi::communicator const&, std::vector<int, std::allocator<int> >*, int, int, mpl_::bool_<false>)+0x4b8)[0x15082ffc2db8]
[lama:1782811] [12] /work/jgrad/espresso/build/src/core/espresso_core.so(gather_global_collision_queue()+0x133)[0x15082ffb66c3]
[lama:1782811] [13] /work/jgrad/espresso/build/src/core/espresso_core.so(handle_collisions()+0x661)[0x15082ffb7641]
[lama:1782811] [14] /work/jgrad/espresso/build/src/core/espresso_core.so(integrate(int, int)+0x848)[0x1508300847b8]
[lama:1782811] [15] /work/jgrad/espresso/build/src/core/espresso_core.so(+0x66fdc1)[0x150830085dc1]
[lama:1782811] [16] /work/jgrad/espresso/build/src/core/espresso_core.so(std::remove_reference<decltype ({parm#2}(std::declval<int>(), std::declval<int>()))>::type Communication::MpiCallbacks::call<std::plus<int>, int, int, int>(Communication::Result::Reduction, std::plus<int>, int (*)(int, int), int, int) const+0x1de)[0x15083009c2fe]
[lama:1782811] [17] /work/jgrad/espresso/build/src/core/espresso_core.so(auto mpi_call<Communication::Result::Reduction, std::plus<int>, int, int, int, int&, int&>(Communication::Result::Reduction, std::plus<int>&&, int (*)(int, int), int&, int&)+0x52)[0x150830086fb2]
[lama:1782811] [18] /work/jgrad/espresso/build/src/core/espresso_core.so(python_integrate(int, bool, bool)+0x6e9)[0x1508300856d9]
[lama:1782811] [19] /work/jgrad/espresso/build/src/python/espressomd/integrate.so(+0x4999a)[0x15070d13799a]
[lama:1782811] [20] /work/jgrad/espresso/build/src/python/espressomd/integrate.so(+0x466f7)[0x15070d1346f7]
[lama:1782811] [21] /work/jgrad/espresso/build/src/python/espressomd/script_interface.so(+0x137855)[0x1506e0266855]
[lama:1782811] [22] /work/jgrad/espresso/build/src/python/espressomd/script_interface.so(+0x134b83)[0x1506e0263b83]
[lama:1782811] [23] /usr/bin/python3.10(_PyObject_MakeTpCall+0x25b)[0x560db57317db]
[lama:1782811] [24] /usr/bin/python3.10(+0x16a7c0)[0x560db57497c0]
[lama:1782811] [25] /work/jgrad/espresso/build/src/python/espressomd/integrate.so(+0x3cbb7)[0x15070d12abb7]
[lama:1782811] [26] /work/jgrad/espresso/build/src/python/espressomd/integrate.so(+0x41443)[0x15070d12f443]
[lama:1782811] [27] /work/jgrad/espresso/build/src/python/espressomd/integrate.so(+0x40ab5)[0x15070d12eab5]
[lama:1782811] [28] /work/jgrad/espresso/build/src/python/espressomd/script_interface.so(+0x137855)[0x1506e0266855]
[lama:1782811] [29] /work/jgrad/espresso/build/src/python/espressomd/script_interface.so(+0x134b83)[0x1506e0263b83]
[lama:1782811] *** End of error message ***
Aborted (core dumped)

jngrad added a commit to jngrad/espresso that referenced this issue Mar 10, 2023
@kodiakhq kodiakhq bot closed this as completed in #4686 Mar 10, 2023
kodiakhq bot added a commit that referenced this issue Mar 10, 2023
Closes #4663

Description of changes:
- the `*_arrows_type_materials` options of the visualizer now have an effect on arrow materials
- the visualizer `freeglut` dependency has been added to the user guide
- issues with the exception handling mechanism have been addressed
- undefined behavior in OpenMPI triggered by the visualizer (#4663) now has a workaround for simulations that use only 1 MPI rank
jngrad pushed a commit to jngrad/espresso that referenced this issue Mar 21, 2023
Closes espressomd#4663

Description of changes:
- the `*_arrows_type_materials` options of the visualizer now have an effect on arrow materials
- the visualizer `freeglut` dependency has been added to the user guide
- issues with the exception handling mechanism have been addressed
- undefined behavior in OpenMPI triggered by the visualizer (espressomd#4663) now has a workaround for simulations that use only 1 MPI rank
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants