SHAD is the Scalable High-Performance Algorithms and Data-structures C++ library. SHAD is designed as a software stack, composed of three main layers:
- Abstract Runtime Interface: SHAD adopts a shared-memory, task-based, programming model, whose main tasking primitives are definide in its runtime abstraction layer; this component represents an interface to underlying runtime systems, which implement tasking and threading; for portability, SHAD can interface with multiple Runtime Systems.
- General Purpose Data-structures: SHAD data-structures offer a shared-memory abstraction, and provide APIs for parallel access and update; data-structures include arrays, vectors, maps and sets.
- Extensions: SHAD extensions are custom libraries built using the underlying SHAD components, and/or other extensions; SHAD currently include graph data-structures and algorithms.
SHAD is written in C++, and requires compiler support for (at least) C++ 11. To enable all of the SHAD's features, please review its Install Dependencies and Runtime Systems requirements.
In publications SHAD can be cited as [SHAD].
[SHAD] | V. G. Castellana and M. Minutoli, "SHAD: The Scalable High-Performance Algorithms and Data-Structures Library," 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Washington, DC, USA, 2018. |
$ git clone https://github.com/pnnl/SHAD.git shad
$ cd shad
$ docker-compose -f docker/docker-compose.yml pull head worker
$ docker-compose -f docker/docker-compose.yml up -d scale worker=2
$ docker exec -u mpi -it dokcer_head_1 /bin/bash
$ cd $HOME/shad
$ mkdir build && cd build
$ cmake .. -DCMAKE_BUILD_TYPE=Release -DSHAD_RUNTIME_SYSTEM=GMT
$ make
To run the unit test of the array on the docker cluster:
$ mpiexec -np 2 -ppn 1 --hosts docker_worker_1,docker_worker_2 \
test/unit_tests/core/shad_array_test
GPerftools is an optional dependency. Of the whole GPerftools framework, SHAD currently uses only tcmalloc when available. We have seen significant performance improvement in using tcmalloc over the standard allocator. Therefore, we recommend its use. In the case it is not available through your package manager, you can follow the following basic instruction to build and install GPerftools. Please refer to the project page to have more detailed information.
$ git clone https://github.com/gperftools/gperftools.git
$ cd gperftools
$ ./autogen.sh
$ mkdir build && cd build
$ ../configure --prefix=$GPERFTOOLSROOT
$ make && make install
where $GPERFTOOLSROOT
is the directory where you want the library to be
installed.
The Google Test framework is only required if you want to run the unit tests. On some system, GTest is not available through the package manager. In those cases you can install it following these instructions:
$ git clone https://github.com/google/googletest.git
$ cd googletest
$ mkdir build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=$GTESTROOT
$ make && make install
where $GTESTROOT
is the directory where you want the library to be
installed.
To fully exploit its features, SHAD requires a supported runtime system or threading library to be installed. SHAD currently supports:
If such software is not available on the system, SHAD can be compiled and used with its default (single-threaded) C++ backend.
SHAD uses the Global Memory and Threading (GMT) Runtime System as backend for commodity clusters. GMT requires a Linux OS, C compiler and MPI. It can be installed using the following commands:
$ git clone https://github.com/pnnl/gmt.git
$ cd gmt
$ mkdir build && cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$GMT_ROOT \
-DCMAKE_BUILD_TYPE=Release
$ make -j <SOMETHING_REASONABLE> && make install
where $GMT_ROOT
is the directory where you want the library to be installed.
Before attempting to build SHAD, please take a look at the requirements in
Install Dependencies. In case gtest is not available, compilation of unit
tests may be disabled setting SHAD_ENABLE_UNIT_TEST
to off. Currently SHAD
has full support for TBB and GMT Runtime Systems. Future releases will
provide additional backends. Target runtime systems may be specified via the
SHAD_RUNTIME_SYSTEM
option: valid values for this option are GMT
,
TBB
, and, CPP_SIMPLE
.
$ git clone <url-to-SHAD-repo> # or untar the SHAD source code.
$ cd shad
$ mkdir build && cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$SHADROOT \
-DCMAKE_BUILD_TYPE=Release \
-DSHAD_RUNTIME_SYSTEM=<SupportedRuntimeSystem> \
# if using TBB \
-DTBB_ROOT=$TBBROOT \
# else if using GMT \
-DGMT_ROOT=$GMTROOT \
# endif \
-DGTEST_ROOT=$GTESTROOT \
-DGPERFTOOLS_ROOT=$GPERFTOOLSROOT
$ make -j <SOMETHING_REASONABLE> && make install
If you have multiple compilers (or compiler versions) available on your system,
you may want to indicate a specific one using the
-DCMAKE_CXX_COMPILER=<compiler>
option.
SHAD's documentation is entirely written using Doxygen. You can obtain a copy of Doxygen through your package manager or following the installation instructions from their website. To build SHAD's documentation, you need to:
$ cd shad/build # cd into your build directory.
$ cmake .. -DSHAD_ENABLE_DOXYGEN=1
$ make doxygen
Once the documentation is build, you can open with your favorite web browser the first page with:
open docs/doxygen/html/index.html # From your build directory
This material was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor the United States Department of Energy, nor Battelle, nor any of their employees, nor any jurisdiction or organization that has cooperated in the development of these materials, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness or any information, apparatus, product, software, or process disclosed, or represents that its use would not infringe privately owned rights.
Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or Battelle Memorial Institute. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.
PACIFIC NORTHWEST NATIONAL LABORATORY operated by BATTELLE for the UNITED STATES DEPARTMENT OF ENERGY under Contract DE-AC05-76RL01830