This directory contains a suite of scripts for placing continuous query and
ingest load on accumulo. The purpose of these script is two-fold. First,
place continuous load on accumulo to see if breaks. Second, collect
statistics in order to understand how accumulo behaves. To run these scripts
copy all of the .example files and modify them. You can put these scripts in
the current directory or define a CONTINUOUS_CONF_DIR where the files will be
read from. These scripts rely on pssh. Before running any script you may need
to use pssh to create the log directory on each machine (if you want it local).
Also, create the table "ci" before running. You can run
org.apache.accumulo.test.continuous.GenSplits to generate splits points for a
continuous ingest table.
The following ingest scripts insert data into accumulo that will form a random graph.
$ start-ingest.sh
$ stop-ingest.sh
The following query scripts randomly walk the graph created by the ingesters. Each walker produce detailed statistics on query/scan times.
$ start-walkers.sh
$ stop-walker.sh
The following scripts start and stop batch walkers.
$ start-batchwalkers.sh
$ stop-batchwalkers.sh
And the following scripts start and stop scanners.
$ start-scanners.sh $ stop-scanners.sh
In addition to placing continuous load, the following scripts start and stop a service that continually collect statistics about accumulo and HDFS.
$ start-stats.sh
$ stop-stats.sh
Optionally, start the agitator to periodically kill the tabletserver and/or datanode
process(es) on random nodes. You can run this script as root and it will properly start
processes as the user you configured in continuous-env.sh (HDFS_USER for the Datanode and
ACCUMULO_USER for Accumulo processes). If you run it as yourself and the HDFS_USER and
ACCUMULO_USER values are the same as your user, the agitator will not change users. In
the case where you run the agitator as a non-privileged user which isn't the same as HDFS_USER
or ACCUMULO_USER, the agitator will attempt to sudo to these users, which relies on correct
configuration of sudo. Also, be sure that your HDFS_USER has password-less ssh configured.
$ start-agitator.sh
$ stop-agitator.sh
Start all three of these services and let them run for a few hours. Then run
report.pl to generate a simple HTML report containing plots and histograms
showing what has transpired.
A MapReduce job to verify all data created by continuous ingest can be run
with the following command. Before running the command modify the VERIFY_*
variables in continuous-env.sh if needed. Do not run ingest while running this
command, this will cause erroneous reporting of UNDEFINED nodes. The MapReduce
job will scan a reference after it has scanned the definition.
$ run-verify.sh
Each entry, except for the first batch of entries, inserted by continuous ingest references a previously flushed entry. Since we are referencing flushed entries, they should always exist. The MapReduce job checks that all referenced entries exist. If it finds any that do not exist it will increment the UNDEFINED counter and emit the referenced but undefined node. The MapReduce job produces two other counts : REFERENCED and UNREFERENCED. It is expected that these two counts are non zero. REFERENCED counts nodes that are defined and referenced. UNREFERENCED counts nodes that defined and unreferenced, these are the latest nodes inserted.
To stress accumulo, run the following script which starts a MapReduce job that reads and writes to your continuous ingest table. This MapReduce job will write out an entry for every entry in the table (except for ones created by the MapReduce job itself). Stop ingest before running this MapReduce job. Do not run more than one instance of this MapReduce job concurrently against a table.
$ run-moru.sh