Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[orientdb] put db conn stuff in sync method #609

Merged
merged 1 commit into from
Feb 10, 2016
Merged

[orientdb] put db conn stuff in sync method #609

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions orientdb/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,15 @@ Then, run the workload:

See the next section for the list of configuration parameters for OrientDB.

## DB creation with the OrientDBClient
This client will create a database for you if the connection database you specify does not exists. You can also specify connection information to a preexisting database.

You can use the ```orientdb.newdb=true``` property to allow this client to drop and create a new database instance during the ```load``` phase.

NOTE: understand that using the ```orientdb.newdb=true``` property will drop and recreate databases even if it was a preexisting instance.

WARNING: Creating a new database will be done safely with multiple threads on a single YCSB instance, but is not guaranteed to work when launching multiple YCSB instances. In that scenario it is suggested that you create the db before hand, or run the ```load``` phase with a single YCSB instance.

## OrientDB Configuration Parameters

* ```orientdb.url``` - (required) The address to your database.
Expand Down
80 changes: 52 additions & 28 deletions orientdb/src/main/java/com/yahoo/ycsb/db/OrientDBClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,30 +80,35 @@ public class OrientDBClient extends DB {
private static final String ORIENTDB_MASSIVEREAD = "massiveread";
private static final String ORIENTDB_NOCACHE = "nocache";

private final Logger log = LoggerFactory.getLogger(getClass());

@Override
public void init() throws DBException {
Properties props = getProperties();
private static final Logger LOG = LoggerFactory.getLogger(OrientDBClient.class);

/**
* This method abstracts the administration of OrientDB namely creating and connecting to a database.
* Creating a database needs to be done in a synchronized method so that multiple threads do not all try
* to run the creation operation simultaneously, this ends in failure.
*
* @param props Workload properties object
* @return a usable ODatabaseDocumentTx object
* @throws DBException
*/
private static synchronized ODatabaseDocumentTx initDB(Properties props) throws DBException {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add javadoc explaining why this needs to be synchronized (it's not obvious because the various clients don't appear to share connection objects).

I think it's because of the automatic table creation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

String url = props.getProperty(URL_PROPERTY);
String user = props.getProperty(USER_PROPERTY, USER_PROPERTY_DEFAULT);
String password = props.getProperty(PASSWORD_PROPERTY, PASSWORD_PROPERTY_DEFAULT);
Boolean newdb = Boolean.parseBoolean(props.getProperty(NEWDB_PROPERTY, NEWDB_PROPERTY_DEFAULT));
String remoteStorageType = props.getProperty(STORAGE_TYPE_PROPERTY);
String intent = props.getProperty(INTENT_PROPERTY, INTENT_PROPERTY_DEFAULT);
Boolean dotransactions = Boolean.parseBoolean(
props.getProperty(DO_TRANSACTIONS_PROPERTY, DO_TRANSACTIONS_PROPERTY_DEFAULT));
Boolean isrun = Boolean.parseBoolean(props.getProperty(DO_TRANSACTIONS_PROPERTY, DO_TRANSACTIONS_PROPERTY_DEFAULT));

ODatabaseDocumentTx dbconn;

if (url == null) {
throw new DBException(String.format("Required property \"%s\" missing for OrientDBClient", URL_PROPERTY));
}

log.info("OrientDB loading database url = " + url);
LOG.info("OrientDB loading database url = " + url);

// If using a remote database, use the OServerAdmin interface to connect
if (url.startsWith(OEngineRemote.NAME)) {
isRemote = true;
if (remoteStorageType == null) {
throw new DBException("When connecting to a remote OrientDB instance, " +
"specify a database storage type (plocal or memory) with " + STORAGE_TYPE_PROPERTY);
Expand All @@ -113,54 +118,73 @@ public void init() throws DBException {
OServerAdmin server = new OServerAdmin(url).connect(user, password);

if (server.existsDatabase()) {
if (newdb && !dotransactions) {
log.info("OrientDB dropping and recreating fresh db on remote server.");
if (newdb && !isrun) {
LOG.info("OrientDB dropping and recreating fresh db on remote server.");
server.dropDatabase(remoteStorageType);
server.createDatabase(server.getURL(), ORIENTDB_DOCUMENT_TYPE, remoteStorageType);
}
} else {
log.info("OrientDB database not found, creating fresh db");
LOG.info("OrientDB database not found, creating fresh db");
server.createDatabase(server.getURL(), ORIENTDB_DOCUMENT_TYPE, remoteStorageType);
}

server.close();
db = new ODatabaseDocumentTx(url).open(user, password);
dbconn = new ODatabaseDocumentTx(url).open(user, password);
} catch (IOException | OException e) {
throw new DBException(String.format("Error interfacing with %s", url), e);
}
} else {
try {
db = new ODatabaseDocumentTx(url);
if (db.exists()) {
db.open(user, password);
if (newdb && !dotransactions) {
log.info("OrientDB dropping and recreating fresh db.");
db.drop();
db.create();
dbconn = new ODatabaseDocumentTx(url);
if (dbconn.exists()) {
dbconn.open(user, password);
if (newdb && !isrun) {
LOG.info("OrientDB dropping and recreating fresh db.");
dbconn.drop();
dbconn.create();
}
} else {
log.info("OrientDB database not found, creating fresh db");
db.create();
LOG.info("OrientDB database not found, creating fresh db");
dbconn.create();
}
} catch (ODatabaseException e) {
throw new DBException(String.format("Error interfacing with %s", url), e);
}
}

log.info("OrientDB connection created with " + url);
if (dbconn == null) {
throw new DBException("Could not establish connection to: " + url);
}

LOG.info("OrientDB connection created with " + url);
return dbconn;
}

@Override
public void init() throws DBException {
Properties props = getProperties();

String intent = props.getProperty(INTENT_PROPERTY, INTENT_PROPERTY_DEFAULT);

db = initDB(props);

if (db.getURL().startsWith(OEngineRemote.NAME)) {
isRemote = true;
}

dictionary = db.getMetadata().getIndexManager().getDictionary();
if (!db.getMetadata().getSchema().existsClass(CLASS)) {
db.getMetadata().getSchema().createClass(CLASS);
}

if (intent.equals(ORIENTDB_MASSIVEINSERT)) {
log.info("Declaring intent of MassiveInsert.");
LOG.info("Declaring intent of MassiveInsert.");
db.declareIntent(new OIntentMassiveInsert());
} else if (intent.equals(ORIENTDB_MASSIVEREAD)) {
log.info("Declaring intent of MassiveRead.");
LOG.info("Declaring intent of MassiveRead.");
db.declareIntent(new OIntentMassiveRead());
} else if (intent.equals(ORIENTDB_NOCACHE)) {
log.info("Declaring intent of NoCache.");
LOG.info("Declaring intent of NoCache.");
db.declareIntent(new OIntentNoCache());
}
}
Expand Down Expand Up @@ -248,7 +272,7 @@ public Status scan(String table, String startkey, int recordcount, Set<String> f
Vector<HashMap<String, ByteIterator>> result) {
if (isRemote) {
// Iterator methods needed for scanning are Unsupported for remote database connections.
log.warn("OrientDB scan operation is not implemented for remote database connections.");
LOG.warn("OrientDB scan operation is not implemented for remote database connections.");
return Status.NOT_IMPLEMENTED;
}

Expand Down