-
Notifications
You must be signed in to change notification settings - Fork 20
Getting Started
We've provided some basic tutorials for how to browse and filter the dataset, load records into TensorFlow, and start training your first video prediction model!
Installing the RoboNet repository is simple. If you're only interested in using the data and training models, your best option is to create a fresh python3 virtual environment and installing the RoboNet repository into it.
sudo apt-get install ffmpeg
git clone https://github.com/SudeepDasari/RoboNet.git
cd RoboNet
pip install -r requirements.txt
python setup.py develop
The handy gdown utility makes downloading RoboNet (without a browser) easy! Try:
pip install gdown
gdown https://drive.google.com/a/andrew.cmu.edu/uc?id=1BkqHzfRkfzgzCfc73NbNnPMK_rg3i1n9&export=download
tar -xzvf robonet_v3.tar.gz
RoboNet is a large dataset: currently it is a 36GB download. If you'd like to get started ASAP with a small testing sample, you can download a small subset (~100M).
gdown https://drive.google.com/uc?id=1YX2TgT8IKSn9V4wGCwdzbRnS53yicV2P&export=download
tar -xvzf robonet_sampler.tar.gz
We've implemented various utility functions to make viewing and filtering the data in RoboNet easy! For example, to load all the RoboNet data and grab the subset of files collected on Sawyer robots try:
from robonet.datasets import load_metadata
all_robonet = load_metadata('~/path/to/hdf5') # path to folder you unzipped in step (1)
sawyer_data = all_robonet[all_robonet['robot'] == 'sawyer'] # pythonic filtering supported
sawyer_files = sawyer_data.get_shuffled_files() # gets shuffled list of sawyer files
We provide code and configuration files for training the inverse models and video prediction models that were tested in the RoboNet paper. The training 'entry point' is scripts/train_model.py. The configuration files contain all the model and data hyper-parameters needed for a run. They can also script grid searches, which is useful when exploring new models.
Below is an example command for training a 4 copies of the same model on on various amounts of the sawyer data present in RoboNet. The configuration file is provided here.
# note this should point to the folder containing the hdf5 folder you unzipped earlier
export DATA_DIR=/path/to/robonet/folder
# experiment checkpoints will be saved in your ~/ray_results folder
python scripts/train_model.py robonet_experiments/gpu/sawyer_grid_search.yaml
Our code automatically logs and saves image and scalar summaries during training. You may view the logging output individually in the file-system, or - for convenience - use tensorboard (command is tensorboard --logdir ~/ray_results/<name>/exp_file).
At some point you may want to use RoboNet for training your own models. For this use case, we provide general purpose code for loading data from RoboNet into tensorflow. Below is example code for loading image-state-action batches from the train set:
from robonet.datasets import load_metadata
from robonet.datasets.robonet_dataset import RoboNetDataset
import tensorflow as tf
# load and filter robonet database
database = load_metadata('/path/to/hdf5')
database = database[database['adim'] == 4]
# create data loader object and grab train/val image tensors
data = RoboNetDataset(batch_size=16, dataset_files_or_metadata=database)
train_images = data['images'] # images, states, and actions are from paired
train_actions = data['actions']
train_states = data['states']
val_images = data['images', 'val'] # can easily get val/test tensors using indexing syntax
s = tf.Session()
img_batch, act_batch, state_batch = s.run([train_images, train_actions, train_states], feed_dict=data.build_feed_dict('train'))
print('image batch shape (B, T, N_VIEWS, H, W, C):', img_batch.shape)
print('robot state batch shape (B, T, STATE_DIM):', state_batch.shape)
print('action batch shape (B, T - 1, ACTION_DIM):', act_batch.shape)
We encourage users to consider contributing data back into the RoboNet pool. Any data would be helpful, regardless of the collection policy used or robot morphology. However, there are certainly some best practices to follow in able to get useful visual data. Namely, make sure that the workspace is clearly visible in every camera angle, and that there are few "wasted pixels" out of the work area. For new users, here is some documentation for our Sawyer environment at Berkeley, along with some general resources and advice.
If you have data you'd like to contribute, please fill out our data submission form.