StoryGAN CLEVR Dataset Generation

This is the code for StoryGAN adapted from CLEVR dataset.

Step 1: Generating Images

First we render synthetic images using Blender, outputting both rendered images as well as a JSON file containing ground-truth scene information for each image.

wget https://ftp.nluug.nl/pub/graphics/blender/release/Blender2.78/blender-2.78b-linux-glibc219-x86_64.tar.bz2
bzip2 -d blender-2.78b-linux-glibc219-x86_64.tar.bz2
tar xopf blender-2.78b-linux-glibc219-x86_64.tar
export BLENDER_PATH="$PWD/blender-2.78b-linux-glibc219-x86_64"
export PATH=$BLENDER_PATH:$PATH

Blender ships with its own installation of Python which is used to execute scripts that interact with Blender; you'll need to add the image_generation directory to Python path of Blender's bundled Python. The easiest way to do this is by adding a .pth file to the site-packages directory of Blender's Python, like this:

echo $PWD/image_generation >> $BLENDER/$VERSION/python/lib/python3.5/site-packages/clevr.pth

where $BLENDER is the directory where Blender is installed and $VERSION is your Blender version; for example on OSX you might run:

echo $PWD/image_generation >> /Applications/blender/blender.app/Contents/Resources/2.78/python/lib/python3.5/site-packages/clevr.pth

cd image_generation
bash gen_img.sh

On OSX the blender binary is located inside the blender.app directory; for convenience you may want to add the following alias to your ~/.bash_profile file:

alias blender='/Applications/blender/blender.app/Contents/MacOS/blender'

The file inc_output/CLEVR_scenes.json will contain ground-truth scene information for all newly rendered images.

You can find more details about image rendering here.

Step 2: Feature Generation

Next we generate ground truth features.

You can generate feature like this:

cd descripton_generation
python vec_questions.py

The file inc_output/CLEVR_dict.npy will then contain feature vectors for generated images.