Probabilistic Neural-symbolic Models

Code for our ICML 2019 paper:

Probabilistic Neural-Symbolic Models for Interpretable Visual Question Answering
Ramakrishna Vedantam, Karan Desai, Stefan Lee, Marcus Rohrbach, Dhruv Batra, Devi Parikh

Checkout our package documentation at kdexd.github.io/probnmn-clevr!

If you find this code useful, please consider citing:

@inproceedings{vedantam2019probabilistic,
  title={Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering},
  author={Ramakrishna Vedantam and Karan Desai and Stefan Lee and Marcus Rohrbach and Dhruv Batra and Devi Parikh},
  booktitle={ICML},
  year={2019}
}

Usage Instructions

1. How to setup this codebase?
2. How to train your ProbNMN?
3. How to evaluate or run inference?

Pre-trained Checkpoint

Pre-trained checkpoints and corresponding config files (with all the hyper-parameters) for all training phases is available with v1.0 release of this repository. Check out the Releases!

Acknowledgments

We thank the developers of:

1. @davidmascharka/tbd-nets for providing a very clean implementation of our core Neural Module Network.

2. @allenai/allennlp for providing an awesome framework which indeed takes masking and padding seriously.

3. @rbgirshick/yacs for providing an efficient package-wide configuration management.

4. @pytorch/pytorch, this needs no reason.