adadamp

See the documentation for more detail, https://docs.stsievert.com/adadamp

Install

https://docs.stsievert.com/adadamp/installation.html

Benchmarks

Good distributed benchmarks:

• Good convnet benchmarks by PyTorch core dev: https://github.com/soumith/convnet-benchmarks/
• Good high level overview on training times: https://wrosinski.github.io/deep-learning-speed-vol1/

Less relevant benchmarks

• MLPerf: https://mlperf.org/
• https://github.com/ryujaehun/pytorch-gpu-benchmark