Horovod cluster

Horovod is a distributed deep learning framework for TensorFlow, Keras, Pytorch and Apache MXNet. It was originally developed by Uber, for the purposes of enabling swift and simple distributed deep learning, and significantly reducing the training time of neural networks. Using Horovod, the training of neural networks can easily be parallelized, allowing the utilization of possibly hundreds of GPUs, with minimal modifications to the original code.