ScaDaMaLe Course site and book

Introduction to Distributed Deep Learning (DDL) with Horovod over Tensorflow/keras and Pytorch

Raazesh Sainudiin and Tilo Wiklund

This is just a set of links and notebooks from the databricks blog on the topic.

It is meant to help you take the fastest steps into DDL including instructions to install and setup the right libraries and environments in databricks.

Please see the following for more details on how this module is brought to you and to be able to take it further for your own goals:

• https://docs.databricks.com/applications/machine-learning/train-model/distributed-training/horovod-runner.html
• the original databricks notebooks are from:
  • https://docs.databricks.com/applications/machine-learning/train-model/distributed-training/mnist-tensorflow-keras.html
• https://horovod.readthedocs.io/en/stable/keras.html
• https://arxiv.org/abs/1802.05799
• https://github.com/horovod/horovod