049_DeepLearningIntro(Scala)

Import Notebook

ScaDaMaLe Course site and book

Thanks to Christian von Koch and William Anzén for their contributions towards making these materials Spark 3.0.1 and Python 3+ compliant.

This is Raaz's update of Siva's whirl-wind compression of the free Google's DL course in Udacity https://www.youtube.com/watch?v=iDyeK3GvFpo for Adam Briendel's DL modules that will follow.

Deep learning: A Crash Introduction

This notebook provides an introduction to Deep Learning. It is meant to help you descend more fully into these learning resources and references:

  • Deep learning - buzzword for Artifical Neural Networks
  • What is it?
    • Supervised learning model - Classifier
    • Unsupervised model - Anomaly detection (say via auto-encoders)
  • Needs lots of data
  • Online learning model - backpropogation
  • Optimization - Stochastic gradient descent
  • Regularization - L1, L2, Dropout
  • Supervised
    • Fully connected network
    • Convolutional neural network - Eg: For classifying images
    • Recurrent neural networks - Eg: For use on text, speech
  • Unsupervised
    • Autoencoder

A quick recap of logistic regression / linear models

(watch now 46 seconds from 4 to 50):

Udacity: Deep Learning by Vincent Vanhoucke - Training a logistic classifier

-- Video Credit: Udacity's deep learning by Arpan Chakraborthy and Vincent Vanhoucke

Regression

Regression
y = mx + c

Another way to look at a linear model

Another way to look at a linear model

-- Image Credit: Michael Nielsen

Recap - Gradient descent

(1:54 seconds):

Udacity: Deep Learning by Vincent Vanhoucke - Gradient descent

-- Video Credit: Udacity's deep learning by Arpan Chakraborthy and Vincent Vanhoucke

Recap - Stochastic Gradient descent

(2:25 seconds):

Udacity: Deep Learning by Vincent Vanhoucke - Stochastic Gradient descent (SGD)

(1:28 seconds):

Udacity: Deep Learning by Vincent Vanhoucke - Momentum and learning rate decay in SGD

-- Video Credit: Udacity's deep learning by Arpan Chakraborthy and Vincent Vanhoucke

HOGWILD! Parallel SGD without locks http://i.stanford.edu/hazy/papers/hogwild-nips.pdf

Why deep learning? - Linear model

(24 seconds - 15 to 39):

Udacity: Deep Learning by Vincent Vanhoucke - Linear model

-- Video Credit: Udacity's deep learning by Arpan Chakraborthy and Vincent Vanhoucke

ReLU - Rectified linear unit or Rectifier - max(0, x)

ReLU

-- Image Credit: Wikipedia

Neural Network

Watch now (45 seconds, 0-45)

Udacity: Deep Learning by Vincent Vanhoucke - Neural network

-- Video Credit: Udacity's deep learning by Arpan Chakraborthy and Vincent Vanhoucke

Neural Network

Neural network

-- Image credit: Wikipedia

Multiple hidden layers

Many hidden layers

-- Image credit: Michael Nielsen