Introduction to Deep Learning

sebastianraschka.comSebastian Raschka

I just sat down this morning and organized all deep learning related videos I recorded in 2021. I am sure this will be a useful reference for my future self, but I am also hoping it might be useful for one or the other person out there.

PS: All code examples are in PyTorch :)

Table of Contents

• Part 1: Introduction
  • L01: Introduction to deep learning
  • L02: The brief history of deep learning
  • L03: Single-layer neural networks: The perceptron algorithm
• Part 2: Mathematical and computational foundations
  • L04: Linear algebra and calculus for deep learning
  • L05: Parameter optimization with gradient descent
  • L06: Automatic differentiation with PyTorch
  • L07: Cluster and cloud computing resources
• Part 3: Introduction to neural networks
  • L08: Multinomial logistic regression / Softmax regression
  • L09: Multilayer perceptrons and backpropration
  • L10: Regularization to avoid overfitting
  • L11: Input normalization and weight initialization
  • L12: Learning rates and advanced optimization algorithms
• Part 4: Deep learning for computer vision and language modeling
  • L13: Introduction to convolutional neural networks
  • L14: Convolutional neural networks architectures
  • L15: Introduction to recurrent neural networks
• Part 5: Deep generative models
  • L16: Autoencoders
  • L17: Variational autoencoders
  • L18: Introduction to generative adversarial networks
  • L19: Self-attention and transformer networks

Part 1: Introduction

L01: Introduction to deep learning

Part 1: Introduction

L01: Introduction to deep learning

L02: The brief history of deep learning

L03: Single-layer neural networks: The perceptron algorithm

🎮 perceptron-pytorch.ipynb | | 24 | 🎥 L3.5 The Geometric Intuition Behind the Perceptron (18:43) | | | 25 | 🎥 L3.6 Deep Learning News #2 (25:01) | 📝 stuff-in-the-news-02.pdf |

Part 2: Mathematical and computational foundations

L04: Linear algebra and calculus for deep learning

L05: Parameter optimization with gradient descent

🎮 adaline-sgd.ipynb | | 41 | 🎥 Deep Learning News #3 (20:24) | 📝 stuff-in-the-news-03.pdf |

L06: Automatic differentiation with PyTorch

L07: Cluster and cloud computing resources

List of cloud resources: https://github.com/zszazi/Deep-learning-in-cloud | | 49 | 🎥 Deep Learning News #4 (28:09) | 📝 stuff-in-the-news-04.pdf |

Part 3: Introduction to neural networks

L08: Multinomial logistic regression / Softmax regression

🎮 softmax-regression-mnist.ipynb | | 61 | 🎥 Deep Learning News #5, Feb 27 2021 (30:59) | 📝 stuff-in-the-news-05.pdf |

L09: Multilayer perceptrons and backpropration

L10: Regularization to avoid overfitting

L11: Input normalization and weight initialization

🎮 weight_kaiming.ipynb

🎮 weight_normal-batchnorm.ipynb | | 89 | 🎥 Deep Learning News #7 (23:33) | 📝 stuff-in-the-news-07.pdf |

L12: Learning rates and advanced optimization algorithms

🎮 adam.ipynb | | 95 | 🎥 L12.5 Choosing Different Optimizers in PyTorch (06:01) | | | 96 | 🎥 L12.6 Additional Topics and Research on Optimization Algorithms (12:04) | |

Part 4: Deep learning for computer vision and language modeling

L13: Introduction to convolutional neural networks

L14: Convolutional neural networks architectures

🎮 2-resnet34.ipynb | | 120 | 🎥 L14.4.1 Replacing Max-Pooling with Convolutional Layers (08:19) | | | 121 | 🎥 L14.4.2 All-Convolutional Network in PyTorch (08:17) | 🎮 3-all-convnet.ipynb | | 122 | 🎥 L14.5 Convolutional Instead of Fully Connected Layers (14:33) | | | 123 | 🎥 L14.6.1 Transfer Learning (07:38) | | | 124 | 🎥 L14.6.2 Transfer Learning in PyTorch (11:35) | 🎮 5-transfer-learning-vgg16_small.ipynb

🎮 5-transfer-learning-vgg16_large.ipynb | | 119 | 🎥 Deep Learning News #10 (20:55) | 📝 stuff-in-the-news-10.pdf |

L15: Introduction to recurrent neural networks

🎮 2_packed-lstm.ipynb |

Part 5: Deep generative models

L16: Autoencoders

L17: Variational autoencoders

🎮 3_VAE_nearest-neighbor-upsampling.ipynb

🎮 5_VAE_celeba_latent-arithmetic.ipynb | | 146 | 🎥 L17.7 VAE Latent Space Arithmetic in PyTorch – Making People Smile (11:54) | 🎮 5_VAE_celeba_latent-arithmetic.ipynb |

L18: Introduction to generative adversarial networks

L19: Self-attention and transformer networks