1、学习路线、学习记录

• 基于官方学习文档：https://pytorch.org/tutorials/
• 基于左侧内容学习，其中点击 “+” 可出现相应学习内容。

（1）Introduction to PyTorch

• Learn the Basics【Done】
• Quickstart【Done】
• Tensors【Done】
• Datasets & DataLoaders【Done】
• Transforms【Done】
• Build the Neural Network【Done】
• Automatic Differentiation with torch.autograd【Done】
• Optimizing Model Parameters【Done】

• Save and Load the Model【Done】

  （2）Introduction to PyTorch on YouTube

• Introduction to PyTorch - YouTube Series

• Introduction to PyTorch
• Introduction to PyTorch Tensors
• The Fundamentals of Autograd
• Building Models with PyTorch
• PyTorch TensorBoard Support
• Training with PyTorch

• Model Understanding with Captum

  （3）Learning PyTorch

• Deep Learning with PyTorch: A 60 Minute Blitz

• Learning PyTorch with Examples
• What istorch.nnreally?

• Visualizing Models, Data, and Training with TensorBoard

  （4）各领域

  （a）Image and Video

• TorchVision Object Detection Finetuning Tutorial

• Transfer Learning for Computer Vision Tutorial
• Adversarial Example Generation
• DCGAN Tutorial
• Spatial Transformer Networks Tutorial

• Optimizing Vision Transformer Model for Deployment

  （b）Audio

• Audio I/O

• Audio Resampling
• Audio Data Augmentation
• Audio Feature Extractions
• Audio Feature Augmentation
• Audio Datasets
• Speech Recognition with Wav2Vec2
• Speech Command Classification with torchaudio
• Text-to-speech with torchaudio

• Forced Alignment with Wav2Vec2

  （c）Text

• Language Modeling with nn.Transformer and TorchText

• NLP From Scratch: Classifying Names with a Character-Level RNN
• NLP From Scratch: Generating Names with a Character-Level RNN
• NLP From Scratch: Translation with a Sequence to Sequence Network and Attention
• Text classification with the torchtext library

• Language Translation with nn.Transformer and torchtext

  （d）Reinforcement Learning

• Reinforcement Learning (DQN) Tutorial

• Train a Mario-playing RL Agent

  （5）Deploying PyTorch Models in Production

• Deploying PyTorch in Python via a REST API with Flask

• Introduction to TorchScript
• Loading a TorchScript Model in C++
• (optional) Exporting a Model from PyTorch to ONNX and Running it using ONNX Runtime

• Real Time Inference on Raspberry Pi 4 (30 fps!)

  （6）Code Transforms with FX

• (beta) Building a Convolution/Batch Norm fuser in FX

• (beta) Building a Simple CPU Performance Profiler with FX

  （7）Frontend APIs

• (beta) Channels Last Memory Format in PyTorch

• Forward-mode Automatic Differentiation (Beta)
• Using the PyTorch C++ Frontend
• Dynamic Parallelism in TorchScript

• Autograd in C++ Frontend

  （8）Extending PyTorch

• Double Backward with Custom Functions

• Fusing Convolution and Batch Norm using Custom Function
• Custom C++ and CUDA Extensions
• Extending TorchScript with Custom C++ Operators
• Extending TorchScript with Custom C++ Classes
• Registering a Dispatched Operator in C++

• Extending dispatcher for a new backend in C++

  （9）Model Optimization

• Profiling your PyTorch Module

• PyTorch Profiler With TensorBoard
• Hyperparameter tuning with Ray Tune
• Optimizing Vision Transformer Model for Deployment
• Parametrizations Tutorial
• Pruning Tutorial
• (beta) Dynamic Quantization on an LSTM Word Language Model
• (beta) Dynamic Quantization on BERT
• (beta) Quantized Transfer Learning for Computer Vision Tutorial
• (beta) Static Quantization with Eager Mode in PyTorch

• Grokking PyTorch Intel CPU performance from first principles

  （10）Parallel and Distributed Training

• PyTorch Distributed Overview

• Single-Machine Model Parallel Best Practices
• Getting Started with Distributed Data Parallel
• Writing Distributed Applications with PyTorch
• Getting Started with Fully Sharded Data Parallel(FSDP)
• Customize Process Group Backends Using Cpp Extensions
• Getting Started with Distributed RPC Framework
• Implementing a Parameter Server Using Distributed RPC Framework
• Distributed Pipeline Parallelism Using RPC
• Implementing Batch RPC Processing Using Asynchronous Executions
• Combining Distributed DataParallel with Distributed RPC Framework
• Training Transformer models using Pipeline Parallelism
• Training Transformer models using Distributed Data Parallel and Pipeline Parallelism

• Distributed Training with Uneven Inputs Using the Join Context Manager

  Mobile

• Image Segmentation DeepLabV3 on iOS

• Image Segmentation DeepLabV3 on Android

  Recommendation Systems

• Introduction to TorchRec

  2、参考

• 官网：https://pytorch.org/

• 官网学习手册
  • https://pytorch.org/tutorials/
• 官网文档
  • https://pytorch.org/docs/stable/