Deep Learning

What you will learn

• Build and train networks in Keras and PyTorch
• Diagnose training with losses, curves and callbacks
• Apply convolutional and sequence models
• Fine-tune large pretrained models

Syllabus

82 lessons in 9 sections. Take them in order, or open any lesson directly.

1. Deep Learning5 min, 6 exercises
2. Phase 1: Neural Network Foundations12 lessons

   1. Phase 1 - Neural Network Foundations5 min
   2. Tensors and Tensor Operations9 min, 6 exercises
   3. Introduction to Neural Networks (The Perceptron)12 min, 6 exercises
   4. Multi-Layer Perceptron (MLP)8 min, 6 exercises
   5. Activation Functions (ReLU, Sigmoid, Softmax)11 min, 6 exercises
   6. Building Neural Networks with Keras (Sequential and Functional API)9 min, 6 exercises
   7. How Neural Networks Learn (Gradient-Based Optimization)8 min, 6 exercises
   8. Autograd from Scratch9 min, 6 exercises
   9. The Same Network in PyTorch7 min, 6 exercises
   10. First Example: Classifying Movie Reviews (IMDB, Binary)10 min, 6 exercises
   11. First Example: Classifying Newswires (Reuters, Multiclass)8 min, 6 exercises
   12. First Example: Predicting House Prices (Regression)7 min, 6 exercises
3. Phase 2: Training Deep Neural Networks10 lessons

   1. Phase 2 - Training Deep Neural Networks5 min
   2. Backpropagation and Optimizers (Adam, SGD)11 min, 6 exercises
   3. Loss Functions: Choosing What to Minimise10 min, 6 exercises
   4. Vanishing & Exploding Gradients8 min, 6 exercises
   5. Batch Normalization9 min, 6 exercises
   6. Regularization & Dropout8 min, 6 exercises
   7. Learning Rate Scheduling7 min, 6 exercises
   8. The Universal Workflow of Machine Learning8 min, 6 exercises
   9. Evaluating Models: Generalization and Validation7 min, 6 exercises
   10. Callbacks and TensorBoard7 min, 6 exercises
4. Phase 3: Computer Vision with CNNs12 lessons

   1. Phase 3 - Computer Vision with CNNs4 min
   2. Intro to Convolutional Neural Networks (CNN) for Images8 min, 6 exercises
   3. Pooling & CNN Architecture7 min, 5 exercises
   4. Famous CNN Architectures (LeNet to ResNet)9 min, 5 exercises
   5. Normalisation Beyond Batch: Layer, Group and Instance8 min, 5 exercises
   6. Transfer Learning - Using Pre-trained Models6 min, 5 exercises
   7. Fine-Tuning and Parameter-Efficient Tuning (LoRA)8 min, 6 exercises
   8. Data Augmentation for Small Datasets6 min, 5 exercises
   9. Image Segmentation7 min, 5 exercises
   10. Interpreting What Convnets Learn (Grad-CAM)8 min, 5 exercises
   11. Object Detection (Bounding Boxes and YOLO)8 min, 6 exercises
   12. Vision Transformers (ViT)8 min, 6 exercises
5. Phase 4: Sequence Models with RNNs7 lessons

   1. Phase 4 - Sequence Models with RNNs4 min
   2. Intro to Recurrent Neural Networks (RNN) for Sequences7 min, 5 exercises
   3. Padding, Masking and Variable-Length Sequences7 min, 6 exercises
   4. LSTM & GRU Networks9 min, 6 exercises
   5. Time Series Forecasting with RNNs6 min, 6 exercises
   6. Advanced Recurrent Layers (Dropout, Stacking, Bidirectional)6 min, 5 exercises
   7. Attention Before Transformers (Additive and Bahdanau)9 min, 6 exercises
6. Phase 5: NLP & Transformers11 lessons

   1. Phase 5 - NLP & Transformers5 min
   2. Text Preprocessing (Tokenization, Stemming, Lemmatization)7 min, 5 exercises
   3. Subword Tokenization (BPE and WordPiece)9 min, 5 exercises
   4. Bag of Words (BoW) & TF-IDF6 min, 5 exercises
   5. Word Embeddings (Word2Vec, GloVe)5 min, 5 exercises
   6. Sentiment Analysis Tutorial5 min, 5 exercises
   7. Named Entity Recognition (NER)6 min, 5 exercises
   8. Attention from Scratch (Queries, Keys and Values)6 min, 5 exercises
   9. The Transformer Architecture5 min, 5 exercises
   10. Sequence-to-Sequence Learning (Machine Translation)5 min, 5 exercises
   11. Evaluating Language Models (Perplexity and BLEU)6 min, 5 exercises
7. Phase 6: Generative Deep Learning9 lessons

   1. Phase 6 - Generative Deep Learning4 min
   2. Autoencoders5 min, 5 exercises
   3. Variational Autoencoders (VAE)6 min, 5 exercises
   4. Generative Adversarial Networks (GANs)12 min, 6 exercises
   5. Diffusion Models (Introduction)7 min, 6 exercises
   6. Evaluating Generative Models (FID, Coverage and Memorisation)8 min, 6 exercises
   7. Text Generation with Language Models6 min, 6 exercises
   8. DeepDream7 min, 6 exercises
   9. Neural Style Transfer6 min, 6 exercises
8. Phase 7: Reinforcement Learning6 lessons

   1. Phase 7 - Reinforcement Learning5 min
   2. Introduction to Reinforcement Learning6 min, 6 exercises
   3. Exploration vs Exploitation (Bandits and Epsilon Schedules)7 min, 6 exercises
   4. Q-Learning & Deep Q-Networks6 min, 6 exercises
   5. Policy Gradients (Intro)6 min, 6 exercises
   6. Actor-Critic and PPO (Intro)8 min, 6 exercises
9. Phase 8: Scaling & Deploying Deep Models10 lessons

   1. Phase 8 - Scaling & Deploying Deep Models4 min
   2. Loading & Preprocessing Data with tf.data6 min, 6 exercises
   3. Custom Models and Training Loops (TensorFlow)5 min, 6 exercises
   4. Distributed Training with tf.distribute5 min, 6 exercises
   5. Hyperparameter Tuning with KerasTuner6 min, 6 exercises
   6. Mixed Precision and Multi-GPU Training5 min, 6 exercises
   7. Serving Models with TensorFlow Serving6 min, 6 exercises
   8. Deploying to Mobile & Edge with TensorFlow Lite5 min, 6 exercises
   9. Model Compression (Pruning, Quantisation and Distillation)6 min, 6 exercises
   10. Limitations and the Future of Deep Learning7 min, 6 exercises
10. Phase 9: Capstone Projects4 lessons

    1. Phase 9 - Capstone Projects4 min
    2. Capstone 1 - An Image Classifier End to End6 min, 6 exercises
    3. Capstone 2 - A Text Classifier End to End6 min, 6 exercises
    4. Capstone 3 - A Generative Model You Can Defend9 min, 6 exercises