Recurrent Neural Networks and Transformers

Overview

This course on Recurrent Neural Networks and Transformers aims to teach students the fundamentals of sequence modeling, recurrent neural networks, and attention mechanisms. By the end of the course, learners will be able to understand the intuition behind RNNs, implement RNNs from scratch, grasp the concept of attention, and explore its applications. The teaching method involves a lecture format with a detailed syllabus covering various topics related to sequential modeling and neural networks. This course is intended for individuals interested in deep learning, specifically in the areas of recurrent neural networks and transformers.

Syllabus

- Introduction
- Sequence modeling
- Neurons with recurrence
- Recurrent neural networks
- RNN intuition
- Unfolding RNNs
- RNNs from scratch
- Design criteria for sequential modeling
- Word prediction example
- Backpropagation through time
- Gradient issues
- Long short term memory LSTM
- RNN applications
- Attention fundamentals
- Intuition of attention
- Attention and search relationship
- Learning attention with neural networks
- Scaling attention and applications
- Summary