From the course: Artificial Intelligence Foundations: Neural Networks
Join today to access over 25,300 courses taught by industry experts.
Transformer architecture: The model that redefined modern AI
From the course: Artificial Intelligence Foundations: Neural Networks
Transformer architecture: The model that redefined modern AI
Transformers are now the backbone of most modern AI systems. In this video, we'll explore what its main building blocks are, and how the encoder, decoder, and attention mechanism work together. Before Transformers, models like Recurrent Neural Networks read text one piece at a time. Each new step depended on the step before it. Imagine reading a book one letter at a time and you aren't allowed to look at the next letter until you completely finish thinking about the previous one. That's how RNNs processed sequences, through recurrence, step by step. The transformer model, introduced in the 2017 paper Attention is All You Need, changed everything by enabling full parallel processing, effectively replacing recurrence with attention. Instead of reading text letter by letter, a transformer can take the entire sentence at once and instantly understand how each word relates to every other word. This shift made global…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.