From the course: LinkedIn AI Academy AI-100: 2 Supervised Learning with Neural Networks
Join today to access over 25,300 courses taught by industry experts.
Transformer model
From the course: LinkedIn AI Academy AI-100: 2 Supervised Learning with Neural Networks
Transformer model
- [Instructor] In the previous video, we learned how encoder decoder neural networks can use attention to improve their accuracy for machine translation. When the decoder forms the representation of a translated word in the target language, it focuses its attention only on the relevant words in the source language. In this video, we will discuss how attention is used in one of the most important neural networks of today. The transformer model. The transformer model, is central to all state-of-the-art natural language processing neural networks today. Transformer neural networks are used to model sequences. For example, a sentence is a sequence of words and a video is a sequence of image frames. The key aspect of the transformer model is a concept called self attention. Through this attention mechanism, when computing the representation of any word in a sentence the transformer model decides how much weight to give`…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.