From the course: TensorFlow: Working with NLP
Join today to access over 25,300 courses taught by industry experts.
Transformer: Architecture overview
From the course: TensorFlow: Working with NLP
Transformer: Architecture overview
- [Instructor] The transformer is made up of two components, an encoder and the decoder. We feed in the English sentence, such as I like nlp into the encoder at the bottom of the diagram. The transformer can act as a translator from English to German. And so the output from the decoder is the German translation ich mag nlp. The transformer is made up not of a single encoder, but rather six encoders. Each of these parts can be used independently depending on the task. So the encoder-decoder models are good for generative tasks that require an input such as translation or summarization. Examples of such models are Facebook's Bi-directional and Auto Regressive Transformers, or BART, and Google's T5 model. Encoder-only models are good for tasks that require understanding of the inputs such as sentence classification and named entity recognition. Examples include the family of BERT models like BERT, AlBERT, RoBERTa,…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.