The Illustrated GPT-2 (Visualizing Transformer Language Models)

• Report this article

Jay Alammar

Jay Alammar

Published Aug 12, 2019

+ Follow

https://jalammar.github.io/illustrated-gpt2/

New blog post visually exploring the insides of the model that dazzled us with its ability to write coherently and with conviction. We also look at other applications of this type of model.

To view or add a comment, sign in

More articles by Jay Alammar

• Three Transformer Papers to Highlight from ACL2020

  Jul 15, 2020

  Three Transformer Papers to Highlight from ACL2020

  I was happy to attend the virtual ACL (https://acl2020.org/) conference for machine learning in natural language…

  180

  3 Comments
• The Illustrated Word2vec

  Mar 27, 2019

  The Illustrated Word2vec

  The Illustrated Word2vec New blog post gently introducing word embeddings, and visualizing the mechanics of the pivotal…

  57

  3 Comments
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)

  Dec 3, 2018

  The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)

  A visual look at some of the leading breakthroughs in NLP in 2018. New blog post! https://jalammar.

  124

  4 Comments
• A Gentle Visual Intro to Data Analysis in Python Using Pandas

  Oct 29, 2018

  A Gentle Visual Intro to Data Analysis in Python Using Pandas

  I just wrote a new blog post! Start here if you've been looking to start learning about data analysis in python using…

  45
• A Visual And Interactive Look at Basic Neural Network Math

  Mar 28, 2018

  A Visual And Interactive Look at Basic Neural Network Math

  I created another interactive set of visualizations explaining some of the basic concepts of machine learning. This…

  52
• A Visual and Interactive Guide to the Basics of Neural Networks

  Dec 15, 2016

  A Visual and Interactive Guide to the Basics of Neural Networks

  I have been trying to learn the basics of machine learning over the last 18 months. I saw a need for gentler…

  58

  8 Comments
• Supercharging Android Apps With TensorFlow (Google's Open Source Machine Learning Library)

  Jan 6, 2016

  Supercharging Android Apps With TensorFlow (Google's Open Source Machine Learning Library)

  The first post of my brand new mobile dev blog. Explaining an example for using sophisticated machine learning…

  21

  1 Comment
• دليل شركات تطوير المواقع وتطبيقات الايفون والاندرويد السعودية

  Aug 13, 2015

  دليل شركات تطوير المواقع وتطبيقات الايفون والاندرويد السعودية

  I have compiled a directory of Saudi web/mobile app design and development companies. Along with freelancers who can…

  65

  11 Comments
• نظرة إلى صناديق رأس المال الجريء (Venture Capital) في السعودية

  May 27, 2015

  نظرة إلى صناديق رأس المال الجريء (Venture Capital) في السعودية

  A look at Venture Capital funds in Saudi Arabia (Arabic) http://www.blogjihad.

  29

  3 Comments

Show more

See all articles