From the course: Programming Generative AI: From Variational Autoencoders to Stable Diffusion with PyTorch and Hugging Face

Unlock this course with a free trial

Join today to access over 24,800 courses taught by industry experts.

Decoding strategies

Decoding strategies

From the course: Programming Generative AI: From Variational Autoencoders to Stable Diffusion with PyTorch and Hugging Face

Start my 1-month free trial Buy for my team

Decoding strategies

“

- [Instructor] So now that we saw kind of the default, or what's baked into the Hugging Face pipelines for generating text, I wanted to talk about how both the default strategies work for turning probabilities into tokens, into text, as well as talk about some alternatives that Hugging Face makes pretty easy. So the first that I want to mention, which actually isn't the default probably because it's usually not what you want and usually not the best, is to essentially do greedy sampling. And with the generation pipeline, we're using the same one that we had in the last video, this is the DistilGPT2, we're using the same prompt of "Jonathan Dinu is." So the default is do_sample=True. So do_sample, you can think of it as what do you want to do with the actual return probabilities from the model itself. So if you specify do_sample=False, we're essentially just taking the maximum probability and using that to generate the text. Now, the important thing to know about this greedy sampling…

Contents