From the course: AI Workshop: Advanced Chatbot Development

Unlock this course with a free trial

Join today to access over 25,300 courses taught by industry experts.

Solution: Prepare the chatbot for deployment

Solution: Prepare the chatbot for deployment

From the course: AI Workshop: Advanced Chatbot Development

Start my 1-month free trial Buy for my team

Solution: Prepare the chatbot for deployment

“

(bright upbeat music) - [Instructor] So how was the challenge? In this challenge, we had to create a chatbot of a Flan-T5 model after doing distillation and doing pruning. I know it's a lot, but it kind of, it's a nice twist to everything that we have seen in this chapter. So don't worry. Let's go with step by step, and as I always say, this is a solution, not the solution, there are many, many ways to do this, so if it doesn't look exactly as what you did, it's super, super fine. So let's connect to a GPU. There we are. Now we have our GPU, and as always we start with the pip installs. There it is, beautiful. So, first we are going to do the distillation. Now, as you are well aware by now, distillation is a process that takes a lot of RAM, RAM in the CPU and RAM in the GPU, which means that in order to make it all fit all the processes in just one collab, we're going to do a, let's call it quote unquote demo distillation. So a very tiny dataset, just a thousand training examples, and…

Contents