From the course: AI Workshop: Advanced Chatbot Development
Join today to access over 25,300 courses taught by industry experts.
Demo: benchmarking our chatbot
From the course: AI Workshop: Advanced Chatbot Development
Demo: benchmarking our chatbot
- [Instructor] In this final demo, we will see how to benchmark our chatbot by measuring its latency, its RAM usage, and how accurate it is with the Rouge score. So let's connect to a GPU. There we go. We do the pip installs as always. There it is. So let's create our chatbot as we have been doing all along. But in this case there will be a little change because we're going to import time and PSU tools to measure the RAM and the latency. We are going to use the Microsoft's Phi-3 model as we have done before, because it's a great chatbot, and check that there is no conversation history. The reason for that, it's because we want to benchmark this chatbot. If we keep sending messages and it remembers those messages, then naturally, each response will take longer, which makes sense. But for a benchmark where we want each measurement to be independent, it doesn't make sense. So we're going to just send messages and get the response without history. We define the app, and the only route…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.