From the course: Building Apps with AI Tools: ChatGPT, Semantic Kernel, and Langchain

Unlock this course with a free trial

Join today to access over 24,800 courses taught by industry experts.

Evaluating GenAI prompt performance

Evaluating GenAI prompt performance - ChatGPT Tutorial

From the course: Building Apps with AI Tools: ChatGPT, Semantic Kernel, and Langchain

Start my 1-month free trial Buy for my team

Evaluating GenAI prompt performance

“

- [Instructor] So we know how to build AI generated responses, but how do we tell if they're any good? Well, there are three techniques to do so. The first is a simple text match. So you can do things, like matching exactly, doing a fuzzy search, or using regex to evaluate the responses. The second is semantic similarity. This is similar to semantic search, which we talked about before, where we compare two similar phrases by using their embeddings. And if we get a high score, then it's a match. And the third one is self eval. This one gets a little bit meta. We're going to ask GPT-4 if the generated response is accurate to the response we had in mind. If it is, great. Otherwise, we say, "Incorrect." Now, let's go take a look. How does text match work? Let's say we have an AI response, like, "I want a basketball." It's coming from some sort of prompt. What we can do is we could run it through a…

Contents