From the course: AI Evaluations: Foundations and Practical Examples

Unlock this course with a free trial

Join today to access over 25,200 courses taught by industry experts.

Defining evaluation criteria from MVP to GA

Defining evaluation criteria from MVP to GA

From the course: AI Evaluations: Foundations and Practical Examples

Start my 1-month free trial Buy for my team

Defining evaluation criteria from MVP to GA

“

- We just learned how can you actually check the quality of your agents, not just accuracy and citation quality, but how the quality of responses, are they helpful, honest, and harmless, and actually come up with numbers. But numbers doesn't build products, products are built by people. And as you are learning AI agents, the new normal in AI agents is to launch them before they are perfect, because it's hard to build perfect AI agents. And that's where you're seeing that you need to iterate with real customers, with real contracts, to actually improve your AI agents. Does this mean you just launch your AI agent and then learn? Or there is a method here where you can take minimum risk, but add maximum value. That's what we're going to talk in this module. So let's get started on that. So your first release is going to be a measurement launch. Here, you will launch to 1%, or even less than 1% of your customer if you have a huge customer base. I'm talking like thousands of users, not…

Contents