From the course: Machine Learning with SageMaker by Pearson

Unlock this course with a free trial

Join today to access over 25,200 courses taught by industry experts.

Comparing model performance using A/B testing

Comparing model performance using A/B testing - Amazon SageMaker Tutorial

From the course: Machine Learning with SageMaker by Pearson

Start my 1-month free trial Buy for my team

Comparing model performance using A/B testing

“

One of the benefits of using SageMaker Pipelines is the ability to very quickly deploy a new model. And why might you want to do that? Well, you get some new data, you retrain your model, and now your model is operating on that new data and providing better predictions on whatever it is trying to predict. Now, how can you determine if this model is better than that model. That's where AB testing comes in. And this is something that Amazon does consider to be in scope for this certification. So what is AB testing? It's a method to compare two versions of a model. So if you are using something like SageMaker Pipelines and you have deployed a new model based on something like data in an S3 bucket or a database changing, then you're going to want to take that new model and determine how much better it's performing against the previous model. And that's what A-B testing is all about. It helps you identify which model is operating better in real world testing. So you're actually going to…

Contents