From the course: Machine Learning with SageMaker by Pearson

Unlock this course with a free trial

Join today to access over 25,200 courses taught by industry experts.

Batch inference and asynchronous inference

Batch inference and asynchronous inference - Amazon SageMaker Tutorial

From the course: Machine Learning with SageMaker by Pearson

Start my 1-month free trial Buy for my team

Batch inference and asynchronous inference

“

In the previous lesson for real-time inference, I mentioned batch inference and how we could use it to save money, even though it might take a little bit more time. So batch inference is processing predictions for large data sets in bulk. This is suitable for tasks that do not require real-time predictions. So with a real-time prediction, it's an immediate response. Like, here's some data. Predict something for me right now, because I need to continue with a task based on the outcome of this prediction. Whereas batch is, give me the result when you have it. I can wait. I'll put results to Amazon S3 for later use. Or with a pipeline, pipe it through other things as well. Asynchronous inference allows prediction requests to be queued and processed asynchronously. So a real-time inference is going to receive the request, process it, and return a response while you wait. And then asynchronous is going to receive the request. And then you go on about your business. And eventually, you come…

Contents