From the course: AI Pricing and ROI: A Technical Breakdown

Unlock the full course today

Join today to access over 25,200 courses taught by industry experts.

Choosing the right hardware for AI models

Choosing the right hardware for AI models

From the course: AI Pricing and ROI: A Technical Breakdown

Start my 1-month free trial Buy for my team

Choosing the right hardware for AI models

“

- [Instructor] Let's talk about hardware costs for AI hosting use cases. In this case, I typically think through three important considerations: Latency, throughput, and model location. Let's start off with latency. Latency requirements depend on your user expectations ranging from milliseconds to days. If you're on social media, and you're trying to find the next post, it should show up very quickly, otherwise you'll turn off the app. And if you're working on more of a business process, like batch document processing, this might take hours or maybe days to do so. It really depends on the use case. What's important to understand here is that different use cases have different latency requirements. The next consideration is throughput, which is the number of requests per second that you need to handle. And finally, we have model location. Model location can mean a number of different things. For example, there…

Contents