From the course: AI Coding: Impacts on System Design and Architecture

Unlock this course with a free trial

Join today to access over 25,300 courses taught by industry experts.

Challenge: Optimize a system for resource efficiency and sustainability

Challenge: Optimize a system for resource efficiency and sustainability

From the course: AI Coding: Impacts on System Design and Architecture

Start my 1-month free trial Buy for my team

Challenge: Optimize a system for resource efficiency and sustainability

“

(upbeat music) - [Instructor] Are you ready for your next hands-on challenge? As AI models grow in size and complexity, so does their carbon footprint. Every extra layer, every inefficient loop, and every underutilized GPU adds up, not just in costs, but in environmental impact. Here's the scenario. You're reviewing an AI-enabled customer support platform. It uses a large language model to assist agents in real-time and generates summaries after every ticket. The current system runs on GPUs 24 by 7, regardless of load, and relies on a centralized cloud environment with no model compression, caching, or usage throttling. Your job is to identify opportunities to optimize the system for both resource efficiency and sustainability. As you analyze the system, consider questions like, could you quantize the model or use a smaller, task-specific version? Are you over-provisioning compute for off-peak hours? Is there a way to batch requests or implement caching for repeated queries? Would…

Contents