Understanding the Energy Demands of AI

Research has highlighted the environmental impact of generative AI, particularly as it relates to the energy demands of data centers. A recent Morgan Stanley report predicts that AI-related industries could emit up to 2.5 billion tons of greenhouse gases by 2030, largely due to the growing need for data centers to support AI workloads. The Green Software Foundation(GSF) Software Carbon Intensity (SCI) Specification provides a practical framework for addressing these concerns. While SCI is applicable to all software, its core principles are particularly impactful in reducing the carbon footprint of AI systems, with the goal being to reduce emissions actively, not just offset them: 1️⃣ Energy Efficiency: Optimizing AI models to use less energy is critical. Techniques like model pruning and distillation help make AI models more efficient by reducing the number of parameters and complexity without sacrificing performance, thus cutting down the energy required for training and deployment. 2️⃣ Hardware Efficiency: Using energy-efficient chipsets and maximizing hardware utilization can help reduce emissions from AI workloads. This involves developing hardware that can handle AI computations more efficiently and extending the lifecycle of existing hardware to reduce the need for frequent replacements, which contribute to emissions during production and disposal. 3️⃣ Carbon Awareness: AI systems can be made carbon-aware, meaning workloads are scheduled to run when energy grids are powered by cleaner, renewable energy. This minimizes the reliance on carbon-intensive power sources and reduces the overall environmental impact. For meaningful progress, policymakers must implement robust regulatory frameworks that support these efforts. Regulations that enforce carbon reporting for AI systems, incentivize the use of renewable energy, and establish standards for emissions will be key to aligning the AI industry with global sustainability goals. By integrating SCI principles with strong policy support, the AI industry can make substantial strides in reducing emissions while continuing to innovate responsibly. (Link - https://lnkd.in/drMQhDEY) #greenai #sustainability #genai

In the early innings of a tech shift, before the layers of abstraction have been built on top, we get a rare, transparent look at the engine room coming together. A decade from now, we won’t be thinking about chip shortages and energy supply, just like we don't consider the vast undersea cable network when we Google something. But today, we are all in the guts of the beast together, witnessing the tech trade-offs being made in real-time. We’ve talked extensively about GPU shortages, but another increasingly urgent chokepoint in our AI endeavors is energy. Training LLMs is obscenely resource-intensive. Consider this: 💡 Training GPT-3 is estimated to use just under 1,300 megawatt hours (MWh) of electricity—about as much power as consumed annually by 130 US homes. 💡 ChatGPT processes ~200M requests daily. To do that, it consumes 500,000 KWh of electricity daily, equivalent to the energy consumption of 17,000 private homes. 💡 According to the International Energy Agency (IEA), a single Google search takes 0.3 watt-hours of electricity, while a ChatGPT request takes 2.9 watt-hours, nearly 10x as much. Each request you make to ChatGPT is equivalent to turning on a 60-watt light bulb for about three minutes. 💡 If ChatGPT were integrated into the 9 billion searches done each day, the IEA says, the electricity demand would increase by 10 terawatt-hours a year—the amount consumed by about 1.5 million European Union residents. And that’s just for searches, to say nothing of other use-cases. 💡 Generating an image using a powerful AI model takes as much energy as fully charging your smartphone. 💡 By the end of the decade, AI data centers could consume as much as 20% to 25% of U.S. power requirements. Today that’s probably 4% or less. Some paint horrifying dystopian futures where super-intelligent AI enslaves humans. While this low-probability eventuality captures our collective imagination and spurs much debate, the terrible climate inevitability we are heading towards doesn’t get the attention it deserves. Even Sam Altman, the poster boy of AI, has noted that we don’t appreciate the energy needs of this technology. The future of AI—and our world—depends on a breakthrough in clean energy. #ArtificialIntelligence #Energy #CleanEnergy #Sustainability 

Is AI's Growth Sustainable? How to Make Generative Applications Greener. The rise of generative AI tools like ChatGPT and others has been remarkable, but their environmental impact is often overlooked. The data center industry, housing these systems, accounts for up to 3% of global greenhouse gas emissions, with energy consumption doubling every two years. Hyperscale cloud providers like Amazon AWS, Google Cloud, and Microsoft Azure play a significant role in powering these models, leading to major carbon footprints. Understanding the carbon footprint lifecycle of AI models is crucial. Large generative models consume extensive energy during training, and fine-tuning can be a more energy-efficient option. Inference sessions, though less energy-intensive, involve many more sessions, contributing to ongoing energy consumption. Efforts to reduce energy usage include employing less computationally expensive approaches like TinyML and using large models only when significantly valuable. To make AI greener, companies can use existing models from providers instead of creating new ones. Fine-tuning existing models on specific content domains consumes less energy and provides more value. Utilizing energy sources from carbon-friendly regions and monitoring carbon emissions can significantly reduce AI's environmental impact. Reusing models and resources, incorporating AI activity into carbon monitoring, and encouraging green AI practices are crucial steps in promoting sustainability. 1. Prioritize Fine-Tuning: Instead of training new generative models from scratch, focus on fine-tuning existing models for specific content domains. Fine-tuning consumes less energy and provides more value to businesses. 2. Explore Energy-Conserving Methods: Adopt energy-conserving computational approaches like TinyML for processing data. TinyML allows running ML models on low-powered edge devices, significantly reducing energy consumption. 3. Re-use and Open Source Models: Opt for reusing open-source models instead of creating new ones. Recycling tech can lower the carbon impact of AI practices and reduce the need for energy-intensive model development. 4. Monitor Carbon Emissions: Include AI activity in carbon monitoring practices to understand the carbon footprint of AI-related operations. Share footprint numbers to make informed decisions about AI partnerships. 5. Choose Green Energy Sources: Select cloud providers and data centers that prioritize environmentally friendly power resources. Running AI models in regions with carbon-free energy sources can significantly reduce operational emissions. Have you already considered the impact of using compute-heavy applications on our planet? Are you tracking the impact of compute in your sustainability report? #genai #aivalue #sustainableai #sustainability

Today, we launched the IEA Energy and AI special report. It is a product of over 50 experts investigating an extremely complex topic, working with the leading AI companies of our time to generate some groundbreaking insights on the furture of AI and energy. Here are just a few takeaways from a rich body of work (free to download: link below): 1. An AI-focussed hyperscaler data centre today could consume as much electricity as 100,000 households. The largest one under construction today could consume as much as 2 million households. Electricity demand from data centres for AI is rising fast. 2. By 2030, data centers globally will require slightly more energy than Japan consumes today. Electricity demand from AI-optimized servers alone set to quadruple by then. 3. Driven by AI use, by 2030, the United States is set to consume more electricity for processing data than for manufacturing all energy-intensive goods combined, including aluminium, steel, cement and chemicals. 4. Half of the global growth in data centre demand is met by renewables, supported by storage and the broader electricity grid.  Dispatchable sources, led by natural gas, also have a crucial role to play, with the tech sector helping to bring forward new nuclear and geothermal technologies as well. 5. Electricity grids are already under strain in many places: unless these risks are addressed, around 20% of planned data centre projects could be at risk of delays. 6. AI could sharpen some energy security concerns and help address others. For example, gallium is a critical metal used in cutting-edge computer chips and power electronics. China currently accounts for around 99% of global refined gallium supply. 7. Concerns that AI could accelerate climate change appear overstated, as do expectations that AI alone will address the issue. Emissions from electricity use by data centres grows grows quickly, but remains around 1% of the total energy sector emissions over the next decade. AI-led efficiencies in the energy sector can help reduce emissions – but this is also far smaller than what is needed to address climate change. Various barriers to AI adoption in the energy sector will need to be overcome to unlock such emissions reductions. The full report is available here: https://lnkd.in/eXcCFNG9

Artificial Intelligence’ escalating demand for energy and water is alarming. Since ChatGPT's launch in late 2022, the computational power needed for AI models has surged. For instance, in 2023, AI workloads consumed approximately 4.3 GW of electricity, akin to Latvia's annual energy use. Each ChatGPT query uses ten times more energy than a Google search, with daily operations needing over 500,000 kWh. This trend strains power grids, risks outages, and significantly contributes to carbon emissions. Google's 2023 report showed a 48% rise in greenhouse gas emissions since 2019, driven by increased data center electricity consumption. These centers not only demand more energy but also vast amounts of water. In 2023, Microsoft used 13 billion liters of water, exacerbating global water scarcity. Water used by data centers evaporates, unlike household water which is recycled, heightening the environmental impact. Experts predict the AI sector could consume 85 to 134 TWh of electricity annually by 2027. Mitigating this requires innovative approaches, such as Google DeepMind's JEST method, which reduces energy use during AI training. Transitioning to renewable energy is crucial, but the solution isn't straightforward. As AI integration in daily tools grows, balancing technological advancement with sustainability becomes imperative.

With Great AI Comes Great Power (Demand) Grid operators are increasingly concerned as electrification accelerates, particularly in industrial heating and transportation (EVs), placing immense pressure on the grid. AI and datacenters will magnify this challenge significantly. A ChatGPT query consumes roughly 10 times the energy of a Google search. If all Google queries were replaced with ChatGPT-like interactions, total annual energy consumption would reach around 9 TWh—about 4% of U.S. data center energy use. Goldman Sachs projects global data center power demand will surge 160% by 2030, rising from 1-2% to 3-4% of total global power consumption. India's Data Center Growth & AI Impact India’s data center capacity is estimated at 950 MW (2024) and is expected to grow to 1,700 MW by March 2025, though this may not fully account for AI-driven power demands. AI power consumption can be categorized into: Training – The computationally intensive process of developing AI models. Inference – Running queries through trained models to generate responses. Inference is likely the dominant AI power demand driver in India, though foundational AI model development could change this. Large models like ChatGPT require significant energy for inference. Alphabet chairman John Hennessy estimated an AI query costs 10 times more energy than a standard search. India’s AI Data Center Expansion Avendus Capital estimates AI-driven data center capacity in India could rise by 500 MW in four years. India’s data center market doubled from 540 MW in 2019 to 1,011 MW in 2023 and is expected to grow at a CAGR of 26% over the next three years, making it one of the fastest-growing globally. Power Demand & Efficiency Gains AI power demand can also be estimated by analyzing GPU sales. It needs to noted though AI compute power will not be evenly distributed worldwide. Europe’s high electricity prices (nearly double those of the U.S.) limit its AI compute capacity to just 4% of the global total. A crucial overlooked factor is efficiency. Data centers are becoming more power-efficient, with Power Usage Effectiveness (PUE) dropping from 2.7 in 2007 to 1.5 in 2021, with leading facilities reaching as low as 1.1. Nvidia’s Blackwell GPUs are reportedly 25 times more efficient than predecessors. Professor Jonathan Koomey’s research, Koomey’s Law, suggests compute energy efficiency doubles every 18 months, aligning with Moore’s Law. The Bigger Picture for India While energy efficiency gains will mitigate some impacts, power tariffs will be crucial to attract AI compute investment. India’s industrial power costs remain among the highest globally, and will be a big determinant of how much AI compute power the country attracts. The AI revolution is here, but its energy footprint must be managed. Balancing power demand, power pricing, infrastructure expansion, and efficiency will determine how India navigates this transformation.

The rapid expansion of AI is poised to transform industries across the globe, with companies expected to invest approximately $1 trillion in the next decade on data centers and their associated electrical infrastructure. However, a significant bottleneck threatens to slow this growth: the availability of reliable power to support the computational demands of AI systems. Today’s AI workloads require immense processing capacity, which is stretching the limits of existing power infrastructure. These demands make it increasingly challenging to secure sufficient electricity to maintain current data centers and, in many cases, prevent the construction of new facilities. AI models are more energy-intensive than the previous cloud computing applications that drove data center growth over the past two decades. At 2.9 watt-hours per ChatGPT request, AI queries are estimated to require 10x the electricity of traditional Google queries, which use about 0.3 watt-hours each; and emerging, computation-intensive capabilities such as image, audio, and video generation have no precedent. The stakes are high. After more than two decades of relatively flat energy demand in the United States—largely due to efficiency measures and offshoring of manufacturing—total energy consumption is projected to grow as much as 15-20% annually in the next decade. A significant portion of this increase is attributed to the expansion of AI-driven data centers. If current trends continue, data centers could consume up to 9% of the total U.S. electricity generation annually by 2030, more than doubling their share from just 4% today. The increasing scale and complexity of AI deployments are forcing companies to confront the harsh reality of existing infrastructure limits. Amazon Web Services recently invested $500M in Small Modular Reactors (SMR), whose technology is not yet commercially operable and isn't anticipated to come online until 2030-2035. Google signed a $100M+ power purchase agreement with an early stage SMR startup that won't have a viable unit until 2030. Microsoft convinced Constellation Energy to restart Three-Mile Island nuclear plant with a 20 year power purchase agreement. Addressing this power bottleneck requires not only technical innovation but also a deep understanding of both the electrical utility landscape and the operational needs of large-scale technology deployments. The solution will not be one size fits all. There will be a combination of many solutions required to solve the short-term immediate gap and long-term infrastructure needs. It will most likely require some combination of the following: intentional locating of data centers, improvements in data center processing efficiency, temporary fossil fuel power generation (natural gas), SMRs and “behind the meter” power purchase agreements.

The AI revolution is often framed as a race for AI MODELS and GPUs. But the new OpenAI-AMD deal makes one thing crystal clear: the next great competition may be for ELECTRONS. Let me explain... Recently, OpenAI struck a multiyear deal with AMD to deploy up to 6 gigawatts (GW) of AMD AI GPUs across its data centers — effectively linking the futures of both companies to an unprecedented expansion in AI compute capacity. It’s a thrilling milestone for the AI and semiconductor industries. But there’s one question I haven’t seen enough people asking: Who will GENERATE THE 6 GW OF POWER needed to run these AMD GPUs? To put this in perspective, 6 GW is a LOT OF POWER. Thirteen U.S. states (plus D.C. and Puerto Rico) each consume less than 6 GW of average power. Meeting this energy demand will place extraordinary stress on existing power grids — meaning future hyperscale AI data centers will almost certainly need to co-locate with their own dedicated energy generation. How can one generate 6 GW of energy? It will take multiple energy production facilities! The Vogtle Electric Generating Plant in Georgia, the largest nuclear plant in the U.S., cost roughly $35 billion to build and produces about 4.6 GW. The West County Energy Center in Florida, one of the largest natural gas plants, cost about $2 billion and produces 3.75 GW. If renewable energy options are considered, their capital costs (and land use) would likely exceed these conventional sources for equivalent capacity. And also note — powering 6 GW of GPUs is just the beginning of the energy demand. The total facility draw includes CPUs, networking, storage, and crucially, cooling. Data centers express this extra energy overhead as PUE (Power Usage Effectiveness). Even with a generous PUE of 1.25, the total demand rises to around 7.5 GW of continuous power — nearly the equivalent of two Vogtle-class nuclear plants operating full-time. You can read more about the OpenAI-AMD deal here: https://lnkd.in/d-6BDRNX #ai #openai #amd #energy #datacenter

«Even putting aside the environmental toll of chip manufacturing and supply chains, the training process for a single AI model, such as a large language model, can consume thousands of megawatt hours of electricity and emit hundreds of tons of carbon. This is roughly equivalent to the annual carbon emissions of hundreds of households in America. Furthermore, AI model training can lead to the evaporation of an astonishing amount of fresh water into the atmosphere for data center heat rejection, potentially exacerbating stress on our already limited freshwater resources. All these environmental impacts are expected to escalate considerably, with the global AI energy demand projected to exponentially increase to at least 10 times the current level and exceed the annual electricity consumption of a small country like Belgium by 2026. In the United States, the rapidly growing AI demand is poised to drive data center energy consumption to about 6% of the nation’s total electricity usage in 2026, adding further pressure on grid infrastructures and highlighting the urgent need for sustainable solutions to support continued AI advancement. The generation of electricity, particularly through fossil fuel combustion, results in local air pollution, thermal pollution in water bodies, and the production of solid wastes, including even hazardous materials. Elevated carbon emissions in a region come with localized social costs, potentially leading to higher levels of ozone, particulate matter, and premature mortality. Furthermore, the strain on local freshwater resources imposed by the substantial water consumption associated with AI, both directly for onsite server cooling and indirectly for offsite electricity generation, can worsen prolonged droughts in water-stressed regions like Arizona and Chile.» https://lnkd.in/eecneEa9

Energy is becoming a critical and growing bottleneck for AI’s future development. A 2024 U.S. Department of Energy report projects that data centers will consume between 6.7% and 12% of total U.S. electricity by 2028, up from 4.4% in 2023, a sharp increase driven largely by AI workloads. That’s nearly a tripling of demand in just 5 years. To put this in perspective, each ChatGPT query uses around 0.34 watt-hours of electricity. At hundreds of millions of queries per day, that translates into tens of millions of kilowatt-hours daily, which is roughly the output of several power plants operating at full capacity. The race to advance AI is now inseparable from the race to expand energy generation. The next breakthroughs in artificial intelligence may depend less on smarter algorithms, and more on our ability to power them.