top of page

Some facts about GenAI energy consumption

Generative AI is all about the GPU, so it's worth knowing exactly the power (electricity) required to keep this technology alive; to fuel the curiosity and needs of the users of platforms such as ChatGPT i.e. people like you.   We've tried to break this down into the simplest terms possible, to better demonstrate the equivalent amounts of electricity used.

​

  • The expected number of NVIDIA H100 GPUs to be sold in 2024 is estimated at 3.5Million.

  • Each GPU, at 61% utilisation uses 3,740KWh per annum.

  • Therefore, the total annual electricity consumption for all GPUs in 2024 is:

    • 13,090,000,000 KWh​

    • 13,090,000 MWh

    • 13,090 GWh 

  • The average household of 2.36 people per household consumes 12,000KWh per annum.​

  • The electricity required to power all GPUs sold in 2024, being utilised at just under two thirds capacity equates to 1,090,833 households

  • A population of 2,574,367, which is the same population of Chicago IL. We hope that's quite a difficult thing to comprehend.

Another way to look at this is the power required per Prompt. i.e. each query entered into ChatGPT, Gemini etc. Research from engineers at Google Brain calculated the below:

​

  • The average Prompt consumes 6.79Wh.

  • There are 180.5 million users of ChatGPT alone.

  • If each of these users entered one Prompt per day that would equate to 446,103,00 KWh per annum.

  • Enough to power 37,175 household per annum (87,734 people)

  • That's just a single application using GenAI compute power. There are many future releases on a similar scale or even larger, i.e. Meta's General Intelligence​

We hope some of these statistics can help you calculate the energy impact you need to consider when utilising Generative AI for your own organisation. Will you buy the compute power as an asset and deploy it in your own data centre? Or will you rent it from a Cloud Service Provider? Either way, you need to do the sums.

What type of electricity though?

Now you know the amount of electricity GenAI GPUs can consume, you now need to consider the source of the energy.  The cloud industry, which is now over 20 years old, has been built on thousands of data centres globally that have been designed for CPU computing needs, storage and networking. Not all of these have been built with sustainable energy or efficiency in mind. With the introduction of 3.5 million power hungry GPUs, where are the existing cloud providers, new cloud providers, or your organisation going to put them? Old data centres are not fit for purpose. That's why there is a new breed of data centre emerging that utilises 100% renewable energy and direct-to-chip liquid cooling for more efficient and environmentally friendly methods of deployment.

If you're choosing to use cloud service providers and/or 3rd party data centres, please ensure you get all the facts about their energy sources and cooling technology.  For a real insight into the environmental cost of cloud, please take time to watch the full version of CLOUDED II from Dark Matter.

Then there's energy efficiency

As well as the source of energy, you need to consider how efficient this energy is being used in your own or a third-party data centre. The most recognised metric is the Power Usage Effectiveness or PUE. In basic terms this is a simple division of the entire power used by the data centre divide by the power used for the IT infrastructure. Levels below 1.2 are deemed to be efficient and organisations are striving to bring this down further. There are also other metrics that can be used and the entertaining video below, which is the best we have found that explains this in more detail.

bottom of page