top of page

The GPU is at the centre of the
GenAI universe

Generative AI is not possible without the GPU. It is the muscle behind training large language and multi-modal models. Once models are trained and moved to deployment (inferencing) a different configuration of compute infrastructure can be used, which includes the CPU. However, GPUs are still very much at the heart of all stages of Generative AI.  If you'd like a simple overview on the difference between a GPU and CPU, please scroll down and find the NVIDIA mythbuster video from 2009.

GenAI requires a lot of GPUs. Typically they are installed in machines of 8 GPUs each and these machines are connected together to form pods, islands, super-clusters of 256, 512, 1,024, 2,048 GPUs and upwards.  This is so they can process PetaBytes of data and create the foundation models behind applications such as ChatGPT (OpenAI/Microsoft), Gemini (Google), Llama 3 (Meta) and many others. GPUs need homes, energy, technical expertise, must be compliant and used ethically. The eco-system surrounding the GenAI GPU is extensive, complex and needs to be fully understood by all those who wish to adopt this technology.

The GPU Ikigai eco-system, surrounded by ethics, technology, energy, location.

The surrounding eco-system can be categorised into four key areas, as shown above:

  • Ethics: How the technology is used and what for what purpose

  • Energy: The type and efficiency of power used

  • Location: Geographically and type of data centre

  • Technology: The configuration, networking, software, cooling and expertise

 Key sub-areas emerge where each of the main categories overlap.​ This is just the tip of the iceberg.

4 elements in harmony.jpg

CPU v GPU NVIDIA's Mythbuster

An entertaining demonstration to show how CPUs and GPUs work and the difference between them.

From the NVIDIA GTC 2024

It has been reported as one of the most important conferences and Keynotes since Apple launched the iPhone. The video below is a great synopsis of what's on the horizon from NVIDIA in 2024 and beyond. We were fortunate enough to be in attendance.

And AMD's Mi300x

NVIDIA is currently the dominant market leader. However, and as you would expect, others are now entering the market and are becoming serious challengers. This is good news as it offer choice to organisations and competition in the marketplace ensuring pricing becomes more competitive. Below is AMD's Mi300x Platform....a very serious challenger to the H100.

Then there's Intel's Gaudi 3...

A different style of promotional video. Intel's GPU for the GenAI generation. Light on spec data and promoting the benefits of Ethernet over proprietary interconnectivity (InfiniBand). Still a way to go for Intel.

bottom of page