AI is revolutionary in its capabilities. It is becoming integrated to all the applications that we use daily such as word processors, spreadsheets, websites and many mobile apps to name a few. But this integration and growth comes at a cost. The technology platforms expand exponentially as the user base grows daily. According to Time magazine in January 2023, the fastest growing platform in the internet was an AI service and it grew from its launch in November 2022 to 100 million consumers by February 2023! In context, it took some well known social media platforms between 9 months to 2.5 years to reach that level.
According to Bloomberg, who commissioned a report in March 2023, training this particular AI platform generated 502 tonnes of carbon emissions – the same as 110 cars in a year! In electrical terms it took 1.287 gigawatt hours! and that was when it was hosted on a well established cloud service. With the expansion and integration into Tier 1 software providers platforms it's use will literally explode.
The cost of the electricity can easily be accounted for, the only problem is that Cloud Hyperscalers generally don't disclose exact figures. These platforms run on GPU technology and GPU Chipset manufacturers also don't generally disclose how many are employed in the solution.
Datacentres using high power compute systems run anywhere between 20kW and 80kW racks and require substantial cooling equipment. Assuming the details for the training of this particular AI system are exact, based on 30kW racks, that was 1.287 gigawatt hours therefore we can recalculate it backwards to determine how much power it actually consumes per hour in a year. We have to use a logical timeframe as an exact timeline for training this AI system is unknown.
So 1.287 Gigawatt hours in a year is 146.92kW, and this is done on every iteration of the language model.
The average PCIe GPU consumes on average 375W (and some are much higher) when at work so we can assume the possible footprint in GPU's (if it was using PCIe based GPU) would be 392 GPU devices!
Keep in mind, although that's not a big figure… it is a training figure, not a production figure! It does need to scale up, and it has.
One article by AP News written in September 2023 published a Tier 1 providers AI element which looked at the cost of cooling this platform and only this platform in an established Data Centre. They received a statement which said their water cooling consumption spiked 34% from 2021 to 2022 to nearly 1.7 billion gallons of water compared to previous years! It also stated that they estimate that this facility consumes 500ml per minute of this natural resource! Realistically that is 30l per hour and a whopping 262,800l per year!
They also managed to get a statement from another Tier 1 Cloud services provider for the same period! They reported a 20% growth, so it can be assumed that across their platform it's consuming somewhere near 250,000l per year!
A statement from a US Water Works said they and their city government will only consider future data centre projects if those projects can “demonstrate and implement technology to significantly reduce peak water usage from the current levels” to preserve the water supply for residential and other commercial needs.
The cooling technology that AI platforms and Cloud Service platforms are typically built on air cooled systems. In the UK, one hosting provider builds their facilities using a 30kW rack footprint so this means they need huge pressure walls for airflow, cooled by water! This is inefficient and a push is needed for GPU / CPU manufacturers to look at dielectric liquid cooling as the default to reduce this huge burden on water consumption.
In our facility in Swindon, we use an immersion unit which uses a total of 15l of water to cool the dielectric fluid and has resulted in 5l per year consumption of water. That's for a 100kW unit. Scale that up to match the training platform… that's 1.5 baths, using 30l of water to fill with a consumption of 7.5l per year!
This shows how Immersion cooling can drive the water cooling efficiency message for GPU and CPU hardware that is compatible.