A single NVIDIA H100 GPU draws 700 watts. An eight-GPU server housing them pulls roughly 10 to 11 kilowatts. A modern AI training rack, fully populated, can sit anywhere from 40 kilowatts to 140 kilowatts, depending on the silicon generation. Most UK data halls were designed around 5 to 10 kilowatts per cabinet. The arithmetic of that gap is what's now reshaping how AI and high-performance computing infrastructure get built.
This article walks through what's driving the shift to liquid cooling, what immersion cooling actually does that air cannot, and the trade-offs involved before committing to a high-density deployment. Carbon-Z is certified to ISO 14001, ISO 27001, and ISO 45001, and we operate air-cooled, direct to chip, and immersion environments, which gives us a working view across the full cooling spectrum.
The hardware has changed faster than the buildings
The rack, not the server, is the relevant unit of analysis here. A modern AI training rack is not a denser version of an enterprise compute rack. It is a different category of object, with different power, cooling, and floor loading requirements.
Two figures tell the story. First, per-GPU power draw has roughly doubled in one hardware generation. NVIDIA's Hopper-generation H100 and H200 each draw 700 watts. The Blackwell B200 draws 1,000 watts. The rack-scale GB200 NVL72 system pulls 120 to 140 kilowatts in a single cabinet and ships with mandatory liquid cooling. Second, the average rack density across the industry sits in the 10 to 30 kilowatt range, with very few sites operating above 30. The gap between what AI hardware demands and what most facilities can sustain is now measurable in multiples, not percentages.
That gap is why "AI-ready" has become a less useful marketing claim, and why the underlying engineering question, namely, how the facility removes heat at scale, has become the more important one.
What air cooling is actually doing, and where it stops
A conventional data hall pushes cold air through perforated floor tiles, lets it pass through the front of a rack, picks up heat as it crosses the IT equipment, and exhausts the warm air at the back. Containment systems separate hot and cold aisles to stop the two from mixing.
This works because air, while a poor heat conductor, is plentiful and cheap to move. At low to moderate densities, the inefficiency doesn't matter. At AI densities, it does for reasons that are physical rather than economic.
Air carries only a small amount of heat per unit volume. To remove tens of kilowatts of waste heat, you have to move a great deal of it, very fast, through a fixed cabinet aperture. The faster you push, the more turbulence you create, and the worse your cooling becomes at the very points that need it most. By the time a rack is dissipating 40 kilowatts, fan velocity is high enough to create acoustic and vibration issues alongside thermal ones. Beyond around 50 kilowatts in pure air-cooled configurations, removing the heat fast enough using air alone becomes impractical, even with rear-door heat exchangers in place.
Three secondary effects compound the primary one:
Thermal throttling at the silicon level , where the GPU detects elevated junction temperatures and reduces clock speed to protect itself, costing a portion of the performance you paid for.
Uneven cooling across a tall cabinet , with the upper U-positions running consistently warmer than the lower ones, which, over time, wears down components asymmetrically.
Cooling overhead that overwhelms the workload , where the energy spent moving and conditioning air begins to rival the energy spent on computing. Industry-wide PUE has held around 1.5 to 1.6 since 2018 , according to Uptime Institute's annual survey.
Air cooling has not failed. It has reached its ceiling, and AI hardware has moved past it.
What immersion cooling does differently
Immersion cooling abandons the airflow model entirely. Instead of moving cold air past hot hardware, it submerges the hardware in a non-conductive dielectric fluid that absorbs heat directly from every surface in contact with it.
The reason this matters at high density is straightforward. Dielectric fluids carry roughly a thousand times more heat per unit volume than air, and the transfer happens at the chip rather than across an air gap, which means there is no thermal resistance from a heatsink-to-air handoff. Every component is cooled at once, including the secondary heat sources that air cooling tends to underserve, such as voltage regulators, memory modules, and storage controllers. There are no fans, no airflow paths to engineer, and no hot or cold aisles to maintain.
The efficiency consequences are well documented. A peer-reviewed comparative study of air and immersion cooling reports immersion PUE figures of 1.02 to 1.10 against air cooling baselines of 1.5 or higher, meaning the cooling overhead drops by roughly 90% per unit of IT load. The rack density ceiling rises from the 30-kilowatt range that defines high-end air cooling into the 100-kilowatt range and beyond.
Immersion does not solve every problem on its own. Two-phase systems require careful fluid management, including pressure containment and condensate handling. Single-phase systems need coolant distribution units that themselves consume power. Hardware vendor warranties vary on what is permitted in fluid, so a technical compatibility review is a sensible first step before any deployment. These are real considerations, not show-stoppers.
When immersion earns its place, and when it doesn't
Immersion is the right answer for a specific subset of workloads. The pattern that the engineering literature supports, and that we see come up in our own technical reviews, is that immersion is justified when a deployment meets at least two of these conditions:
Sustained rack density above 25 to 30 kilowatts at full load, not peak
Continuous high utilisation, such as inference at scale, batch training runs, or HPC simulation jobs that occupy the cluster for days or weeks at a stretch
Current generation GPU silicon, particularly Blackwell or its rack-scale variants, requires liquid cooling as a vendor baseline rather than an option
Long deployment horizons of three years or more, where the lower component temperatures of immersion meaningfully extend hardware life
Workloads that don't meet at least two of these conditions usually run perfectly well in modern air-cooled environments. Networking, storage, general enterprise compute, and inference at modest scale all sit below the threshold where immersion's economics start to dominate.
How we approach it at Carbon-Z
We run Immersion Cooling and Direct-to-Chip cooling alongside conventional air-cooled racks under the same operator. Our immersion environments are single-phase as standard, with two-phase available on consultation for specific workloads, and they are built to support sustained rack densities from 50 to 200 kilowatts at a measured PUE of 1.03 or below. The hardware our engineering team has validated for deployment includes NVIDIA H100, H200, and A100 GPU nodes, alongside AMD EPYC and Intel Xeon CPU-based servers and custom HPC builds.
Most AI deployments do not start at full density on day one. A common pattern is staging hardware in air-cooled racks while the model and its workload are validated, then transitioning into liquid-cooled environments as utilisation and density rise. Operating multiple cooling regimes under a single operator removes the friction that comes with moving between providers mid-deployment.
If you are sizing infrastructure for a GPU cluster and want a candid view on whether immersion, direct to chip, or a mix is the right fit, our engineers can carry out a technical compatibility review on your specific hardware. The starting point is usually a power assessment , which sets out what the hardware actually demands and where the most sensible cooling architecture sits within that.
What to weigh before signing anything
Selecting a cooling regime and a facility that can host it is more consequential than most procurement processes treat it. Three questions worth pressing on:
The first is sustained versus peak capability. A facility may quote a headline density figure that it can handle for a short burst, but cannot maintain across a full IT load running continuously. Retrofitted immersion infrastructure is particularly prone to this gap.
The second is the fluid lifecycle. Dielectric fluids have long service lives, often a decade or more, but they are not free, and replacement and disposal need to be accounted for in any total cost calculation.
The third is contract flexibility. AI hardware moves on roughly a two-year refresh cycle. The cooling environment you commit to in 2026 should not lock you out of the architecture that becomes standard in 2028.
The shift to liquid cooling at the high end of the market is no longer optional. It has already happened at the hyperscalers and is happening across enterprise AI deployments now. The decision in front of most organisations is no longer whether to engage with it, but how to do so without overbuilding for workloads that do not need it or underbuilding for workloads that do.