How heat leaves the building

A watt in is a watt of heat out. Every joule of electricity that reaches a GPU die and doesn’t turn into stored energy — and almost none of it does, since a chip has nowhere to store energy — leaves the die as heat. The entire cooling side of a data center exists to move that heat somewhere it can be released to the outside world without the chip, the board, or the room getting hot enough to fail. What has changed with AI accelerators is not the physics, which is universal, but the amount of heat concentrated in a small area, and that change in degree has forced a change in method.

Why density forced the change

A general-purpose server draws a modest amount of power spread across a rack of many chips, and a fan blowing air across a heatsink is enough to keep each chip within its operating range. An AI accelerator die concentrates far more power into a single package, packed at higher density per rack than before. Air is a poor conductor of heat compared to liquid, and moving enough air fast enough to carry away that much heat from that small an area takes more airflow, louder fans, and more floor space for air handling than most buildings can practically provide. Liquid conducts heat far better per unit volume, so moving the same number of watts takes far less flow and far less space. That single fact — not a preference, a constraint — is why AI racks pushed liquid cooling from a niche option to the default.

Air, hybrid, and liquid

Three cooling modes coexist on the site today, and most racks actually use a mix of the first two.

Air-cooled components are still present even in a liquid-cooled rack: air-cooled heatsinks & vapor chambers spread heat from lower-power chips like memory and networking silicon, and fans move air across them. Cold plates typically only cool the GPUs; the rest of the tray still needs airflow, which is why fans and heatsinks are a hybrid-era holdover rather than something liquid cooling eliminates.

Hybrid describes exactly that arrangement: liquid handles the highest-power dies, air handles everything else on the same tray, and the room still needs some air-handling capacity — crah / crac / fan walls / in-row units and air handling, economizers, filtration — sized to the residual air-cooled load and to serve as backup rather than to the full IT load.

Liquid, specifically direct-to-chip, is now the default path for the GPU itself. It’s also possible to skip cold plates entirely and submerge trays in dielectric fluid — immersion cooling systems — either in an open tank or a sealed chamber where the fluid boils and recondenses against the heat, though this remains a smaller-volume alternative to direct-to-chip.

The path from die to atmosphere

Follow one watt from the silicon to the sky.

It starts at the die, crosses a thermal interface material and a lid or heat spreader — chip-level thermal: lids, heat spreaders, tims — that exist purely to get heat off the package as efficiently as possible before it reaches anything mechanical. From the lid, it’s picked up by a direct-to-chip cold plates: a metal plate with internal microchannels, clamped or bonded directly onto the package, that carries coolant across the die by conduction rather than airflow.

That coolant leaves the cold plate and joins others at a rack manifolds, which distributes supply and collects return flow for every cold plate in the rack, connected through quick disconnects that let a tray be swapped without draining the loop, over hoses, tubing, fittings, valves that carry the flow between components.

All of that rack-level, “technology-side” coolant meets a coolant distribution units, which is the boundary between the electronics and the building. Inside the CDU, a plate heat exchanger and pump — cdu internals: plate heat exchangers, pumps, filters — transfer heat from the technology-side loop into a separate facility water loop, without mixing the two fluids. This separation matters: the loop touching the electronics can be a specific, controlled dielectric or glycol mixture, while the facility loop can be ordinary water plumbed at building scale.

The facility loop — facility water loop hardware — carries that heat out to wherever it’s finally rejected to the outside air. That’s usually a chillers & compressors, which uses a compressor and refrigeration cycle to produce chilled water for the loop, sometimes paired with or replaced by a dry coolers, adiabatic/evaporative units, cooling towers, fans & motors that rejects heat straight to outside air (or to air with evaporative assistance) without running a compressor at all when outdoor conditions allow it. Either way, the last step is the same: heat that started as electricity on a die ends as slightly warmer air leaving the building.

A growing side path skips the atmosphere for part of that heat: liquid-cooled racks run their facility loop hotter than legacy air-cooled buildings did, which puts the rejected heat closer to a temperature a district heating network can actually use, making heat reuse interfaces more viable than they were for air-cooled halls.

Where to go next

• direct-to-chip cold plates — the component that actually touches the die’s heat load.
• coolant distribution units — the boundary between the rack’s coolant and the building’s.
• chillers & compressors — where heat finally leaves the water loop.