The Shift to Liquid Cooling: Preparing Your Datacenter for NVIDIA GB200 NVL72

NVIDIA GB200 NVL72 at a Glance

The NVL72 operates as a rack-scale system. All 72 GPUs exist within a single NVLink domain, acting as one large accelerator for training and inference.

Why Air Cooling Fails at 120 kW Densities

Thermal load is the heat a rack must shed to keep silicon within operational temperature limits. Nearly all electrical power drawn by a GPU leaves the chip as heat. A 120 kW rack is a 120 kW heater.

Air is a poor carrier for that much heat. Water holds roughly 3,500 times more thermal energy per unit volume than air (4.18 MJ/m³·K versus 0.0012 MJ/m³·K). Removing 120 kW with a 20 °F air temperature rise requires roughly 19,000 CFM of air through a single rack footprint (calculated via sensible-heat formula: BTU/h = 1.08 × CFM × ΔT).

Industry references cap practical air cooling at 25–30 kW per rack. Pushing limits with containment and rear-door heat exchangers might reach 50 kW. None of that touches 120 kW. When thermal management falls short, GPUs throttle clock speeds. The hardware stays online, but throughput collapses, destroying the return on investment (ROI) of generative AI data centers.

Direct-to-Chip (D2C) Liquid Cooling on the NVL72

Direct-to-chip (D2C) liquid cooling is a thermal management technology that circulates liquid coolant through cold plates mounted directly atop heat-producing silicon. On the NVL72, these plates cover the Grace CPUs, Blackwell GPUs, and NVLink switches. Heat transfers into the fluid rather than the surrounding room air.

[Image Placeholder: Diagram of NVIDIA GB200 NVL72 direct-to-chip cooling loops. Alt-text: "Schematic showing primary and secondary liquid cooling loops connected via a CDU for the GB200 NVL72."]

The system relies on two distinct fluid loops:

• Secondary Loop (Technology Cooling System, TCS): Circulates high-purity coolant through the rack manifold and server cold plates.
• Primary Loop (Facility Water System, FWS): Transports captured heat out of the building to a chiller, cooling tower, or dry cooler.
• Coolant Distribution Unit (CDU): The liquid-to-liquid heat exchanger bridging the two loops. It isolates server-side coolant from facility water while regulating flow rates and supply temperatures.

A small residual air load remains because networking and storage modules lack cold plates. Operators often manage this residual heat using a Rear-Door Heat Exchanger (RDHx)—a water-fed coil attached to the rack's exhaust. Treat this residual air load as a strict design input.

Datacenter Retrofit Checklist for the GB200 NVL72

1. Facility Water and CDU Capacity

Start with heat rejection. Everything else depends on the facility's ability to exhaust thermal energy. Published CDU options for the NVL72 include:

• In-rack CDUs: 200 kW capacity (4 °C approach temperature) to 250 kW.
• In-row CDUs: 1.3 to 1.35 MW capacity. These serve roughly ten racks at nominal load.
• Liquid-to-air CDUs: 180 kW to 240 kW units for sites lacking a chilled facility water plant.

Facility water must strictly meet the chemical and particulate limits outlined in the ASHRAE Liquid Cooling Guidelines for Datacom Equipment. Secure exact supply temperature, flow rate, and pressure-drop requirements directly from your OEM.

2. Piping, Manifolds, and Connections

Pressurized fluid networks replace overhead air ducts. Compute trays connect to the rack's liquid manifold via blind-mate quick-disconnect couplings.

• Match pipe and fitting materials (stainless steel, copper, specialized polymers) to the CDU vendor's wetted-metal specifications to prevent galvanic corrosion.
• Install point-leak detection tape at manifolds, under CDUs, and at loop low points.
• Execute rigorous pressure testing on all CDUs, manifolds, and cold plates before introducing compute loads.
• Determine secondary loop routing. Vendors like HPE support both top-feed and bottom-feed configurations.

3. Structural Load and Floor Integrity

Liquid is heavy. A fully populated NVL72 rack weighs up to 1.47 tonnes (3,245 lb). Add the operational weight of an in-row CDU, filled stainless steel manifolds, and heavy power busways. Compare this aggregate point load against your raised floor or concrete slab rating. Require a structural engineer to sign off, as legacy floors were rated for significantly lighter air-cooled racks.

4. Power Delivery Upgrades

Delivering 120 kW to a 600 mm footprint is an extreme electrical challenge. Each 33 kW power shelf requires an independent feed (typically IEC 60309 connectors). Upstream infrastructure must handle peak surges. HPE recommends provisioning overhead busways, PDUs, and upstream breakers for a peak electrical design of 192 kW. Dual-feed (A/B) redundancy doubles this footprint requirement.

The Business Case for Liquid Cooling AI Infrastructure

• Energy Efficiency (PUE): Liquid cooling eliminates massive CRAC (Computer Room Air Conditioning) fan loads. Warm-water D2C systems can push annual Power Usage Effectiveness (PUE) toward 1.06, as demonstrated at NREL's ESIF data center.
• Compute Density: Consolidating 72 GPUs into one footprint slashes the square footage required for AI training clusters. However, operators must account for the new floor space consumed by in-row CDUs and primary pipe routing.
• Heat Reuse and ESG: Liquid loops return water at much higher temperatures than air systems. Facilities can route this high-grade waste heat into municipal district heating networks or internal facility heating systems, directly supporting corporate sustainability goals.

Frequently Asked Questions

• How much power does a GB200 NVL72 rack use? OEM specifications list nominal power draws between 120 kW and 132 kW. However, peak electrical design requirements can reach 192 kW per rack to account for transient loads.
• Can an air-cooled datacenter host a GB200 NVL72? Not without retrofitting. The chips require direct liquid cooling. A facility must install a Coolant Distribution Unit (CDU) and a liquid loop. If the site lacks a facility water plant, specialized liquid-to-air CDUs must be deployed.
• Is a rear-door heat exchanger (RDHx) enough to cool the rack? No. Standard RDHx units handle 30–50 kW of air-exhaust heat. For the NVL72, an RDHx only manages the residual heat from air-cooled networking and storage components, while D2C cold plates handle the primary 120 kW load.
• How heavy is a GB200 NVL72 rack? A fully loaded rack weighs between 1.36 tonnes and 1.47 tonnes (roughly 3,245 lb) depending on the OEM configuration and fluid volume.
• What is a Coolant Distribution Unit (CDU)? A CDU is a specialized heat exchanger that sits between the datacenter's primary water loop and the rack's secondary liquid loop. It regulates coolant temperature, controls fluid flow rates, and physically isolates server-safe coolant from raw facility water.