[Report] Overheated GPUs in the AI Era... Cooled with Water

Rapid Rise of Water-Cooled Cooling Systems
Cooling Supercomputers and GPUs Heated by Water and Liquids
Less Noise and Space Usage Compared to Air Cooling
Widespread Adoption Expected... Need for Proactive Preparation

At the headquarters of MK Score, a company specializing in building and operating artificial intelligence (AI) full-stack platforms located in Namyangju, Gyeonggi Province, there is a unique room. In the wooden house-style building, a large supercomputer system simulating a data center has been installed. Pipes attached to both sides of the supercomputer were carrying liquid. Cold liquid enters and comes out as hot water. Each supercomputer was adorned with numerous pipes. When asked if there was a risk of the expensive electronic equipment, worth tens of millions of won, malfunctioning due to liquid leakage, they explained that the system is thoroughly prepared to prevent such incidents.

Kim Jong-hoon, Executive Director of MK Score, is introducing a chiller for a water-cooled cooling system. Photo by Baek Jong-min

The advantages of water cooling were immediately apparent. Although NVIDIA’s ‘H100’ graphics processing units (GPUs) were running, the noise level was different from other data centers. Usually, when visiting a data center, a smartwatch on the wrist warns of noise levels around 80db, but on the 21st of last month, it was only about 60db. Kim Jong-hoon, Executive Director at MK Score, explained, "The water cooling system cools the computer’s heat, resulting in less noise." According to Kim, compared to air cooling, water or lubricant-based cooling maximizes data center efficiency while reducing electricity consumption and noise, offering various advantages.

Cooling electronic servers with liquid is a technology that did not exist in the past. The AI era has changed the situation. As servers using GPUs that emit a lot of heat have increased, cooling technology has gained attention. Just as motorcycles cool their engines with air, large cars use radiators to cool with water, following the same principle. According to MK Score, typical air-cooled servers consume an average of 94 kilowatt-hours (kWh), whereas water cooling reduces electricity costs by more than 10%, using about 82 kWh. This is a significant advantage in AI server environments with high power consumption.

The introduction of NVIDIA’s latest ‘Blackwell’ GPUs is making water cooling technology essential in the industry. Blackwell GPUs consume 1000 to 1200 watts of power, higher than before. They deliver more powerful performance than the previous H100 GPUs, but the increase in heat generation is a ‘necessary evil.’ It is difficult to effectively control this heat with conventional air cooling methods. Microsoft, which provides computing resources for OpenAI’s ChatGPT, announced that it was the first in the world to apply water cooling to the Blackwell GB200 servers it introduced globally.

Supercomputers using a water-cooling system have hoses or pipes connected to them. Cold liquid flows in through the hose on the right, and heated liquid that has cooled the heat flows out through the hose on the left. It is easy to see that the temperature of the hose on the left is noticeably higher than that on the right. Photo by Baek Jongmin

Introducing a water cooling system also increases spatial efficiency. Numerous cooling fans outside the data center can be eliminated. Instead, a chiller, a refrigeration cooling device, is required. A specially made chiller was also placed outside the MK Score headquarters. Chillers were mainly used for cold storage but are now used for advanced data servers. Kim said, "Chillers are also expected to face supply bottlenecks. If not ordered in advance, timely supply may not be possible." He advised that when designing data centers for future new GPU introductions, preparations for adopting water cooling systems must begin early.

The competing method to water cooling is liquid immersion cooling. This method involves submerging servers or computers entirely in liquid to dissipate heat. However, NVIDIA has not officially supported this method. An NVIDIA representative emphasized on social media that water cooling is a basic requirement for data centers housing next-generation GPU servers.

Water cooling has already been applied domestically in South Korea. The Korea Institute of Science and Technology Information (KISTI), which operates the national supercomputer, uses a half water cooling system that does not connect pipes directly to the computers. Lee Sik, Director of KISTI’s National Supercomputing Center, said, "For the upcoming sixth supercomputer, it seems inevitable to adopt water cooling that directly connects pipes or hoses inside the system."