IBM's Hot-Water Supercomputer Goes Live

ibm-aquasar

A close-up of the IBM Aquasar server blade shows the piping and processor enclosures supporting the water-cooling system.

IBM has delivered a supercomputer cooled by hot water to the Swiss Federal Institute of Technology Zurich (ETH Zurich), which IBM says marks "a new era in energy-aware computing." The Aquasar cooling system uses water at temperatures of up to 140 degrees, and as a result consumes up to 40 percent less energy than a comparable air-cooled machine. The system also uses waste heat to provide warmth to university buildings, reducing Aquasar's carbon footprint even further.

The deployment comes a year after ETH and IBM scientists announced plans to collaborate on chip-level water-cooling and energy reuse. The supercomputer, which is now fully operational, consists of special water-cooled IBM BladeCenter Servers designed by IBM scientists in Zurich and Boblingen, Germany. Each of the blades will be equipped with a high-performance liquid cooler on each processor, as well as input and output pipeline networks and connections, which allow each blade to be connected and disconnected easily to the entire system (see photo above). For direct comparison with traditional systems, Aquasar also includes air-cooled BladeCenter servers.

Six Teraflop System
The Aquasar system achieves a performance of six Teraflops and has an energy efficiency of about 450 megaflops per watt. In addition, nine kilowatts of thermal power are fed into the ETH Zurich's building heating system.

A unique feature of the system is the chip-level cooling with a water temperature of about 60 degrees C (140 degrees F) will keep the chip at operating temperature well below the maximum 85 degrees C (185 degrees). The high input temperature of the coolant results in an even higher-grade output heat of about 65 degrees C. Chiller systems in traditional data centers typically use much lower water temperatures of 45 to 50 degrees F to cool air for use in data center cooling.

The pipelines from the individual blades link to the larger network of the server rack, which in turn are connected to the main water transportation network. The water-cooled supercomputer will require about 10 liters of water for cooling, and a pump ensures a flow rate of roughly 30 liters per minute.

Closed Circuit Cooling System
The entire cooling system is a closed circuit: the cooling water is heated constantly by the chips and consequently cooled to the required temperature as it passes through a passive heat exchanger, thus delivering the removed heat directly to the heating system of the university in this experimental phase. This eliminates the need for energy-hungry chillers.

"With Aquasar, we make an important contribution to the development of sustainable high performance computers and computer system," said Prof. Dimos Poulikakos, head of the Laboratory of Thermodynamics in New Technologies, ETH Zurich. "In the future it will be important to measure how efficiently a computer is per watt and per gram of equivalent CO2 production." ETH Zurich will use the supercomputer for multiscale flow simulations for its research in nanotechnology and fluid dynamics.

"With Aquasar we achieved an important milestone on the way to CO2-neutral data centers," said Dr. Bruno Michel, manager of Advanced Thermal Packaging at IBM Research – Zurich. "The next step in our research is to focus on the performance and characteristics of the cooling system which will be measured with an extensive system of sensors, in order to optimize it further."

Here's a video offering a high-level overview of the Aquasar system, which runs about 2 minutes.

This diagram explains how Aquasar will work. Click the image for a larger version.

ibm-aquasar-schematic-470