ABB Partnership With HPE Cray Could Lead to Hot Swappable Liquid-Cooled PDUs

Three-phase power systems are the life blood of the modern data center, but because electrical generators tend to be old, various countries’ electricity standards mandate different voltage levels. There is a movement under way to adopt 380V direct current as an international standard, among other reasons, because it dissipates much less heat.

But there’s another benefit that’s been hanging around toward the bottom of the list, up to now preceded by asterisks and words like “Other” and “Also,” that now begins to look like the all-time champion for buried ledes: A power system that uses far less copper — as much as 95% less — can be designed more compactly. And such a design becomes a candidate for an inexpensive form of liquid cooling, that could in turn reduce data center temperatures.

That’s one of the goals of an expanded partnership between Cray — now the HPC division of Hewlett Packard Enterprise — and Zurich-based power systems maker ABB. They’re looking to prototype a power delivery system for supercomputers that could dramatically reduce heat emissions, while at the same time cutting expenses.

And if it works for supercomputers, ABB may push for broader adoption throughout enterprise and hyperscale data centers. One possible ticket in: tying innovations in power systems design for supercomputing to advancements in cooling systems for business computing.

“We’ve seen liquid cooling in data centers quite frequently,” remarked Peter Raadsen, product line leader for ABB’s embedded power line, in an interview with Data Center Knowledge.  “But liquid cooling and power are a more recent wedding.”

Much of the design of enterprise servers is still rooted in PC architecture, especially the 12V, 5V, and 3.3V rails that emerged from the ATX form factor. It takes higher amperage to funnel lower voltages, and that’s where the energy loss from resistance comes from. The move toward 48V DC racks has been driven by a need to prioritize energy efficiency and copper conservation over low-cost manufacturing.

But 48V DC turned out to be a handicap for supercomputing. That side of the industry was accustomed to power delivery of 600V AC and 480V AC, stepped down to 208V AC or 120V AC. A rectifier would then convert that power to DC for battery storage, then back to AC for distribution throughout facilities, and then back to DC again for routing to PDUs.

In 2006, Lawrence Livermore Labs launched a study into whether it would just be better to distribute DC power straight from the delivery system to the server racks. LLNL concluded, with the equipment available at the time, that a theoretical efficiency improvement of at least 28% was feasible, if that delivery methodology was applied to 2008 data centers. The voltage level LLNL chose was 380V DC.

“When you have a supercomputer, which is the ultimate gas guzzler,” Raadsen said, “you definitely want to look at ways to get that power from the front end down into the innards in the most effective way. Increasing current over the power distribution is not going to work, because the amount of copper necessary is going to be excessive. The only way to go is to raise the voltage.”

Yet that high voltage presents several, shall we say, challenges for operators. “If you touch 380V,” said Raadsen, “it’s an event you may remember afterwards.”

Now, ABB is advocating a +/- 190V DC solution.

“What it means is that, if you’re an earthed person and you touch any of the two wires, you’re receiving 190V,"  he explained, "which is still a relatively safe voltage. Not as safe as low voltage, which is below 60V, but it’s relatively safe.” The delta between the two feeds still add up to 380V, enabling systems to be powered with 380V while cutting personal risk by half.

ABB’s and HPE’s just published case study claims that their dual-polarity voltage factor is capable of reducing the amount of copper involved in power delivery by a factor of 14, over a delivery system based on PC-derived architecture. Heat dissipation goes down, electricity lossage is dramatically reduced, and both production and delivery costs are slashed.

And as PDUs become smaller, they become easier to cool over a given area of space.

Stage one in the evolution of liquid-cooled power, Raadsen suggested, involves affixing a PDU to a liquid-cooled cold plate, which is a very common method with respect to cooling processors and data storage devices. The next stage, however, would be extraordinary: equipping PDUs with built-in liquid cooling circuits.

“This allows to unplug a power supply and re-plug another supply, without having to reach for your bag of screwdrivers and undo the entire power supply,” he told us. In other words, not only would such a design be liquid-cooled, but also hot-swappable, invoking design elements from the world of telephone data equipment, whose power components have been hot-swappable for some time.

Andre Schaerer, ABB’s vice president of digital partnerships, told Data Center Knowledge to expect a joint announcement with HPE regarding power for smart sensor-equipped manufacturing equipment, during the HPE Discover virtual event which begins June 23.