Google Contributes 48V DC Data Center Rack to Open Compute

This month, we focus on the open source data center. From innovation at every physical layer of the data center coming out of Facebook’s Open Compute Project to the revolution in the way developers treat IT infrastructure that’s being driven by application containers, open source is changing the data center throughout the entire stack. This March, we’ll zero in on some of those changes to get a better understanding of the pervasive open source data center.

Google has joined the open source data center and hardware design community created and led by Facebook, called the Open Compute Project, the company announced Wednesday.

OCP, through which Facebook opened up a lot of the specs and designs for its custom data center hardware – compute, storage, and networking – as well as data center racks, power, and cooling systems, has become a hub where companies that build hyperscale data center infrastructure collaborate with vendors and with each other to solve common technology problems.

Microsoft joined OCP in 2014, contributing an OCP-based cloud server design, and Apple joined last year. While Google has open sourced a lot of innovative infrastructure technology, it has resisted going all-in with OCP until now.

And Google is joining with a contribution of its own. The company said it will contribute a spec for a data center rack with 48V power distribution and a new form factor that will enable OCP racks to fit into Google data centers.

One of the early glimpses on technology in Google data centers was a 2006 paper the company published, describing its 12V power infrastructure design for data center racks. Three years later, however, the company started evaluating 48V power distribution, because it promised to provide better efficiency and performance for more power-hungry high-performance computing systems, powered by GPUs and high-power CPUs.

Facebook’s Open Racks are backed up by UPS systems with battery cabinets that provide 48V DC power. The Open Rack spec recommends 12V AC as primary power, however.

Google’s rack supports servers that use 48V power and 48V rack-level UPS systems. “As the industry's working to solve these same problems and dealing with higher-power workloads, such as GPUs for machine learning, it makes sense to standardize this new design by working with OCP,” John Zipfel, technical program manager at Google, wrote in a blog post.

The system supplies 48V power all the way to the motherboard. There is only one conversion step, from DC to DC, stepping down the voltage for each individual component, such as CPU, memory, or disk, depending on what that component needs.

This reduction of conversion steps has resulted in a 30 percent improvement in energy efficiency, Urs Hölzle, senior VP of technical infrastructure at Google, said from stage at the Open Compute Summit in San Jose, California, Wednesday, where he announced the contribution and the addition of Google to the list of OCP members.

“It’s something that we’ve deployed at scale, so we have several years of experience now,” he said.

Google is currently working with Facebook to create a rack standard that suppliers could use, in hopes of pushing it into mass production. If Google’s contribution is accepted by OCP and the standard is developed, Google will deploy racks based on that standard in its data centers, and there are indications that Facebook would do the same, Hölzle said.

“We’re joining OCP for that purpose,” he said, explaining that there was no good reason to have multiple versions of a 48V rack.

Another major difference between Google’s rack and the current Open Rack design is the form factor. Google’s racks are shallower, and its data centers are designed in a way that cannot support the full-depth Open Rack.

“This is something that we need,” Hölzle said. “Our rows aren’t wide enough.” The shallow form factor should work well with most modern motherboard designs, he said.

The racks are backwards compatible to a certain extent. Google still does deploy 12V server trays into 48V racks from time to time. It requires an extra DC to DC converter to step the voltage down at the tray level and costs extra, so the company tries to avoid it when it can.