Facebook Follows Google to Data Center Savings

facebook-powerdistribution

A graphic of the new data center power distribution system being implemented at Facebook, which replaces a central UPS with a battery built into the power supply.

Facebook is stepping up its efforts to make its data centers cheaper and more efficient, and is following in the footsteps of Google in several of its key initiatives. Facebook says it's streamlining its servers, and also plans to adopt a novel power distribution design pioneered by Google.

The social network's plans were discussed by Amir Michael, a server and data center engineer at Facebook, in a Nov. 17 engineering Tech Talk in Palo Alto, Calif. Michael joined Facebook in March after six years on the data center team at Google, where he designed cooling and electrical systems.

"The industry has discussed many ways of optimizing servers and data centers," Michael said. "We don't claim to be the original thought leader on many of these things, but we are gonna be one of the few companies who's actually going to begin implementing them."

Shift to On-Board Battery
That's true of the most significant change outlined by Michael: a new power distribution design that removes traditional uninterruptible power supply (UPS) and power distribution units (PDUs) from the data center. The new design shifts the UPS and battery backup functions from the data center into the cabinet, adding a 12 volt battery to each server power supply.

In April, Google revealed its use of a power supply that integrates a 12 volt battery, allowing it to function as an on-board UPS. The company cited this design as a key factor in the exceptional energy efficiency data it has reported for its data centers, including Power Usage Effectiveness (PUE) ratings between 1.1 and 1.2.

Major Efficiency Gains
Facebook says it expects to gain similar efficiency benefits, reducing the energy loss during power distribution from the current 35 percent to about 15 percent. The company said it can lower its power bill by simplifying how electricity travels to its servers. 

In most data centers, a UPS system stands between the utility power grid and the data center equipment. When there is a grid outage, the UPS taps a large bank of batteries (or in some cases, a flywheel) for “ride-through” power until the generator can be started. The AC power from the grid is converted into DC power to charge the batteries, and then converted back to AC for the equipment.

Saving on Equipment
In addition to improving efficiency, the new design could yield enormous savings on equipment purchases. "You no longer need to buy a traditional UPS and PDU system," said Michael. "On the capex side, it's a huge win. This can save millions of dollars that you no longer have to spend on a UPS system. We hope to see the industry move to a model like this."

Facebook's enormous growth is clearly giving it leverage with its vendors, which are working with the company as it customizes its equipment. An example: typical servers use 208 volt power to the servers. Most power supplies can also support the 230 volt and 240 volt configurations now being implemented to capture extra efficiency.

Facebook's new distribution scheme calls for 277 volt power to the servers. "We're working with power supply vendors to create a (server) power supply that will accept 277 volts on the input," said Michael. 

Stripped-Down Servers
Michael said Facebook is also stripping down servers, noting that most servers are tailored for enterprise customers, and include cosmetic features and extras that can impede air flow for cooling. 

"I would prefer to have an ugly server," said Michael. "An ugly server is a beautiful server because I paid less money for it. We can remove a lot of connectors and a lot of cost from the motherboard itself. This can come in the form of removing video cards, removing extra disk connectors we don't need, (and) removing PCIe connectors which really don't provide us with any functionality."

Few Details on Timeline
Michael didn't provide details on Facebook's progress in implementing these new approaches, but the power distribution plans were discussed in the future tense. Michael also outlined the company's plans to make greater use of free cooling, which uses fresh air rather than chilled water to cool servers. Facebook is known to be using free cooling in at least one of its data centers.

Facebook also said it intends to raise the temperature in its data centers, a strategy that expands the number of hours in which you can use free cooling. This is a strategy that is being widely implemented by large data center users, including Google. "Because we're taking more ownership of our server design, we can now design it in a way which allows us to raise the temperature in the data center," Michael said.

RELATED STORIES: