eBay Containers Push Frontiers of Efficiency

ebay-mercury-crane

A crane lifting a data center container onto the roof of the eBay Project Mercury data center in Phoenix. The rooftop container deployment has shown exceptional efficiency in early monitoring. (Photo: eBay)

eBay's new containerized data center in Phoenix is one of the most efficient data centers ever built, recording exceptional performance on industry metrics while operating in temperatures as high as 115 degrees F, according to a report released today.

The eBay facility provides some of the best evidence to date that the industry's efforts to share best practices on design and operations have dramatically improved the energy efficiency of data centers.

eBay's Phoenix facility features high-density server deployments that are cooled with hot water, a strategy that works in data centers operating at elevated temperatures. The eBay facility uses traditional raised-floor space, but also deploys servers in data center containers on the roof of the building.

These strategies have yielded unusual efficiency. The Phoenix data center recorded an average Power Usage Effectiveness (PUE) of 1.35 during a one-week period in January, while the rooftop containers had a partial PUE of 1.018. PUE is an energy efficiency metric that compares a facility’s total power usage to the amount of power used by the IT equipment, revealing how much is lost in distribution and conversion. That means that at peak efficiency, the IT equipment in eBay's containers was using more than 98 percent of the available power, an accomplishment which appears to have no precedent.

Ultra-Efficient at 115 Degrees

Perhaps even more impressively, the 20-rack rooftop container achieved a partial PUE of 1.046 in August 2011 when the outdoor temperature was 46 degrees C (115°F). The container, housed outdoors in direct sunlight, was filled with high-density racks averaging 26 kW per rack.

Significantly, eBay's extraordinary ratings for Power Usage Effectiveness (PUE) are being touted by The Green Grid, the group that has championed the use of PUE. The Green Grid is citing the eBay project (PDF) as an example of the benefits of the group's best practices, which are brought together in its Data Center Maturity Model (DCMM).

"This facility is an example of what is possible by establishing and relying on industry best practices for achieving maximum energy efficiency and low environmental impact in data centers,"  said Brad Brech, IBM representative and Green Grid board member. "The Green Grid is at the heart of this mission and the organization will continue to look for new and innovative ways to promote resource efficiency and sustainability within the business computing ecosystem."

Standardization Supports Scalability

A key to eBay's performance is the level of standardization of its server infrastructure. eBay optimized on two server designs - one for high-performance computing clusters, and one for massive Hadoop clusters to perform data analytics. That's a big change from the company's existing infrastructure, which used 15 different hardware designs for about 80 percent of its compute loads, and dozens of custom configurations for the remainder.

Standardizing the servers and data center design allowed eBay to build a modular, flexible facility that could handle extreme density and  scale as needed.

"We wanted to push the envelope in terms of what had been achieved in data center efficiency to date," said Dean Nelson, Senior Director of eBay Global Foundational Services. "eBay was able to achieve both goals because we took a metrics-based approach to the process, which included ensuring that our server and data center RFP processes used PUE and TCO to optimize our supply chain, taking a modular, scalable, and forward-thinking data center design, and aligning ourselves with The Green Grid's DCMM from the beginning."

Large users such as Google, Yahoo, Facebook and Microsoft have reported industry-leading ratings in the 1.07 to 1.2 range. These numbers demonstrate the potential for ultra-efficient data centers, but are sometimes critiqued as having limited relevance for enterprise data centers that must engineer their facilities for many types of applications and workloads. The average PUE for data center facilities is about 1.8, according to the Uptime Institute.

Another criticism was that PUE reports sometimes were reported using different methodology, resulting in apples-to-oranges comparisons. The Green Grid developed the partial PUE metric to define data center energy use "within a boundary," allowing providers to report PUEs for pods within a multi-tenant building or containers.

The eBay facility and its use of the Data Center Maturity Model will be discussed at The Green Grid Forum 2012, to be held March 6-7, 2012 at the DoubleTree Hotel in San Jose, Calif.

mercury-rooftop

eBay has deployed 4,920 servers in these containerized data centers on the roof of its new data center in Phoenix, which have performed exceptionally on energy efficiency metrics. The servers will support eBay’s new search engine and its Hadoop clusters. (Source: eBay)