The Facebook Data Center FAQ. (Page 2)

We continue with the Facebook Data Center FAQ (or Everything You Ever Wanted to Know About Facebook’s Data Centers).

How Many Servers Does Facebook Have?

Rows of servers inside the new Facebook data center in Prineville, Oregon.

Rows of servers inside the new Facebook data center in Prineville, Oregon.
“When Facebook first began with a small group of people using it and no photos or videos to display, the entire service could run on a single server,” said Jonathan Heiliger, Facebook’s former vice president of technical operations.

Not so anymore. Facebook doesn’t say how many web servers it uses to power its infrastructure. Technical presentations by Facebook staff suggested that as of June 2010 the company was running at least 60,000 servers in its data centers, up from 30,000 in 2009 and 10,000 back in April 2008.

It’s unclear what that number is today, but it’s bound to be in the hundreds of thousands. In its most recent annual SEC filing, Facebook reported that it owned about $3.63 billion in “network equipment” as of the end of 2015 — up from $3.02 billion in 2014.

What kind of servers does Facebook use?

In a marked departure from industry practice, Facebook has disclosed the designs and specs for its data centers and nearly all hardware they house. In April 2011 the social network launched the Open Compute Project, through which it is releasing the details of its energy efficient data center design, as well as its custom designs for servers, network switches, power supplies and UPS units.

Facebook’s servers are powered by chips from both Intel and AMD, with custom-designed motherboards and chassis built by Quanta Computer of Taiwan and other original design manufacturers. It has also experimented with ARM-powered servers.

“We removed anything that didn’t have a function,” Amir Michael, former hardware engineer at Facebook, said in a past interview. “No bezels or paints.”

Windmill, open sourced in 2012: The Server V2 design, also known as “Windmill,” was a power-optimized, bare-bones motherboard for Intel Xeon processors designed to provide lowest capital and operating costs. It did away with many features that vendors usually include in servers but that aren’t necessary for Facebook’s needs.

The cabling and power supplies are located on the front of the servers, so Facebook staff can work on the equipment from the cold aisle, rather than the enclosed, 100-degree plus hot aisle.

Facebook’s servers include custom power supplies that allow servers to use 277-volt AC power instead of the traditional 208 volts. This allows power to enter the building at 480/277 volts and come directly to the server, bypassing the step-downs seen in most data centers as the power passes through UPS systems and power distribution units (PDUs). The custom power supplies were designed by Facebook and built by Delta Electronics of Taiwan and California-based Power One.

Facebook contemplated installing on-board batteries on its servers, but settled on in-row UPS units. Each UPS system houses 20 batteries, with five strings of 48 volt DC batteries. Facebook’s power supplies include two connections, one for AC utility power and another for the DC-based UPS system. The company has systems in place to manage surge suppression and deal with harmonics (current irregularities).

For a comprehensive list of servers, network switches, and other hardware Facebook has designed and open sourced through the Open Compute Project, visit our Guide to Facebook’s Open Source Data Center Hardware.

What kind of software does Facebook Use?

Facebook was developed from the ground up using open source software. The site is written primarily in the PHP programming language and uses a MySQL database infrastructure. To accelerate the site, the Facebook Engineering team developed a program called HipHop to transform PHP source code into C++ and gain performance benefits.

Facebook has one of the largest MySQL database clusters anywhere, and is the world’s largest users of memcached, an open source caching system. Memcached was an important enough part of Facebook’s infrastructure that CEO Mark Zuckerberg gave a tech talk on its usage in 2009.

Facebook has built a framework that uses RPC (remote procedure calls) to tie together infrastructure services written in any language, running on any platform. Services used in Facebook’s infrastructure include Apache Hadoop, Apache Cassandra, Apache Hive, FlashCache, Scribe, Tornado, Cfengine and Varnish.

The company’s engineers continue building new infrastructure software components, contributing many of them as open source projects.

NEXT: How Much Does Facebook Spend on Its Data Centers?

The Facebook Data Center FAQ | Page 2 | Page 3 | Page 4