Facebook Scales Servers with Retooled Chef

fb-nc-hall-470

Facebook has adopted Opscode's Private Chef to help manage its fast-growing infrastructure, like this data hall in the company's North Carolina data center.The new version of Chef has been rewritten to enhance its scalability. (Photo:Rich Miller)

Facebook was looking for improved tools to help it manage its growing armada of servers. The team at Opscode was looking for new ways to improve the scalability of its Chef tool for automating infrastructure.

The result: Facebook has deployed a new version of Chef that has been rewritten from the ground up to handle the Internet's toughest scalability challenges. Using Private Chef, Facebook's infrastructure team can manage thousands of servers, configurations, and access policies.

Chef is an open-source framework that simplifies the process of deploying servers. It delivers a flexible model for reuse, enabling users to deploy infrastructure as code across any operating system from Windows to Unix and Linux, across physical, virtual or cloud infrastructures. It's one of three leading offerings to automate the provision and management of huge cloud infrastructures,along with Puppet and CFEngine.

Facebook Helps Refine Open Source Code

Facebook helped field-test the open source code base for Chef 11, which helped speed its delivery to market. "This was a collaboration for some time," said Christopher Brown, chief technology office for Opscode. "With Hosted Chef we've learned a lot of lessons about how Chef works at scale. We found some areas where we wanted to improve the scalability. At about the same time, Facebook came along to give us a platform to scale on."

A key advantage was that implementing Chef required fewer changes to Facebook's infrastructure than other tools it tested.

"Opscode Private Chef provided an automation solution flexible enough to bend to our scale dynamics without requiring us to change our workflow," said Phil Dibowitz, Production Engineer at Facebook. "Private Chef's basis on open-source Chef also aligns with our own open philosophy allowing us to contribute back to the greater Chef community."

"Scale matters, and Chef 11 supports that," said Jay Wampold, Opscode's VP of Marketing. "Facebook is really the poster child for scale. I think we've answered the scale problem for the foreseeable future."

Shift from Ruby to Erlang, CouchDB to PostgresSQL

Chef 11 is written in the Erlang programming language and uses a PostgresSQL database. That's a change from previous versions, which used Ruby as the configuration language and CouchDB as the database.  Opscode says that Chef 11 reduces memory usage 10X over the previous, Ruby-based versions of Chef, while the Erlang-PostgreSQL combo is delivering significantly greater scale. The company says the new Chef server can support up to 10,000 clients on a single server, while only using a fraction of the CPU previous Chef generations required.

The switch from a NoSQL database like CouchDB to an SQL offering may add to the ongoing debate about the merits of the two approaches. But the shift to an SQL database will also make Opscode's offerings more attractive to enterprise customers using Private Chef to automate infrastructure in their own data centers.

"We're packaging a lot of automation capabilities and delivering them to the enterprise," said Wampold. "The enterprise market is going through a transformation right now, and rapidly moving towards managing complex infrastructure. The role of automation is critical."

“Facebook's infrastructure is both truly unique and a model for the future of enterprise computing," said Adam Jacob, Chief Customer Officer, Opscode. "Their use of Private Chef to automate and manage this large-scale infrastructure illustrates the power of Chef in solving some of the most critical and complicated IT challenges on the planet."

New Release of Private Chef

In conjunction with today's announcement of Facebook as a customer, Opscode announced the new version of Private Chef, whose new features include an updated management console, activity reporting, push client runs, role-based access control, and multi-tenancy. Private Chef simplifies systems management through reusable "recipes" and "cookbooks" of code-based configuration and deployment commands

"In just four years, Chef has become the open source standard for infrastructure automation, with tens of thousands of users and thousands of contributors using Chef around the globe," said Jacob. "This new generation of Chef provides greater scale, functionality, performance, and support for both open source users and enterprise organizations. In other words, this stuff rocks, so bring us your hardest infrastructure challenges and let's get to work.”

For the first time, Opscode now offers enterprise-class commercial support to open source users of Chef 11. That includes live system support and cookbook code troubleshooting, with Standard and Premium packages to meet different levels of needs.