Vendors Take Facebook Data Center Switches to Market

A group of network hardware and software companies are bringing to market data center network hardware Facebook has designed in-house for its own use.

Cumulus Networks, the maker of a full Linux distribution designed for white box switches, is set to deliver today what its CEO described as the first public demo of its OS running on configurations of Facebook’s Backpack and Wedge 100 switch platforms.  This demo is scheduled to take place at an invitation-only technical conference hosted today by Facebook at its Menlo Park, California, headquarters.

As both Cumulus and Facebook confirmed, Cumulus will make Backpack configurations with pre-loaded Linux generally available for pre-order today. The hardware manufacturer is Cumulus partner Celestica.

“[Backpack] is the first modular system ever to be running Cumulus,” explained Cumulus Networks CEO Josh Leslie, in an interview with Data Center Knowledge.

“At a high level, Facebook has been breaking a lot of ground, building very cost-effective, very efficient data center systems.  They’re ground-breaking in the sense that they’ve been incredibly open with how they’ve done things.  In parallel, Cumulus has been building an open, disaggregated network operating system that has gotten an increasing amount of maturity and traction in the marketplace.  And what’s new is that, this is the first time the two will meet.”

The move is part of a string of announcements made today in support of Facebook's Backpack and Wedge 100 switches as commercial products. Barefoot Networks announced a variation on one of Facebook's themes: a so-called "Wedge 100B" that replaces its own switch logic with the company's Tofino fully-programmable PISA-based switch.  In a similar vain, Cavium inserts its own XPliant programmable switch for what's being called "Wedge 100C."

See also: Guide to Facebook's Open Source Data Center Hardware

Disaggregation You Can See

Facebook released its Backpack modular data center switch platform last November, using a hardware architecture that models the way software-defined networks typically subdivide traffic.  With Backpack, blocks of separate switch elements are delegated traffic along the data, control, and management planes.  These elements are fitted to the chassis orthogonally, with the line and fabric cards mounted horizontally and the control and management modules mounted vertically, in a configuration that opens up natural airflow.

Its thermal design, Facebook claims, enables its optical fabric to tolerate operating temperatures of 55 degrees Celsius / 130 degrees Fahrenheit, which is at the high end of network engineers’ typical tolerance scales. (Although when Facebook originally said it supported “55C optics,” some folks were Googling the phrase to find out what grade of fiber it meant.)

“We create hardware that can run multiple software options,” wrote Facebook consulting engineer Luis MartinGarcia for his company’s engineering blog touting today’s news, “and we develop software that supports a variety of hardware devices, enabling us to build more efficient, flexible, and scalable solutions.  By building our data centers with fully open and disaggregated devices, we can upgrade the hardware and software at any time and evolve quickly.”

Fitting One Decoupling with Another

Cumulus Linux amends the typical Linux operating system configuration to include management and support functions for switch hardware.  Network control protocols such as LLDP are treated by the Linux kernel as though they were applications in the user space.  As the company’s chief scientist, Dinesh Dutt, explained in a company video, this enables an opportunity for automation that closed switch architectures could not possibly enable: integration of network control tasks with configuration management and CI/CD tools, such as Puppet and Chef.

Arguably, this opportunity makes white-box switches a more suitable fit for the more microservices-oriented architectures that Web-scale organizations like Facebook need to run.  Granted, there are very few of these organizations in the world, at least in number.  And more commonly-sized enterprises have historically been more comfortable with all-in-one hardware configurations and out-of-the-box management tools.

So it’s interesting that Cumulus should drive an evolutionary course toward the enterprise side, kicking white-box switches more in the direction of everyday businesses.

“These are loosely-coupled systems,” said Cumulus' Leslie, speaking about his company’s architecture.  “We don’t want to do a bunch of special things to make our software work on their hardware.  We want to do a bunch of simple things.  That’s where the value is for customers.

“If they want to move later to next-generation Facebook hardware; or to a fixed configuration system from a Dell, a Mellanox, or an HPE, or if they want to graduate off a chassis later to a more modern architecture, they can do all those things and keep the software and operational model constant.  And that’s very much part of the value proposition of this type of model.”

Making Network Configuration Ordinary Again

Cumulus Linux began making serious inroads in 2015, through deals with vendors including HP (now HPE) and Supermicro to provide bare-metal switches that were certified as “Cumulus-ready.”

A December 2016 Puppet webinar [registration required] told the story of implementing Puppet-automated switches running Cumulus Linux at New York-based 3D printing service Shapeways.  It’s an example of a hyperscale architecture finding a home in a situation that’s not nearly as hyper as Google or Facebook.

“No more having a tool to handle an initial deployment of a switch, another tool to monitor the configuration of that switch, and then another tool to handle ad hoc remediations as necessary,” explained Carl Caum, Puppet’s senior technical marketing manager, during the webinar.

“Traditionally, when you’re trying to get a switch onto the network... it takes hours or days to get it live, and get it actually serving value.  Then for your ongoing management, you have these manual and custom processes.  But with Cumulus Linux and Puppet, you can have ONIE Boot install the operating system onto your new rack switch, and then hand off to Puppet to handle... not just the initial configuration, but also ongoing management, which reduces your time-to-value to minutes or seconds.”

Cumulus’ efforts notwithstanding, there may be two emerging classes of data center switch hardware taking shape, where the on-premise data center looks and works fundamentally differently than for cloud service providers, ISPs, and Web-scale providers.

In an effort to prevent a barrier from forming between those classes, last week Cumulus unveiled its own white-box switch components, called Cumulus Express, with its network Linux pre-loaded.  Tuesday’s demonstration appears to show, however, that Cumulus has no intention to compete with its own hardware partners.  Rather, a plurality of options plays into what Leslie described as his company’s value proposition.

“If there’s some function that you want from the physical network,” he said, “you don’t want to be beholden to the vendor that built those tools, to make it available to you or not.  If you’re building some [network] layer above us, we’re going to work well for you, because we’re going to give you as much control over that environment as is possible, and that’s going to be valuable to you in terms of achieving your business objectives.”