Facebook’s Effort to Break Up the Data Center Interconnect “Black Box” Gains Steam

About two years ago, Facebook revealed a project it had in the works whose goal was to do with data center interconnect technology the same thing it had done with servers, storage, and network switches.

The company said it was working with a number of vendors to disaggregate hardware from software in the traditionally monolithic systems that sit on the ends of fiber cables linking data centers to each other. At an event in London this week, a crucial piece of technology built as part of this effort saw its formal unveiling.

Transponder Abstraction Interface is an API that allows vendors to write software for Voyager, the Facebook-designed transponder and router for DWDM (Dense Wavelength Division Multiplexing) networks. DWDM is what enables multiple signals to be sent inside a single optical fiber at different frequencies, completely isolated from each other.

Most data centers communicate with each other over long distances using DWDM. They can be data centers within a single metro or a thousand miles away from each other.

Besides unbundling software from hardware in optical inter-data center networks, TAI abstracts differences in multiple vendors’ hardware, so a single piece of software doesn’t have to be rewritten for each product.

As Cumulus Networks, a key software company behind TAI, put it, the API “removes the friction for network OS vendors to support new hardware and allows chip vendors to take their solutions to market more easily with existing software that can support them out of the box.”

NTT Electronics, subsidiary of the Japanese telecommunications giant NTT Corp. that’s also deeply involved in the project, set up a TAI demo at the TIP Summit this week in London. TIP, which stands for Telecom Infra Project, is an offshoot of the Open Compute Project, the Facebook-founded open data center hardware program.

The demo combined TAI with the Switch Abstraction Interface (the API Facebook and Microsoft built to decouple data center network hardware and software), SONiC (Microsoft’s open source network management software stack, which powers its global Azure cloud platform), and white-box hardware.

Both Voyager and TAI exist because of Open Packet DWDM, Facebook’s disaggregation-driven approach to data center interconnect architecture.

Thanks to DWDM technology, optical fiber’s capacity has grown since the 1990s, while the cost of delivering every bit has gone down, Facebook network engineers wrote in a blog post two years ago, introducing Open Packet DWDM.

In recent years, however, the progress has slowed down. The world’s hunger for bandwidth, however, has only grown. The amount of video content consumed over the internet continues going up, and the anticipated mass adoption of virtual and augmented reality is expected to drive bandwidth demand to balloon even further.

The biggest opportunity to continue improving optical communications now is in disaggregating software from hardware in the interconnect systems. The traditionally monolithic systems have been tightly integrated “black boxes” that include transponders, filters, line systems, and control and management software. If disaggregated, each component can be improved independently, which means bandwidth and cost efficiency gains can be realized sooner, the engineers wrote.

Disaggregation for the sake of faster innovation and lower costs has been a constant theme for infrastructure engineering at Facebook. The company has applied this philosophy to all other hardware it has designed for its data centers, be it servers, storage, or network switches, much of which it has open-sourced through the Open Compute Project.