Hybrid Packet-Optical Circuit Switch Networks are New Data Center Standard

Daniel Tardent is the Vice President of Marketing at CALIENT Technologies, Inc.

daniel_tardent_tn

Almost a decade ago, before the ubiquity of smart phones, tablets, cloud computing and video streaming, university and industry researchers were already predicting the need for a hybrid packet-optical circuit switch data center network.

What they anticipated back in 2005 was data centers growing to support tens of thousands of servers and becoming more modular. Only with this hybrid approach, they suggested, could servers be connected to achieve the high levels of performance needed sustainably, in terms of both power and cost.
Now, the day has come when the needs of data centers have caught up with these far-sighted researchers –- driven by the demands of big data, video, virtualization, cloud applications, mobile data and the need to store and replicate vast amounts of data.

These factors generate persistent, dynamically changing traffic patterns, especially in large cloud data centers where a relatively small number of applications can consume vast amounts of server resources. They also result in very large east-west data flows and the need for very low levels of over-subscription. The net result is that traditional packet-based aggregation networks can suffer from bandwidth constriction resulting in increased latency and degraded server and application performance.

The Need for Optical Switches

To solve this using an all-packet switch configuration would mean designing the data center network for worst-case traffic levels; however, this approach causes a large increase in capital cost, and requires more resources to configure. It also potentially adds to the problem by introducing additional latency due to the increased size of the switch fabric.

In the hybrid packet-optical circuit switch network, optical circuit switches (or photonic switches) are installed in a data center to augment packet-based switching to create a hybrid solution. Network management scripts or software-defined networking (SDN) are used to redirect data flows from one network to another. The flow detection and redirection can be based on a number of factors including scheduled events such as virtual machine or data migrations, or real time events such as a network analytic requesting more bandwidth.

Calient-Figure-1-sm

Figure 1: Hybrid Packet-OCS Data center Network Architecture

The OCS fabric offers extremely low-latency paths (less than 60ns) between packet-based top-of-rack switches (TORS) providing excellent support for latency-sensitive applications. It also scales without upgrade to support line rates of 10 Gbit/s, 40 Gbit/s, or 100Gbit/s and beyond as the uplinks on TORS are upgraded. Finally, it supports the demands of today’s low over-subscription scenarios far more cost effectively than all packet-based solutions.
The link setup time of an optical circuit switch is typically 25ms. This is dictated by the fact that electrostatic re-positioning of micro-mirrors is required to achieve the switching. In the packet world, 25ms seems quite high. However, in a hybrid architecture like this, such setup times are completely acceptable, because most large flows persist for minutes or more and so the OCS setup time is irrelevant. In the interim period before an OCS connection is made, the packet network will continue to transport the traffic flow and no data is lost.

The Role OF SDN

To complete the solution, the hybrid packet-OCS network equipment needs a means to control it. This can range from simple scripts to a software-defined network (SDN) implementation with high levels of network intelligence.

Figure 2 shows the well-known multi-layer SDN model with application, control, and infrastructure layers.

Calient-Figure-2-sm

Figure 2 – SDN Model Layers

The important feature is that the packet and optical circuit switches can coexist together in the infrastructure layer with coordinated control from the upper layers.
How this comes together in large data centers is a management plane and a control plane containing both packet and circuit elements as shown in Figure 3.

Calient-Figure-3-sm

The management plane creates and manages topologies and related configurations, and analyzes the various flows within the network during run-time in coordination with photonic and routing-switching control planes.

Based upon operational needs (such as run time triggers, scheduled and cyclic patterns, maintenance activities etc.) it creates new topologies and related configurations and propagates the configurations to the respective control planes for asynchronous execution.

The control planes orchestrate the topology related configuration changes: The photonic engine manages the topology changes across the optical circuit switch fabric while the routing-switching engine further manages the topology changes across various packet-based routing and switching fabric elements.

Both engines maintain the status of execution of each step in the configuration flow and communicate the run time statuses and replies for command processing for topology changes with the management plane.

Hybrid Data Center of Today

The hybrid data center network was first envisioned by researchers a decade ago and is now being realized in commercial data centers where huge traffic loads and variable flow patterns are demonstrating the limitations of traditional all-packet based network architectures.

The hybrid packet-OCS network philosophy offers the capability to handle large persistent data flows with unlimited bandwidth, thereby freeing up the packet network and removing bandwidth constriction. It also offers ultra-low latency (<60ns), which is very important to modern latency-sensitive applications. In contrast, lower over-subscription all-packet-based networks can have high levels of latency due to the size of the fabric. And finally, it provides the ability to scale to 100G and beyond.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.