The Evolution of the Rack-Scale System in the 5G Era

As the world enters the 5G era, the common impression is that 5G will bring down consumer prices of information services. In the 3G days, 1 GB of traffic cost around $141. The figure was slashed to a little over $4 in the 4G days—and may be further lowered to dozens of cents when 5G connection becomes widely available. The free-fall of traffic price means an increase in terminal access, which means the data loads of data centers need to withstand the access pressures that grow across orders of magnitude.

The rack-scale system — one goal, multiple iterations

The 3G and 4G era witnessed the formation and rapid development of cloud data centers and highlighted the emergence of the rack-scale system. Within just several years following its inception, the rack-scale system designed for cloud data centers has become a dominant server form among data centers globally. The Open Rack and Scorpio open standards introduced by OCP and ODCC open computing communities have become the de facto standards in the rack-scale industry. The Open19 community came later, its self-developed Open19 standard boasting unique features, though failing to rival the aforementioned communities in terms of scale.

In the 5G era, the scale of the mobile internet will continue to expand, and the Internet of Things may see significant development. All these factors will drive the development of technologies for next-generation cloud data centers. With data centers growing in both size and quantity, the cost will be dramatically reduced, especially the cost for load processing per unit.

Origin of the three major open standards

The technologies of the three rack-scale service standards are by no means complicated, but each of their technical merits is the essence of long-term technical practices of CSP and server manufacturers. Inspur, as a part of ODCC, developed the world’s first rack-scale system as early as 2010. This product adopted the vertical insert design for nodes and featured centralized power supply and centralized heat dissipation, but failed to become widely accepted in the market. Inspur then redesigned the product and released versions 1.5 and 2.0, changed the vertical insert to a horizontal insert, which led to tremendous sales as a result. Inspur servers still seize a 60% market share of Scorpio servers even today.

The three standards have their respective focuses. Both Scorpio and Open Rack address the needs of large-scale and hyperscale CSP data centers, adopting 21-inch-wide racks, and frequently overlapping development and releases. The communities engaged in close collaboration and contributed content that was highly consistent between the two orgs; where there are differences, the major discrepancies lie in the data center needs in China versus the United States. Back then the power supply current of Chinese racks was only 12A-16A, far underperforming its U.S. counterpart. China needed to increase the device density while lowering the total power within a limit. The two sides also demonstrated a big difference in infrastructure. Open Rack adopts a centralized power supply and in-node heat dissipation, while Scorpio utilizes centralized power supply and fan walls for cooling to reduce fan power and accommodate more devices.

Open19 paves a different path from OCP and ODCC. It targets traditional data centers in hopes to improve designs based on the traditional 19-inch form factor and increase the device density of traditional racks. In this sense, the Open19 standard is the most universally applicable.

Pushing physical limitations with high-voltage power supply and liquid cooling

Currently, the three major standards have reached their limit with regard to enhancing device density. Technically, the power for racks in modern data centers can be increased to over 20KW, but power consumption beyond this point can no longer be sustained by air cooling. To further increase device density, it becomes necessary to introduce a liquid cooling method with higher heat dissipation efficiency. Thus, OCP is developing a new generation of full cabinet server specifications that introduce support for a 48V AC & DC power supply. Racks conforming to the new standard support a total power range of 15KW-33KW. In order to support the high power of 33KW, the new standard also adds liquid cooling components, and raises the rack height from 41OU to 44OU and the maximum weight from 1400Kg to 1600Kg, in a bid to further enhance the space utilization of data centers.

Open Rack 3.0 involves new technologies such as liquid cooling and high-voltage power supply, and many more pending implementations. The standard has not been officially released and is currently open for public proposals.

The Scorpio standard

Open Rack 3.0 marks a leap forward from its predecessor, featuring marked improvements like high-voltage power supply, liquid cooling, and support for 19-inch devices. In contrast, the upgrades of ODCC's Scorpio are more incremental, with version 1.0 released in 2011, 2.0 at the end of 2013, 2.5 at the end of 2016, and 3.0 in 2019.

Scorpio v1.0 achieved modularization of the rack-scale service, including unified fan walls, RMC management, and PSU power supply. 2.0 achieved standardization in operating interfaces, parameters, and various protocols to facilitate the development of the ecosystem. 3.0 achieved high resilience and normalization, including higher power density, improved power supply and heat dissipation, better compatibility, and hyperscale flexible deployment.

10000 nodes delivered in 8 hours_1

10000 nodes delivered in 8 hours

Baidu has deployed Scorpio servers on more than 20,000 racks. Compared with rack-mounted servers, Scorpio enables favorable performance in terms of energy efficiency, density, and overall cost, especially the significant improvement in deployment speed which has improved Baidu's ability to respond to business emergencies. In 2018, Baidu purchased and deployed 50,000 servers in less than one month to support the Spring Festival Gala’s (a major nationwide broadcasting event in China) "Red Packet" program. What's more impressive is that the last 10,000 Scorpio servers were delivered and deployed within just eight hours. It was a record setting deployment by Baidu and its supplier Inspur.

Scorpio 3.0 and Open Rack 3.0 – same but different

Scorpio 3.0 standard is almost identical to Open Rack 3.0 in its overall philosophy. It supports the 19-inch specification, liquid cooling, and 48V power supply. Because the two standards share basically consistent materials, technologies, and application requirements while seeking the optimal solution under the same constraints, they are destined to head for the same destination.

Nevertheless, there are some differences between Scorpio 3.0 and Open Rack 3.0. For example, the power per rack of Scorpio 3.0 is 20-30Kw, instead of 15-33KW. This illustrates differences in not only technology implementations but also business needs and data center environments in China and the United States.

The unique appeal of Open19

At the inception of the Open19 community, both Scorpio and Open Rack standards were in their 2^nd iteration. Since the purpose of the Open19 standard varied greatly from those of OCP and ODCC, it could not leverage the strengths of the Scorpio and Open Rack standards, leading to slow development. But Open19 has its own unique advantages. It designates a “brick” as a modular unit by which systems are composed, making them better adapted to extreme physical environments. Moreover, Open19 invests more in reliability and redundancy, which makes it more competitive in edge computing scenarios.

The Open19 standard uses classification hardware with full power isolation in the server. There is no shared distributed buses on the rack, and each server is fully protected, monitored, and powered by a separate power channel. The power channels are isolated, and each server is equipped with a protective electronic fuse.

Cloud Data Center 2.0 — accelerating open compute synergy

Each era will see the emergence of a specific form of data center. In the Internet services era, the data center was shaped by telecommunication standards. In the cloud computing era, the cloud data center emerged. In the 5G era, an upgraded 2.0 version of the cloud data center may come into being. Liquid cooling, high-voltage power supply, and scale support for heterogeneous accelerated computing will be typical data center upgrades in hardware facilities.

Open compute synergy will gain speed as data centers evolve. It was evident at the 2019 OCP Summit the common purpose of the three communities and their willingness to adopt all forms of collaboration to advance open computing development. In this endeavor, OCP, ODCC, and Open19 communities are partners rather than rivals, exchanging ideas and working together to accelerate the adoption of open standards into the mainstream.