It's Happening. Arm Server CPUs are Coming to the Data Center

Arm Holdings, the Cambridge, UK-based fabless processor architecture company whose chips are best known for powering smartphones and IoT devices, has been upping its game in the data center market over the last few years, and those efforts are starting to bear real fruit.

Several cloud providers, including the world's largest one, AWS, are running Arm-based chips in their data centers, and last year, Fujitsu's Fugaku supercomputer at a researth center in Japan, powered by Arm CPUs in combination with GPU accelerators, won the title of the world's fastest supercomputer.

In March Arm announced Armv9, the next generation of Arm CPUs, which will serve as the company's backbone for at least the next few years. According to Arm, the new chips will perform better than older models in areas like digital signal processing and machine learning and make Arm systems generally more robust and secure.

All this while sipping power when compared to processors from data center market leaders Intel and AMD.

Arm may also be about to change owners, depending on whether Nvidia will receive all the approvals necessary for its $40 billion agreement to acquire it from SoftBank Group. Even ahead of the sale, Nvidia has been staking a lot of its data center product roadmap on the Arm architecture. Arm powers its BlueField SmartNICs, and its upcoming CPU for AI workloads, codenamed "Grace," is based on the British company's architecture.

Arm in the Enterprise Data Center

All this makes one wonder whether any enterprises have been deploying Arm servers in their data centers or colocation facilities. The answer: a negligible amount, if any. Intel and AMD's x86 architecture continues ruling the roost here.

"I think it's probably a pretty small fraction still," Zachary Smith, managing director of bare metal at Equinix, told us, commenting on the presence of Arm in Equinix's retail colocation data centers. "If you don't count people who have racked and stacked IBM mainframes and other things that are running on RISC architecture and whatnot, the primary workload is still x86. The deploy base over the past 15 to 20 years has just been so dominant x86 that if you would have walked in three or four years ago, you would have been hard pressed to find AMD, although that's changed dramatically in the past three or four years."

We heard the same from other industry sources. They all agreed, however, that Arm adoption in the data center is on the uptick.

"I would say right now we're just at the beginning," said Jeff Wittich, chief product officer at Ampere, which designs Arm server CPUs for hyperscalers and smaller cloud providers. "If I go back three years or so, you could fairly say there was zero Arm in the data center. Obviously, there was some in SmartNICs, you had some other devices, maybe some edge devices, network-processing things, but in the core data center you had nothing."

Arm in the Cloud Data Center

Some of the largest cloud providers are all-in on Arm. Nearly all are using Arm extensively behind the scenes to power their infrastructures, with some offering the architecture to their customers, either for bare metal deployments or as Infrastructure-as-a-Service. Similar to on-prem data centers, however, most Arm servers running in cloud data centers have been in devices like SmartNICs for offloading network processing tasks from CPUs.

"Today probably the biggest usage of Arm-based processors is in Ethernet adapters or SmartNICs, and it just so happens that the cloud service providers have led the adoption of this. But other market segments are also adopting them," said Vlad Galabov, head of the Cloud and Data Center Research Practice at Omdia (owned by Informa, which also owns DCK).

"This enables network protocols, storage protocols, or any type of SDM [security device manager] features to be run on the Arm processor that's on the Ethernet adapter. That just means that the CPU doesn't need to run the protocol, so it alleviates the CPU and it also makes data transfer faster. That's probably the broadest application today, because this market is probably more mainstream than other markets."

That's been changing in the last couple of years, as more and more cloud server providers have begun offering Arm servers to their customers.

"AWS has Graviton2, and then some of the Chinese CSPs have some things from Huawei," Wittich said. "From our side, we've been shipping for revenue since the second half of last year. Oracle Cloud should go live here really soon; they've already announced what they have. Another one that's been a little under the radar [is] Alibaba Cloud, [which] made Ampere Altra available...

"I think you're going to see a couple more big hyperscale CSPs make some announcements this year and bring some things to market," he added. "All of the big infrastructure and service providers [will] have their own Arm-based instances."

From Accelerators to Servers

One of the things that was keeping Arm servers out of data centers was a lack of software that could run on the architecture.

"Over the last two years, a couple things have obviously changed," Wittich said. "The performance of the Arm cores has dramatically improved, that's one thing. Secondly, when you look at what people like Ampere have done with the cores, I think we've finally been able to build an SOC around them that actually does perform for the data center, that acts the way you would expect a server to act, and can connect up to all the devices you'd expect to connect up to, such as the data center's accelerators and smart NICs.

"The other thing is that the software ecosystem is there now," he added. "It was a fair question two or three years ago to ask whether anything really ran. A lot of things ran, but there were some holes." These days, he said, all of the major Linux server distributions support Arm, and FreeBSD just made Arm64 a tier-one architecture. "Windows runs, all the main hypervisors run, there's not really any holes anymore."

Equinix's Smith told us that with operating system and hypervisor support, he's seeing his bare metal cloud customers become more willing to at least consider running workloads on Arm instead of insisting on x86.

"We have a lot of customers who say, I don't care whether it's Arm or x86," he said. "They're like, 'I'm running Kubernetes and it works on both, so it doesn't really matter to me.' They're starting to say, which one's faster or cheaper or which one has more memory. That's where I've seen a big shift in the conversations. We even had some customers with multiple thousands of server requests say, 'We're willing to do Arm, that would be fine. Show us, then show us the economics, and let's test the workload.'" 

Arm Servers On-Prem

Everyone we spoke with for this article thought Arm servers were on their way to biting a bigger share of the on-prem data center market within two to five years, even if their visions weren't exact matches.

Smith said that the uptake he sees for Arm has much to do with its flexibility, since device manufacturers can design Arm chips to meet a specific need instead of having to shoehorn in an off-the-shelf processor from Intel or AMD.

"The digital transformation trend is going to continue and go faster, which means that you're going to see more companies who have highly specific digital workloads," he said. "Those people are going to do all they can to find the right architecture for their software. So, five years from now we have more digital service providers than we do today, and it will be economically viable for them to pick the right tool. As such, I believe that we will see more diverse workloads and alternative architectures."

This doesn't necessarily mean that the rise of Arm will come at the expense of Intel and AMD.

"I also think that we'll see the traditional CPU manufacturers respond. I mean, Pat is not going to just wait around," Smith said, referring to Intel's CEO Pat Gelsinger. "I think it's going to be good no matter what, but we're going to see new types of architectures come out, in terms of the idea of what a computer looks like, very quickly. Jensen's view of the world in terms of a computer is pretty different from Intel's, where it's CPU-centric at Intel and, accelerator- or data-centric at Nvidia," he said, referring to Nvidia's chief exec Jensen Huang.

"I think that in two years maybe we don't see a big change, but we see more operating models. We see some people who used to buy this stuff go away and other people come in who buy a lot more for very specific software. Then, five years from now, we have highly specialized clouds, and those work on more integrated architectures, which very easily could move the needle."

Omdia's Galabov said that because of improvements in Arm brought about by offerings such as Nvidia's Grace and ArmV9, he's expecting a noticeable increase of Arm use by 2025.

"At the moment we expect that Arm-based servers are going to become mainstream. They're going to get close to 10% by 2025, but I think that they could have an acceleration of their curve because of the new performance at the very end of that horizon," he said. "Following 2025, I saw a forecast from a vendor about how in 2030 a huge portion of the market could be Arm."

Galabov added that if five years ago he had forecast that Arm would one day be a major server architecture "people would have thought I'm crazy. But today, seeing a forecast that Arm could even outpace x86 by 2030 is not outlandish. I am not forecasting it, I don't think it's likely, but it's not outlandish."