Ampere Unveils 256-Core Processor in Data Center Power Play

Ampere Computing has announced plans for a faster, more efficient 256-core server processor to help address soaring data center power demands. The startup chipmaker is also building a new joint solution that combines Ampere’s Arm-based CPUs with Qualcomm’s AI inferencing chip.

Ampere executives today (May 16) said its forthcoming 256-core AmpereOne chip – which will be available next year – will deliver 40% better performance than any CPU on the market today while using the same amount of power as its 192-core AmpereOne chip that was released last year.

Turning Up the Data Center Amps

AmpereOne is a custom, Arm-compatible chip that is designed to meet high-performance requirements while being power-efficient, which makes the processor ideal for not only cloud-native, general-purpose uses, such as databases, web servers, and media delivery, but also AI inferencing, said Jeff Wittich, Ampere’s chief product officer.

For even better AI inferencing performance, Ampere today announced it is developing a joint solution featuring its CPUs with Qualcomm Cloud AI 100 Ultra accelerators for AI inferencing. Supermicro will sell a server powered by both chips, Wittich said.

“Data centers are increasingly consuming more power, and AI is a big catalyst for this,” he told Data Center Knowledge. “We can come in and help with a more efficient solution, whether it’s the most gigantic models with Qualcomm or smaller models that just run on CPUs.”

Related:Cerebras Introduces ‘World’s Fastest AI Chip’ and New AI Server

Analyst Response: Moving Fast in a Competitive Market

Since launching the company in 2018, Ampere CEO Renee James – who was previously Intel’s president – has positioned Ampere as an Arm-based chip alternative to AMD and Intel in the server processor market.

Its cloud provider customers include Oracle Cloud, Google Cloud, Equinix Metal, and Tencent Cloud. The startup is also hoping to make inroads in enterprise on-premises data centers with Ampere-powered servers from Hewlett Packard Enterprise, Supermicro, and other hardware makers.  

While Ampere had a lot of early success with the major hyperscalers, a number of its customers, including Google Cloud, Microsoft Azure, and Alibaba Cloud, have either built or are planning to build their own in-house Arm chips. This could affect Ampere’s business, said Patrick Moorhead, founder and chief analyst at Moor Insights & Strategy.

“Ampere was a first mover with Arm in the hyperscale data centers and had early success. It’s unclear how Google’s Axion and Microsoft’s Cobalt [Arm CPUs] will impact the business there, but it can’t be a positive,” Moorhead said in an interview with Data Center Knowledge.  

Related:Red Hat Strengthens AI and Hybrid Cloud Push with Latest Software

While Ampere is still a niche player in the overall CPU market, it has built competitive products, said Jim McGregor, founder and principal analyst at Tirias Research.

“They’ve got a very competitive argument. They keep innovating with each generation, but the other guys aren’t standing still,” McGregor told DCK. “You have to remember that there’s still a lot of legacy software for x86. There’s still a lot of support for x86.”

Arm CPUs have only captured 9% of the market, while x86 chips still dominate with Intel owning 61% and AMD reaching 27% market share in 2023, according to Omdia. Other companies that have produced Arm CPUs include Amazon Web Services and Nvidia.

Qualcomm Partnership

Ampere executives said the company is targeting cloud service providers and enterprises with its joint AI inferencing solution with Qualcomm.

Cloud service providers will be able to provide scalable inferencing services to their customers in a much more cost-effective and power-efficient way than if they used Nvidia GPUs and x86 CPUs, Wittich said.

Another potential customer is enterprises that prefer to run AI inferencing on-premises because they don’t want to expose their data in the cloud, he added.

Ampere’s CPUs alone can run eight billion to 13 billion parameter large language models (LLMs), Wittich said.

For example, in April, Oracle Cloud Infrastructure announced that it was running Meta’s eight billion parameter Llama 3 on Ampere CPUs. Ampere today said benchmarks show that Llama 3 running on the 128-core Ampere Altra CPU with no GPU provides the same performance as an Nvidia A10 GPU with an x86 CPU while using just one-third of the power and costing 28% less.

Meanwhile, the joint Ampere-Qualcomm chip solution can run inferencing on much larger LLMs, Wittich said.

“When you get to hundreds of billions of parameters or a trillion-parameter model, that’s a specialized enough type of workload that you might want to scale out across something that’s really specialized to do that task – and that’s where the Qualcomm solution comes in,” he said.  

Ampere is the second company to partner with Qualcomm on AI inferencing. AI hardware startup Cerebras, which builds an AI chip for AI training, recently collaborated with Qualcomm, so models trained on Cerebras’ hardware are optimized to run inferencing on Qualcomm’s Cloud AI 100 Ultra accelerator.

The Qualcomm partnership is a good strategy for Ampere, said McGregor of Tirias Research. AI inferencing is a huge market because enterprises want to use AI to create new products and services they can monetize as well as for internal uses, such as improving productivity and analyzing data to make more intelligent decisions.

“Ampere was already well-positioned with high-core count CPUs, especially for more traditional or smaller models. This gives them another option for even more performance on larger models on the inferencing side,” McGregor said. 

Market Outlook: Fresh Partnerships, New Horizons

For Ampere to capture additional market share, the company must expand beyond its hyperscaler customers and sell into more second-tier and next-wave cloud providers, said analyst Matt Kimball of Moor Insights & Strategy.

It was a smart move by Ampere to partner with Qualcomm on AI inferencing because it brings diversity to Ampere’s revenue stream, Kimball said.

As more companies train their models, they need to deploy those models, and a lot of AI inferencing is done on CPUs today, Moorhead said. When performance requirements increase, customers need an accelerator like Qualcomm’s, he added. 

“Ampere has been dependent on cloud service providers for general-purpose compute for the most part. Being able to establish another path in an adjacent workload/market with Qualcomm should bring upside and perhaps position the company for other partnerships,” Kimball told Data Center Knowledge.

To further diversify its market, Ampere has also partnered with NETINT Technologies to create a joint hardware solution that combines Ampere CPUs with NETINT’s Quadra T1U video processing chips to allow companies to run complex video applications, Wittich said. The joint solution enables video transcoding and real-time subtitling of video streams, he said.

AmpereOne: Core Values

Ampere said its 256-core AmpereOne processor is a 3nm chip with 12-channel DDR5 memory. The company will also upgrade its existing 192-core AmpereOne chip from 8-channel to 12-channel DDR5 memory this year. The increased memory bandwidth will improve performance significantly, Wittich said.

Ampere and AMD have led the CPU market by building chips with the highest core counts, McGregor said. For example, AMD’s fourth-generation AMD EPYC processors offer up to 128 cores. 

Meanwhile, Intel’s next-generation “Sierra Forest” Xeon server processor, which is expected this year, will feature 144 cores and reach up to 288 cores.

High core counts are vital for applications, such as communications, web services, database access, and media streaming, McGregor said.

“There are certain segments of the market where the number of cores matter,” McGregor said “The higher the core count, the more efficient you are going to be, and the higher ROI you are going to get out of it.”