Microsoft Unveils Its First Custom-Designed AI, Cloud Chips

(Bloomberg) -- Microsoft Corporation unveiled its first homegrown artificial intelligence chip and cloud-computing processor in an attempt to take more control of its technology and ramp up its offerings in the increasingly competitive market for AI computing. The company also announced new software that lets clients design their own AI assistants.

The Maia 100 chip, announced at the company’s annual Ignite conference in Seattle on Wednesday, will provide Microsoft Azure cloud customers with a new way to develop and run AI programs that generate content. Microsoft is already testing the chip with its Bing and Office AI products, said Rani Borkar, a vice president who oversees Azure’s chip unit. Microsoft’s main AI partner, ChatGPT maker OpenAI, is also testing the processor. Both Maia and the server chip, Cobalt, will debut in some Microsoft data centers early next year. 

“Our goal is to ensure that the ultimate efficiency, performance and scale is something that we can bring to you from us and our partners,” Microsoft Chief Executive Officer Satya Nadella said at the conference. Maia will power Microsoft’s own AI apps first and then be available to partners and customers, he added.

Microsoft’s multi-year investment shows how critical chips have become to gaining an edge in both AI and the cloud. Making them in-house lets companies wring performance and price benefits from the hardware. The initiative also could insulate Microsoft from becoming overly dependent on any one supplier, a vulnerability currently underscored by the industry-wide scramble for Nvidia Corporation’s AI chips. Microsoft’s push into processors follows similar moves by cloud rivals. Amazon.com Inc. acquired a chip maker in 2015 and sells services built on several kinds of cloud and AI chips. Google began letting customers use its AI accelerator processors in 2018. 

For a company of Microsoft’s scale, “it’s important to optimize and integrate” every element of its hardware to provide the best performance and avoid supply-chain bottlenecks, Borkar said in an interview.  “And really, at the end of the day, to give customers the infrastructure choice.”

Microsoft will also sell customers services based on Nvidia’s latest H200 chip and Advanced Micro Devices’ MI300X processor, both intended for AI tasks, some time next year. Still, the industry seems to be embarking on a lasting shift toward in-house chips. The transition is particularly bad news for Intel Corp., whose own AI chip efforts are running behind. Meanwhile, with Cobalt, Microsoft is joining efforts by Amazon and AMD to grab share in the server chip market, which Intel currently dominates.

Maia is designed to help AI systems more quickly process the massive amount of data required to do such tasks as recognize speech and images. Azure Cobalt is a central processing unit that will come with 128 computing cores – or mini processors – putting it in the same league as products from Intel and AMD. The more cores the better because they can quickly divide work into small tasks and do them all at once. Cobalt also uses Arm Holdings designs, which proponents say are inherently more efficient because they were developed from designs used in battery-powered devices like smartphones. Both chips will be manufactured by Taiwan Semiconductor Manufacturing Co.

Microsoft has engaged in some chip customization before, including working with AMD to design Xbox processors and  developing specialized chips for the HoloLens goggles and Xbox’s Kinect motion controller. But Maia and Cobalt are the biggest and most general-purpose efforts yet—ambitious moves in a tough and expensive industry to crack.

The old joke about Microsoft is that it doesn’t get things right until version 3.0, but in the chip space that’s typically the case for every vendor. Borkar, who spent 27 years at Intel, said she’s confident Microsoft’s first efforts more than meet the mark. “We are going to deploy these next year,” she said.

The company also announced Copilot Studio, software that lets clients customize AI assistant software from Microsoft or build their own AI assistants from scratch. Customers can also design ways for Microsoft’s copilot software to show up in their own existing apps. 

“Customers were very consistent in their feedback,  like, if I just want to get supply chain data in a copilot experience don’t make me go build a whole other bot,” said Charles Lamanna, a Microsoft vice president.