Nvidia Has Become the Data Center Accelerator Company

Over a decade ago, the name Nvidia was mentioned in the context of a company called ATI, as a producer of high-powered graphics accelerators that appealed to the then-rising PC gaming market.  A “design win” for Nvidia was when its chips were selected to power Microsoft’s Xbox 360.  On the horizon was something called “high-definition television,” so when Nvidia’s design engineers spoke about the future, they referenced the dream of commanding the home theater market.

On the company’s back burner, whose heat was often turned down to “Lo” and occasionally to “off,” was a project called Compute Unified Device Architecture (CUDA).  It was a little notion that graphics processors — which run small jobs such as shading triangles in very large parallel schemes — could be leveraged for running small everyday tasks, such as running data models, using the same parallel pipelines.

It was a side project that even some Nvidia officials told me at the time was like the minority third-party in a European parliament, sounding sensible enough but never garnering enough votes.

In the intervening decade, someone rebuilt the back burner into a fiery kiln.  Nvidia and the rest of the pipelined processor industry (which includes AMD) began calling the technology GPGPU, where the first “GP” stands for “general purpose.”  And now the industry is experiencing such astonishing growth that no company in the semiconductor space can avoid noticing it, and any firm invested solely in FPGA accelerators must find itself this morning in the midst of a cold, re-awakening shower.

Thursday night, during Nvidia’s quarterly earnings call for fiscal 2017, the company once declared dead or dying for having been left out of Sony’s PlayStation 4 informed the world that its annual revenue growth from data center projects — where GPGPU is now focused — grew almost 300 percent annually to $240 million.

What counts as a “design win” for Nvidia today?  Amazon’s choice of Nvidia Tesla K80 GPU-based accelerators for powering its P2 instance type, announced last September.  And yes, Nvidia won Microsoft over yet again, this time by adopting Tesla M60 accelerators for Azure’s N-Series instances.

Amazon AWS Chief Evangelist Jeff Barr explained the phenomenon in a company blog post last September:  “Processors have reached the upper bound on clock rates, but Moore’s Law gives designers more and more transistors to work with,” Barr wrote.  “Those transistors can be used to add more cache and more memory to a traditional architecture, but the von Neumann Bottleneck limits the value of doing so.  On the other hand, we now have large markets for specialized hardware (gaming comes to mind as one of the early drivers for GPU consumption).  Putting all of this together, the GPU scales out (more processors and parallel banks of memory) instead of up (faster processors and bottlenecked memory).”

It’s a verification of the observations that Intel made back in the spring: essentially, that the types of speed gains that data centers had anticipated would always come from CPUs, must from now on be found by broadening the platform beyond CPUs, and focusing one’s definition of performance from general purpose to specific tasks.  While Intel and other semiconductor firms make investments in FPGA technology to rebuild Gordon Moore’s golden track of progress, CSPs and investors are realizing Nvidia had been growing the same class of solution organically since the early 2000s.

In its last fiscal quarter, Nvidia earned $2.004 billion in revenue, a gain of 54 percent year-over-year, and 40 percent over the previous quarter.  For a company whose seasonality used to be determined by the strength of gaming during the holidays, these numbers are huge for a fall quarter.

“The thing that really made it possible,” said Nvidia CEO Jen-Hsun Huang during yesterday’s quarterly conference call [our thanks to Seeking Alpha for the transcript] “was the transformation of our company from a graphics processor to a general-purpose processor.  And then, probably the more important part of that is transforming from a chip company to a platform company.

“With GPU acceleration, the benefit is incredible,” Huang continued.  “It’s not 20%, it’s not 50%, it’s 20 times, 50 times.  And that translates to, most importantly, for researchers, the ability to gain access to insight much, much faster.  Instead of months, it could be days.  It’s essentially like having a time machine.  And secondarily, for IT managers, it translates to lower energy consumption and, most importantly. . . a substantial reduction in data center cost.  Whereas you have a rack of servers with GPUs, it replaces an entire basketball court of cluster of off-the-shelf servers — so, a pretty big deal.”

The center of server hardware is decentralizing, and the CPU is no longer the center of gravity.  That’s good news for companies like AMD, which lost its lead in that department anyway.  And that’s great news for Nvidia, which placed its GPU in just the right orbit, well in advance.