NVMe-over-TCP Pioneer Lightbits Supercharges Data Center SSD Storage

Lightbits Labs pioneered the new NVMe-over-TCP standard that enables fast data center SSD storage access over a traditional IP network. Now, backed by a recent Intel investment and technology partnership, the startup aims to capitalize on its invention with storage software that lets data center operators disaggregate and pool their storage.

The four-year-old company, with headquarters in San Jose, California, and Kfar Saba, Israel, came out of stealth in March 2019 with the release of its flagship LightOS software-defined storage solution for cloud service providers and enterprises building private clouds.

The company has since launched two other products: a storage accelerator card that offloads storage functions from a storage server’s CPUs to speed performance and a data center storage appliance for customers that want an all-in-one solution with LightOS and solid state drives pre-installed.  

Lightbits’ technology solves several problems faced by companies that want to take advantage of NVMe (non-volatile memory express) SSDs in their data center storage architectures, analysts and Lightbits executives say.  

Historically, most enterprises and cloud providers have deployed NVMe SSDs in data centers as direct-attached storage because it provides the performance they or their customers need for databases and other data-intensive workloads, such as analytics and artificial intelligence. But that results in poor storage utilization.

Related:How the Latest NVMe Specs Make Flash Storage in the Data Center Better

To disaggregate storage but get the speed of the NVMe protocol, organizations previously had several NVMe-over-fabric options – Fibre Channel, Infiniband, and other remote direct memory access (RDMA) fabrics, but they were complex, expensive, and required specialized equipment.

Enter Lightbits and NVMe-over-TCP. The company’s LightOS, which takes advantage of the new standard, resolves those issues by providing fast performance and low latency as if flash storage is local on direct attached storage in the data center. But it’s done over a network on standard Ethernet TCP/IP networking equipment, Lightbits Chief Strategy Officer Kam Eshghi explained.

“We don’t need to touch the network or compute infrastructure,” he told DCK. “All we are doing is dropping storage servers that have a standard NVMe-over-TCP interface, and suddenly, applications running on those compute nodes can access high-performance storage that’s remote.”

The result is improved storage utilization, resulting in lower total cost of ownership. By disaggregating storage from compute, customers no longer need to overprovision storage resources, Eshghi said.

Related:What Switching to NVMe Means for the Data Center

Executive Pedigree and Backing from Tech Heavyweights

Before receiving an investment by Intel Capital in September (amount not disclosed), Lightbits had raised $50 million in venture capital from Cisco Investments, Dell Technologies Capital, Micron, Square Peg Capital, and WRVI Capital.

Analysts say Lightbits Labs has good potential for success in the massive data center storage market. The company, which has 80 employees and is led by CEO Eran Kirzner, has good technology, a first-mover advantage, and executives with a track record of success, they say.

Lightbits’ co-founder and chair Avigdor Willenz sold his previous company, AI chip designer Habana Labs, to Intel, while chief technology evangelist Amir Michael sold his previous company, data center management software developer Coolan, to Salesforce.

“If you look at business acumen, those are two great examples, but they are also firing on all cylinders on the technical side,” said Eric Burgener, research VP at IDC’s Enterprise Infrastructure Practice.

Some Lightbits engineers previously worked at a flash storage startup called DSSD, which EMC acquired in 2014. DSSD was building a custom NVMe-over-fabrics system, and that experience allowed them, as Lightbits employees, to write the initial specification for NVMe-over-TCP, Burgener told DCK.

The standard, ratified in November 2018, included input from companies such as Intel, Facebook, Micron, Cisco, and Dell EMC, said Eshghi, one of the alumni of DSSD, where he was VP of strategic alliances.

“This engineering team had experience resolving performance problems at that level of scale,” Burgener said. “A lot of them came to Lightbits and worked on the NVMe-over-TCP implementation. They were instrumental in writing the standard that ultimately got approved. So, it’s not surprising that they were the first vendor to come out with an implementation of it.”

Early Market Lead

Lightbits’ competitors include startups Pavilion Data Systems and Excelero, as well as the major storage vendors, such as Dell EMC, Hewett Packard Enterprise, Hitachi, IBM, and NetApp, Burgener said.

While Lightbits was first to market with an NVMe-over-TCP solution, Pavilion recently added support for the new protocol, but the two companies have different architectures and are going after different customers, Burgener said.

For example, Pavilion, which supports other NVME-over-fabric protocols, sells only high-performance appliances that targets customers with very high-end analytics and latency-sensitive environments, such as financial services, healthcare, and media and entertainment, he said.

Lightbits’ appliance is where its business crosses over with Pavilion’s, but Lightbits’ LightOS software offers cloud providers and enterprises a familiar scale-out architecture for data center SSD storage, where they can add nodes to increase performance and capacity, he said.

In the meantime, the large storage vendors plan to support NVMe-over-TCP, but they probably won’t enter that space for another 12 to 24 months, Burgener said. That means Lightbits can grab early market share in the still emerging market for NVMe-over-TCP storage.

“Right now, very little of this is being sold. Lightbits just started selling it,” Burgener said. “However, if you look at the market it is replacing, which is direct attached storage, that is a large multi-billion-dollar-a-year market. Lightbits can’t replace all of it, but they can replace some percentage of it that’s more performance sensitive.”

When Burgener spoke to cloud providers and enterprises several years ago, many didn’t like a non-standard network protocol, called RDMA over Converged Ethernet (RoCE), that some startups like Pavilion supported. It required special, non-standard host bus adapters, drivers, and software.

But those same organizations were excited to try NVMe-over-TCP when it became a standard. Many of them were beta testers for Lightbits, he said.

“The play for Lightbits is to go to these guys and say, ‘I have a better model for you. It’s more efficient. It lets you share storage with better capacity utilization, and you get access to enterprise class storage services, such as replication and deduplication. It runs on industry standard hardware and software,’” Burgener said. “For users of direct attached storage struggling with those issues, this looks interesting.”

In fact, Lightbits executives believe the company’s primary competition consists of customers that have standardized on direct attached storage, Eshghi said.

Gaining Customers

The company has started making inroads in the data center storage market.

It doesn’t sell to hyperscalers, such as Amazon, Google, or Facebook, because they build their data centers using their own custom software. But Lightbits has sold to smaller cloud service providers that offer SaaS and IaaS. It also counts a large hedge fund and financial services company among its enterprise customers, Eshghi said.

In early December, Lightbits released LightOS 2.1, and in the announcement positioned the software as an alternative to AWS Elastic Block Storage.

“For on-premise private cloud, LightOS offers EBS-like functionality with better performance and lower cost than the recently announced Amazon EBS io2 Block Express Volumes,” Eshghi said.

Cloud service providers can also use LightOS to provide their customers a high-performance block storage service that competes with AWS, he added.

Intel Investment and Partnership

Eshghi said Intel’s investment and technology partnership with Lightbits is significant. The two companies are optimizing their products together, so Lightbits’ software can provide better performance and improved total cost of ownership when used with Intel Xeon processors, Intel Optane Persistent Memory, Intel QLC 3D NAND SSD, and other Intel technology, he said.

GigaOm analyst Enrico Signoretti said Intel’s partnership and investment into Lightbits validates the startup. The market potential for NVMe-over-TCP is huge, he said, but whether Lightbits finds success depends on its ability to execute on its strategy.

“The technology is there. Their roadmap is quite good,” Signoretti said. “But the problem with every startup is they have to find a way to market themselves. That’s a real challenge, and that is common to every startup. Time will tell.”

Lightbits executives are optimistic they will succeed based on the advantages of their technology. Its unique benefits, according to Eshghi, include:

“Customers are asking for software-defined. They want something easy to deploy and scale, which is why NVMe-over-TCP is taking off,” Eshghi said.