SC14: DataDirect Previews IO Accelerator for Supercomputers

At this week's SC14 supercomputing conference in New Orleans, DataDirect Networks announced performance tests and customer adoption of its Infinite Memory Engine, an application-aware acceleration engine and buffer cache.

Jeff Sisilli, senior director of product marketing at DDN, said the software drastically accelerates IO in high performance computing environments by virtualizing commodity SSDs into a single pool of non-volatile memory-based fast data storage. This resource pool then sits between compute clusters and the parallel file system.

This software solution from DDN -- known primarily as a hardware company -- was developed over several years and directly targets improvements in checkpointing and IO-intensive HPC application performance. The company says that placing IME between the compute and parallel file system removes POSIX semantics that bring cluster performance to its knees due to the proliferation of small and mal-aligned IO that can often make up 90 percent of the HPC workload. DDN says that by shielding parallel file systems from fragmented IO in cache users can run jobs at or near line rate with more jobs in parallel, resulting in faster time to insight.

While it won't be released until 2015, DDN has installed preview versions of its IME technology at top supercomputer sites where high bandwidth and high IOPS workloads were required.

Through a collaboration with Intel DDN also launched the latest generation of its EXAScaler Lustre appliance. Built on its Storage Fusion Architecture (SFA), DDN added enhancements to the new EXAScaler for improving performance and simplifying deployment. DDN says this new release delivers over 4.8 petabytes of usable storage, 100 Mbps sustained scalable per-drive performance, up to 40 Gbps sustained throughput and up to 1.5 million IOPS. All of this is in a single rack and powered by Intel's Enterprise Edition for Lustre.

Michael Vildibill, vice president of product management at DDN, said, "A massively scalable, efficient appliance designed for the HPC and Big Data markets, EXAScaler delivers extreme file system performance to help simplify and reduce the cost of building petascale computing solutions for data hungry applications across sectors including scientific research, trading simulation, climate modeling, and energy exploration."