Dell and Cloudera Roll Out Spark-Powered In-Memory Processing Appliance for Big Data

Dell, Intel and Cloudera have co-engineered an in-memory processing appliance to run Apache Spark, a stream processing framework for real-time big data analytics. It runs much faster than Hadoop MapReduce runs in-memory, according to Dell.

Pitched as a quick way to deploy a Hadoop cluter, the solution scales up to 48 nodes.

Its roots stemming from a project at University of California Berkeley, Spark is built on top of the Hadoop Distributed File System but isn't limited to MapReduce, which was designed for batch processing. Spark was made for cluster computing and storing data in memory of the cluster nodes for quick access.

Dell's new appliance combines Spark with Cloudera Enterprise, essentially a Hadoop distribution, but one that comes with features like Spark, as well as cluster management and support.