Object Storage: The Future of Scale Out

Tom Leyden is Director of Product Marketing for DataDirect Networks. Tom has more than 15 years’ experience working for disruptive IT companies (from emerging stage to acquisition) including six years in cloud computing. He is based in Belgium.

Tom-Leyden-tn

I’m going to let you in on a secret which will make you sound really smart at your next industry get together. “Object storage is the future to solving all of our data needs.”

Got that? Okay, now let’s talk about why.

Object storage is a new storage paradigm that is seeing growing interest as it addresses the challenge of efficiently storing massive volumes of unstructured data. Big Unstructured Data.

Depending on which analyst firm you talk to, you will hear storage growth predictions that vary between a factor 30 and 40 for the next decades. That means we will all be storing 30-40 times as much digital data ten years from now, compared to today. At the same time companies will only invest an additional 50 percent in personnel to manage their storage infrastructures, meaning that the average storage operator will have to manage 15-20 times as much storage in the next ten years. This calls for storage platforms that require little management and which scale out virtually unlimited.

ddn-1

Billions of Users, Always Online

And here is how you can sound even smarter, especially when talking to the object-storage skeptic. Major cloud application providers like Google, Facebook and Twitter have deployed object storage to meet the requirements of their fast-growing user base - billions of users are storing trillions of objects in infrastructures that were designed to scale infinitely and perform with the lowest latency.

Pay as you go services like Amazon S3 enabled a whole ecosystem of SAAS providers who are now collectively looking at deploying in-house solutions to further optimize costs and performance. Enterprises and other verticals are following closely: private storage clouds, integrated with collaboration platforms are replacing the concept of file servers.

Power to the Applications

While traditional file based storage is still a smart option for some companies, today’s file based storage platforms don’t scale nearly effectively enough to meet the explosion of storage needs. Not only do file-based storage platforms fail to scale sufficiently, they’re also becoming obsolete as more and more applications are designed to use REST API’s – the default interface for object storage platforms – to talk directly to the storage, without additional (file system) layers in between.

So, why should companies use Object Storage for unstructured data and how is it different from NAS or SAN?

ddn-2

The biggest problem with those systems is scalability: NAS does not really scale as a single system, at least not when talking about petabyte-size environments. SANs are already pretty complex when deployed with a file system layer on top. Scaling-out makes the problem a lot worse.

Object Storage Offers Solutions to Big Data Problems

Object Storage is essentially just a different way of storing, organizing and accessing data on disks; it is a storage infrastructure to store … objects, which are essentially files with lots of metadata added to them. The backend architecture of an object storage platform is designed in such a way that all the storage nodes are presented as one single pool; there is no file system hierarchy. The architecture of the platform, and new data protection schemes (vs. RAID, the de-facto data protection scheme for SAN and NAS) allow this pool to scale virtually unlimited while keeping the system simple to manage.

Users access object storage through applications that will typically use a REST API, an internet protocol, optimized for online applications. This makes object storage ideal for all online, Cloud environments. When objects are stored, an identifier is created to locate the object in the pool. Applications can quickly retrieve the right data for the users through the object identifier or by querying the metadata – information about objects like the name, when it was created, or by who. This is a lot faster than trying to locate a file through a file system.

Ultimately as data explodes, more and more companies will be forced to look at their storage solutions and invest in sustainable, long-term solutions. We believe that object storage will play a major role in addressing those problems. Object storage solutions are designed to scale beyond petabytes as one single system and optimize overall TCO without compromising performance or durability. This results in a platform for a variety of storage cloud solutions, including online collaboration, active archives, cloud backup and worldwide data distribution.

So at the next cocktail party or industry mixer, when people start talking about trying to manage the exploding growth of unstructured data, feel free to walk them through how object storage can address their problems. It’s a smart solution to a growing problem.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.