'Digital Universe' Nears A Zettabyte

digital-universe-2010

The Great Recession hasn't slowed the breakneck growth of the Digital Universe. In 2010 the volume of digital information created and duplicated in a year will reach 1.2 zettabytes, according to new data from IDC.

What the heck is a zettabyte? It's equivalent to about 1 million petabytes, 1 trillion terabytes and 1 quadrillion gigabytes. In short, it's an almost unimaginably large number. The annual Digital Universe report is an effort to visualize the enormous amount of data being generated by our increasingly digital lives.

Here's an example: By 2020, IDC projects annual data generation will reach 35 zettabytes. That's enough data to fill a stack of DVDs reaching halfway to Mars.

Storage Implications
All those emails, texts, instant messages, documents, pictures, videos, and social network invitations have to live someplace. Given the storage implications of this data deluge, it's no surprise that the IDC report is backed by EMC Corp., the largest provider of storage systems.

But the report's humongous numbers provide food for thought - and not just thinking about buying more storage. The incredible growth of the Digital Universe challenges the IT community to address how it will manage this firehouse of data.

The biggest challenge isn't how much data we're creating - it's all the copies of it we're making, requiring that the same data to be stored on multiple storage devices. Seventy five percent of all the data in the Digital Universe is a copy, according to IDC.

Case for Deduplication
This factoid makes a compelling argument for the future of deduplication, which saves space on storage devices by eliminating duplicate copies of the same file or data. EMC demonstrated its appetite for this technology when it acquired Data Domain, a leading player in deduplication, after a bruising bidding war with rival NetApp.

The immense volume of information may also create a major staffing challenge. IDC estimates that the number of files to be managed will grow by a factor of 67 between 2009 and 2020, while staffing to manage that data will grow by a factor of 1.4.

IDC foresees automation and management tools as key to tackling this challenge, along with a gradual migration to cloud platforms that offer scalability and favorable economics.

How accurate has the report been? The initial 2007 report predicted that the Digital Universe would reach 988 exabytes this year, indicating that the growth has actually exceeded the projections.