'Digital Universe' to Add 1.8 Zettabytes in 2011

digitaluniverse

The IDC Digital Universe study projects enormous grpwth in the amount of data beng created and stored each year. (source: IDC, EMC)

More than 1.8 zettabytes of information will be created and stored in 2011, according to the latest IDC Digital Universe Study sponsored by EMC. That's a mind-boggling figure, equivalent to 1.8 trillion gigabytes -enough information to fill 57.5 billion 32GB Apple iPads. It also illustrates the challenge in storing and managing all that data.

The annual Digital Universe study is notable for its effort to quantify and describe the enormity of data our society is generating. But the impressive numbers also underscore a big opportunity, focused squarely on the data center industry.

50-Fold Growth in Information Projected

"Over the next decade, the number of servers (virtual and physical) worldwide will grow by a factor of 10," IDC predicts. "The amount of information managed by enterprise data centers will grow by a factor of 50, and the number of files the data center will have to deal with will grow by a factor of 75, at least. Meanwhile, the number of IT professionals in the world will grow by less than a factor of 1.5."

The implications for the data center industry are two-fold - the demand for physical storage space will continue to increase, and data centers will need to adapt to keep pace. In some cases, that will mean increased automation to allow small staff to manage immense volumes of data. But it will also accelerate the shift toward new infrastructures developed to enable cloud computing and "big data."

Changes Ahead for the Data Center

"Data center architectures and organizational models will need to evolve as big data applications pervade a company's infrastructure," according to IDC. "The IT architectural and organizational approach used in clustered environments like a large Hadoop grid is radically different from the converged and virtualized IT environments driving most organizations' data center transformation strategies."

The survey reinforces the growth trends in storage, virtualization, security and cloud computing - four key business sectors for the study's sponsor, EMC (which also owns VMware and RSA Security).

Finding Value in Big Datasets

One of the biggest challenges involves "big data" - how to manage huge datasets and extract business value from them."Big data will inject high-velocity requirements associated with capture and analysis, as well as results/predictive reporting," IDC writes. "big data deployments require new IT administration and application developer skill sets. People with these skills are likely to be in short supply for quite a while.

"The biggest challenge, however, is the cultural challenge," the report continues. "Today, many of these big data projects are best described as "junior science projects" with a small core of servers and storage assets. However, unless managed closely, these small projects can quickly turn into the next 'Manhattan project" with companywide and industrywide business, organizational, and legal consequences."

This graphic depicts some of the streams of big data (click to enlarge).

digital-bigdata-470