Cutting Big Data Problems Down to Size

Jeff Rauscher, Director of Solutions Design for Redwood Software, has more than 31 years of diversified MIS/IT experience working with a wide variety of technologies including SAP, HP, IBM, and many others.

JEFF RAUSCHER
Redwood

Recent International Data Corporation (IDC) research indicated that our digital universe will grow from 130 exabytes to 40,000 exabytes by 2020.* (An exabyte is one quintillion (1018) bytes, or one billion gigabytes.)

This seems almost unbelieveable when you consider that Internet traffic only grew past the one exabyte per month mark in 2004, but it’s quite possible. What it shows is the remarkable rate our digital world is expanding. According to one of our partners, IBM, 2.5 quintillion bytes of data are created every day. In fact, the rate of data creation is growing so quickly that 90 percent of the data in the world today was created in the past two years.**

This is big news for business, too. Some analysts estimate that the worldwide volume of business data alone is estimated to be doubling every 1.2 years. It fuels a whole range of new developments across every industry – whether that is generating personalized recommendations, informing supply chain adjustments, processing customer transactions or simply analyzing customer sentiment and feedback. While all of this data offers the promise of valuable insight, managing its sheer volume, as well as the speed with which it is being created is a tremendous challenge. How can companies turn piles of data into useful information?

Big Data Only Useful When Analyzed

“Big Data” is an industry buzzword – heralded as the game-changer for businesses by media and analysts alike. It’s been described as the cure-all for everything from profitability and efficiency woes to insurance against revenue drops and a fast way to find new business opportunities. Nevertheless, Big Data can only provide value when an enterprise can efficiently process it.

To do this, previously inconceivable volumes of data have to be analyzed and understood—very quickly. Most IT enterprises today are not equipped to do this. In fact, the phrase “Big Data” itself came from a reference to data that is too vast, too unstructured, or gathered too quickly for existing IT systems to manage. But enterprises still try. Today, businesses turn to a variety of big-name CRM or ERP solutions to sort through the volume and attempt to identify hidden business insights. With the ever-growing volumes of data, this can be as difficult as finding the proverbial "needle in a haystack."

Data center professionals in search of these information “needles” may need to look no further than to the example of other parts of the enterprise, such as manufacturing or logistics. In these areas, a series of many complex and interconnected steps are executed automatically to achieve the desired result.

Automation Needed

Cutting Big Data down to size and transforming it into “Big Information” is similar to running other business and IT processes. Automation can help support the constant, rapid, accurate and reliable processing of any series of related, dependent tasks in a logical sequence. If you automate the way you process Big Data, you can quickly bring it down to size.

Many of our customers successfully use automated processes that are connected and coordinated across departments, applications and locations to handle some of their most daunting Big Data tasks – such as keeping a data warehouse up-to-date, updating retail customer transaction information, or coordinating entire supply chains—from ordering to delivery. Connecting automation throughout the enterprise brings some of the most complex activities, like dealing with Big Data, back to human scale. It builds on the processes that have already been automated – including individual applications or tasks – pulling everything together without requiring micromanagement.

Smart businesses use automation everywhere – including in how they deal with data. As with the original Industrial Revolution, when people realized how much they could support manufacturing automation, corporate leaders are now spearheading a new Informational Revolution. They’re using connected, automated IT processes to manage huge volumes of data. Connected IT and business process automation provides the flexibility to expand and grow markets, even as data complexity blossoms. The smart businesses of tomorrow will rely on it to transform high-volume and highly complex activities from a Big Data burden into valuable, actionable information.

Endnotes
*IDC Marks Big Data Analytics for Explosive Growth, Newsfactor Bu$iness Report, 8/5/2013
**Improving Decision Making in the World of Big Data, Forbes, 3/25/2012

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.