Survey: "Big Data" Often Misunderstood

A recent survey shows that only 27 percent of industry leaders surveyed understood what "big data" means. Meanwhile, many of those that understand it say they don't have adequate tools to manage their data and mine it for business value.

The survey conducted by IT security specialists LogLogic and Echelon One, included 207 responses from individuals at the director level or above within their organization representing a variety of industries and was done in January and February.

The survey results show significant inconsistencies in practice, according to Bob West, Founder and CEO of Echelon One.

"While big data, cloud needs and compliance requirements are clearly major concerns, the majority of companies are not prepared to deal with any of them adequately," said West. "It’s fascinating to see the rift, and the overwhelming percentage of companies surveyed are not prepared to manage big data properly, monitor cloud environments effectively, or report network and device activities properly."

Big data is often characterized by data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes can range from a few dozen terabytes to many petabytes of data in a single data set. Increasingly businesses face the challenges created by the world of huge data sets and the technology needed to mine the secrets they contain and convert them into useful business intelligence. How retailers leverage big data was recently covered by the New York Times in How Companies Learn Your Secrets.

The survey, focused on big data, cloud environments and IT data, found that there are considerable gaps between theory and practice, with regard to preparation for, and management of big date and cloud environments.

In the category of big data, 49 percent of respondents said they were somewhat or very concerned about managing big data and 38 percent admitted to not having a clear understanding of what it is. More than half of the respondents lack the tools required to manage data from their IT systems, leading to the use of separate, disparate systems and even spreadsheets.

In the category of cloud environments, 72 percent of the respondents said they lack the tools to manage data for their existing and future cloud environments.

Under IT data, respondents identified compliance, security and IT operational efficiency as three of the top four drivers for deploying a log management system.

Mandeep Khera, Chief Marketing Officer of LogLogic, said that big data is about many terabytes of unstructured data.

“Information is power, and big data if managed properly can provide a ton of insight to help deal with security, operational and compliance issues,” Khera said. “Organizations of every size are collecting more data from a variety of resources within the enterprise and the could infrastructures, and many organizations are not using the right tools and processes to manage these data. If this pattern continues, we will see enterprises falling father behind, unable to derive actionable insights, which can help organizations make intelligent decisions.”