Data Fabrics: What CIOs Need to Know

In the enterprise, data is the fuel that generates insights. It drives faster fraud alerts, greater understanding of customers, and deeper dives into potential sources of value for the organization.

But harnessing the power of the ever-growing store of data has grown more complex and difficult as the data itself has grown. Today there are more data sources coming from many different places. Some of them may be real time. Some of them are structured and some are unstructured. They are stored on-premises or in the cloud. The data needs to be cleaned and validated and managed before it can be used in analytics and machine learning. It needs to be integrated with the applications that will use it on the front end. Also, all this work behind the scenes needs to be done super-fast.

Data fabric is emerging as an integration concept that could potentially come to the rescue. But just what is a data fabric?

What is a Data Fabric?

In simple terms, a data fabric is a layer of technology that sits between the big pile of all different kinds of data on the one side and the applications that consume that data on the other side and enables them to work together without the need for integrating each individual data stream with each application on the other side. Omdia analyst Bradley Shimmin describes data fabric as an API layer or a data-as-a-service layer.

“The fabric depends on a lot of things underneath it being done -- the principles and practices of DataOps,” Shimmin says. “That’s the juice that makes the engines go.”

Maybe in the past, data scientists or data engineers managed and maintained multiple data pipelines to do the same kind of work. Data catalogs are currently shouldering the load here now, Shimmin says. But the growing complexity of enterprise data and the speed with which organizations want to work with that data has driven the demand for this idea of a layer in the middle to manage this work.

Great idea, but how real is the idea of data fabric for the typical enterprise organization right now? 

Hot or Hype?

While the concept may be at the top of technology-oriented C-suite executives’ minds right now, data fabric won’t arrive overnight.

In the Gartner Emerging Technologies Hype cycle report released Aug. 23, 2021, “data fabric” sits right at the “Peak of Inflated Expectations” (along with Nonfungible Tokens or NFTs, and decentralized identity).

Organizations recognize the value of the concept of data fabric. But it’s an emerging technology. Vendors are working on solutions and building components of the data fabric concept into their existing software. Some vendors may offer solutions that incorporate a lot of what data fabric is supposed to do.

However, Gartner predicts that data fabric will only reach the “Plateau of Productivity” in five to 10 years. In the meantime, vendors are refining their solutions and enterprises are testing them. It’s very likely that one of an enterprise’s existing vendors is developing some form of data fabric. Forrester identified several vendors in a report it issued last year.

Forrester called data fabric “a hot, emerging market,” in its Enterprise Data Fabric Forrester Wave report published in Q2 of 2020.

Forrester said the leaders in the market were Talend, Oracle, SAP, Denodo Technology, Cambridge Semantics, and IBM. Strong performers included Qlik, Cloudera, DataRobot, Syncsort, Hitachi Vantara, TIBCO Software, and InfoWorks. Challengers included Informatica and Solix Technologies.

In its 2020 Magic Quadrant for Data Integration Tools, Gartner named the following companies as leaders: Informatica, IBM, Talend, SAP, Oracle, Denodo, and SAS.

Data Fabric in the Cloud

Public cloud players are looking at the market, too. In May, Google announced the preview of Dataplex, which it describes as “an intelligent data fabric that allows organizations to centrally manage, monitor, and govern their data across data lakes, data warehouses, and databases, creating one integrated experience to help automate data management and power analytics at scale.”

All these organizations recognize just why that’s important. It’s not for the data itself. It’s about what the data can do for the enterprise.

As Gerrit Kazmaier, Google VP and GM for database, data analytics, and Looker, said at an event announcing the technology, “Turning data into ability is the key to digital transformation.”