DataOps: Advancing Data Management

Dan Potter is VP of Product and Marketing and Data Integration Expert for Attunity. 

As we enter a new year, it is time for digital transformation leaders to not only reflect on the developments in 2019 but also to prepare for what lies ahead. As we look back on the changes over this past year such as the increasing popularity of Snowflake or the surprising merger of Cloudera and Hortonworks, we have to start thinking about what market changes we can expect to see throughout 2019 and how will they disrupt the data management industry.

Looking ahead, there are certain solutions that will gain significant importance with special attention being placed on a more collaborative data management approach – DataOps. This methodology is expected to be a major topic of conversation.

DataOps?

DataOps is an evolving approach to agile data management that involves the critical combination of people, processes and technology. It is just beginning to appear on the Gartner HypeCycle and being discussed by other industry analysts as a practice to consider. Focused on improving communication, integration and automation of data flows across the organization, DataOps holds great potential for the transformative effects it will have on data processes. The aim is to help enterprises implement a process to effectively manage and use their ever-increasing data stores in an effective manner, thus reducing the cycle time of data analytics.

However, DataOps is still a new concept and while many may be discussing the concept, there are limited solutions, frameworks or even guidelines to follow. It is the start of the market evolution with many enterprises attempting to provide a loose interpretation. As a result, it is difficult for IT managers and data scientists to know where to begin or how to define the metrics for success. Much like the recent hype and conflicting definitions around AI, we can expect a similar response around DataOps as the discipline gains popularity.

2019: The Year of Education

Based on this challenge, 2019 will be the year dedicated to education on DataOps, and initial discussions will ensue on how to best leverage the principles-based practice for successful implementation. The goal of technology supporting DataOps is to empower enterprises with efficiency and agility by automating data delivery and processes with appropriate levels of security, quality and metadata.

The evolution of this practice can be likened to how DevOps has evolved over the years and its now critical role in IT infrastructure management. Similar to how DevOps changed how applications are built by applying agile practices to product testing and development, DataOps will revolutionize how data is shared, integrated and made available.

As the amount of data being gathered every day increases exponentially, more and more enterprises will turn to DataOps as way to capture and manage their ever-changing data in a more efficient and flexible manner. Ultimately, the technology will improve how data loads are shared, integrated and automated, allowing organizations to move data at the speed of change to stay competitive.

And not only will DataOps help enterprise managers improve the speed and accuracy of their business insights via the automation analytics-ready data pipelines, but the practice will eventually transform how data is consumed across the enterprise overall. The solution will break down the silos in IT operations and build greater accuracy and speed of data analytics. By leveraging real-time integration technologies such as change data capture (CDC) as part of the overall set of principles, DataOps is expected to disrupt typical data processes across the industry as a whole.

As with any other up-and-coming solution on the market, DataOps will be hyped in the coming months while enterprises and industry analysts determine how to optimize it and what practices it will cover. This year will be the time when we educate ourselves on it as it begins its transition from an abstract idea to a tangible practice. Discussions, in-depth research and workshops will start to emerge at industry trade show as we will see DataOps further revolutionize the use of information in this dynamic data era.

Opinions expressed in the article above do not necessarily reflect the opinions of Data Center Knowledge and Informa.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating.