The Next Wave in Integrated IT Departments: DataOps

Patrick Hubbard is Head Geek at SolarWinds.

DevOps adoption in enterprise IT is showing real progress. While not all teams live in sprints or move cards on Kanban boards, they are finding that the culture of DevOps pays real dividends and isn’t as hard to adopt as feared. And for their next trick, many of these teams then go on to consider other *Ops, like DevSecOps and DevNetOps, all strategies with the same goals: agility, faster deployment, improved end-user experience, and “smart” operational decision-making. Interestingly, the highest performing of these teams often arrive spontaneously to one of the smartest approaches—DataOps.

What makes DataOps smart is that it’s based on culture and process changes, not radical new technology, because the foundation of DataOps is data: IT infrastructure management’s bread and butter. Gathering and analyzing existing data sets in new ways facilitates informed operational decision-making in an organization, and is the next step in further developing high-performing teams. Learning to assure programmatic collection of actionable data is the crucial add to the standard practices that power DataOps.

DataOps: Business Growth through a Data-driven Culture

At its core, DataOps is a data management method that facilitates communication, integration, automation, and collaboration between product management, data engineers, data scientists, and business operations. In essence, DataOps is the natural outgrowth of the practices that allow an operations team to become high-performing in the first place.

Adopting a DataOps mindset will be a natural extension for high-performing teams with an established DevOps culture, as DataOps leverages similar skillsets and tools as those used in a DevOps environment. Better tools are finally available for easier analysis of huge datasets, and even machine and deep learning (ML), without needing to engage PhD data scientists. But for most organizations, a key facilitator is a willingness to do a little selling outside Ops. A greater opportunity to the business may be missed if the only consumer of data-driven operations is operations.

To extend their culture to include DataOps with minimum fuss, a team should ideally be comfortable and experienced with agile environment constructs like the three pillars of observability (logs, metrics, and tracing), and simultaneously present DataOps to business leaders in a way that will be well-received. That means sharing valuable data that’s useful for the business and easily understood by those without a technological background. This, in turn, increases the value of the IT and operations departments, transforming these teams from cost centers to knowledge hubs.

Still, regardless of its proven value to a business, we still have a long way to go: according to NewVantage Partners’ 5th annual survey of senior business and technology executives, 85 percent of respondents reported that their firms have started programs to create data-driven cultures, but only 37 percent report success thus far. The hindrances include managerial understanding, organizational mis-alignment, and general organizational resistance. This juxtaposition—that data is needed to meet business needs and help improve the bottom line, but management and the organization may be resistant to implementing a data-driven culture—can be managed by adopting something new: a data mindset.

Adopting a 'Data Mindset'

In most organizations, pertinent, actionable data is already being collected. But how can the data—beyond infrastructure or tracing data—be leveraged and analyzed to affect the business in a meaningful way? Providing consumable business insights through data can affect the way an organization operates overall. Consider the case of a parts sub-supplier for a large auto manufacturer. We’ll call them JiffyDash.

JiffyDash is a single customer, third-party provider of dashboards in the same town as a large automaker, building low-volume interior parts for a specific trim level coming off the main line. JiffyDash is expected to fabricate and ship parts by truck in fewer than two hours. If the parts sub-supplier doesn’t hit its targets, the manufacturer mainline must shut down its car production because the parts aren’t there. For that auto manufacturer, it’s less about gathering the data infrastructure metrics, and more about analyzing the data pertaining to delivery and dependability: what’s truly affecting their bottom line. And all of that—data, orders, production, shipping, inspection—is available in one place: IT. For JiffyDash, DataOps provides assurance and dashboards to keep the customer informed and happy about the things they care about, like delay risk.

DataOps and the Impact on Technology Professionals

The DataOps mindset goes above and beyond the primary responsibilities of a technology professional and may not (immediately) streamline day-to-day tasks such as improving service delivery or deconstructing applications faster. Moreover, it is a crucial step to take toward transformation and expansion of a technology professional’s role. The insights the IT and operations teams can bring to a business, through the data collected and analyzed, can transform staff and departments into an invaluable source of expertise that business leaders can rely on. Having this data and analysis in hand may even lead them to a seat at the strategy table.

While learning new skills will require an additional time commitment on the technology professional’s part, developing these skills can help them better understand their work environments through metrics and automated measurement. Working toward DataOps adoption will also pay off in knowledge base, value-add, better communication with the business leaders, and for some, to even train algorithms over time to get more insight faster. As such, it’s important for technology professionals to carve out time to “get ahead” of their jobs and make the connection between the bigger picture from both an IT and a business perspective.

Here are a few best practices to help kickstart a DataOps strategy in your organization:

Become a data scientist, (or at least a data healer). In DataOps, it’s crucial for the technology professional to take a more scientific and analytical approach to data, understanding their responsibility to turn data into a business asset rather than packing it up into a file or a table without considering its potential value. This can be accomplished by leveraging tools that provide insight into logs or system metrics and extract useful information from the gathered data. Essentially, technology professionals are connecting business indicators through data that should be shared with business management teams.

Take a formal approach to learning analytics. The DataOps mindset calls for not only critically considering how to leverage the data being collected, but also taking the initiative to analyze and present of that data in a cohesive way. While adopting the “data mindset” is the first step of DataOps integration, analytics is a science, and requires a more formal approach to education. This academic approach will become increasingly beneficial the more integrated a technology professional becomes (mathematics will undoubtedly creep in as the IT administrator and/or DevOps professional delves further into the model definition and training), as (most) pros will need to develop a fundamentally new skillset.

Use data to tell a story. Wading through an ocean of data and extracting and analyzing the pertinent pieces are the first steps of successful DataOps adoption. After that data is collected and analyzed, however, it must be distilled and crafted into a compelling narrative that can be easily understood across the organization, particularly by business leaders and executive decision-makers. DataOps can help the C-suite operate their businesses more effectively if operations in the broader sense can turn this data into reports that are “business digestible” and actionable. As such, data—even niche data collected for specific business units or use-cases—must be polished and presented in a consumable way to the whole company. As a newly minted “analyst,” the technology professional’s job is to get to the heart of what the business leaders value.

Engage the legal team. Ensuring that the data being harvested and operational metrics being collected comply with an organization’s legal policies is crucial. Companies may have certain discoverability metric limits, so it’s necessary to ensure that any data gathered and analyzed conforms to the legal team’s regulations. While data can have significant value-add to a business, collecting data that shouldn’t be widely shared throughout the business may have an adverse effect.

Share knowledge through regular reporting. DataOps helps facilitate effective collaboration between product management, data engineering, data science, and business operations. Disseminating the skills and attitudes that make DataOps successful to individuals throughout the organization can not only make the organization more productive, but can also help the DataOps specialist grow their skills and capabilities. Additionally, collaborating with others during the development of metrics and during the analysis process can help those tasked with managing and analyzing this data extract more insight: observing data with others offers more perspective and intel into what analytics may be useful for and additional insight into how the data may be consumed. Involving others can also increase interest in DataOps, in turn helping to grow DataOps within an organization.

Conclusion

At the end of the day, adopting a DataOps mindset is of considerable value from both a business and IT management perspective. While it may initially feel like one more addition to already overflowing plate, data can provide unparalleled insight to help guide a company’s decision-making and simplify the technology professionals job. Technology professionals who find time to gather, analyze, and distill data can add significant value to their business.

Leveraging best practices such as strengthening data science and analytics skills, learning to translate data into compelling business insights, and sharing knowledge or collaborating with team members will help IT and DevOps departments adopt a DataOps mindset, and in turn develop the skills to become truly data-driven.

Opinions expressed in the article above do not necessarily reflect the opinions of Data Center Knowledge and Informa.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating.