When Monitoring Runs Amok

Gerardo Dada is VP of Product Marketing and Strategy for SolarWinds Cloud.

Anyone who’s ever binged on their favorite junk food (or even Netflix these days) knows that “too much of a good thing” can quickly become a reality. All too often the same principle can be applied to organizations that leverage cloud monitoring tools. With so many tools available, it’s tempting to monitor everything. However, an abundance of metrics can be redundant, confusing and ultimately transform monitoring from a simple, behind-the-scenes action to a daily responsibility that requires a significant investment in time from your engineering team.

That’s not to say monitoring isn’t important; the cost of downtime alone makes monitoring operational metrics a necessity. Still, without proper planning and implementation, monitoring tools can eventually become a project in and of itself that requires attention and resources that would be better spent somewhere else, while failing to provide the value technology companies expect from monitoring systems.

The truth is, building an effective infrastructure is no easy task. Although the number of commercial and open source monitoring tools on the market has exploded in the last 10 years, it’s unlikely that any one tool can deliver exactly the data and insights that are most valuable to the business. From monitoring bandwidth, security systems, servers, code management and implementation metrics, all the way to high-level business metrics, there are a plethora of data points available to collect. On top of that, as businesses expand, they often feel they need more monitoring power to keep things running smoothly. Consequently, organizations will patch several tools together – each providing different metrics – to create a massive, complex monitoring systems that require dedicated management resources. In these cases monitoring becomes a task itself, rather than providing businesses with a seamless foundation of actionable data.

Business expansion doesn’t need to herald the creation of a “Frankenstein” monitoring system, though. Many start-ups have successfully grown their businesses without losing valuable resources to managing monitoring infrastructure. Take Slack for example, a San Francisco-based start-up that delivers a team communications application to 1.1 million users each day – and doubles its user base every three months. With that kind of exponential growth, not to mention a small team of engineers, Slack’s Ops team has to be smart to keep pace, and that means avoiding mammoth, costly monitoring systems. Deploying one solution that neatly aggregates metrics and results has allowed Slack engineers to become more engaged with data, which equals better availability, security and performance. Unfortunately, streamlining a monitoring system can be an arduous process. In many instances large monitoring systems are simply too interconnected to dismantle without causing significant outages and downtime; at other times they may seem like a necessary evil for a rapidly expanding business. But at a time when engineering talent comes at a premium, you should look to leverage tools that don’t require two engineers to manage them. So what does an ideal monitoring infrastructure look like? It’s different for every organization, but by keeping the following best practices in mind you can ensure the days of managing a massive, patchwork monitoring system are safely behind you.

Avoid Over-Monitoring

In many cases this issue can be addressed by evaluating your organization’s response to two questions. The first, for whom am I monitoring? Are metrics more important to the operations engineer, the product manager, the C-suite? Even within the engineering contingent there may be a wide array of monitoring needs. Second, it’s tempting to monitor everything. But for organizations currently bogged down with complex infrastructures, you likely already know that too many metrics can be redundant and confusing. IT pros should ask themselves “what metrics do I really need?” to keep things running smoothly, without drowning in alerts and data. At the end of the day, investing in a separate tool for each group in your organization is costly and inefficient. Your organization must identify the most valuable audience and metrics to avoid requiring multiple tools.

Assess Resources as you Grow

For organizations that are currently expanding, it may seem necessary to substantially grow your monitoring capabilities as well. But it’s important to note that monitoring is a means to an end – not a task itself. And while monitoring is mission critical and can provide significant increases in time to resolution, it is not a profitable business asset. As your business grows, you should assess the resources you’re allocating to accurately determine what, if any, expansion or investment is needed.

Focus on the Data

It’s easy to lose sight of the point of monitoring – the resulting data that informs business and operational decisions – when your business is weighed down with a complex infrastructure. To see the bigger picture, your business should invest in a new comprehensive monitoring tool based on who truly utilizes metrics, which metrics will deliver the most valuable data for actionable insights, and how much makes sense to spend on a self-sufficient monitoring tool? Once you’ve dismantled your existing, inefficient infrastructure and replaced it with a streamlined tool, you will be able to better realize the benefits of monitoring while at the same time more thoughtfully deploy additional systems or monitoring metrics.

In today’s digital environment, quality of user experience can mean the success or failure of a business. Whether your company provides a specialized service to a small user base or is experiencing explosive growth, maintaining insights into the infrastructure and application level is critical for the operations team. Almost more critical for organizations, however, is to remember that monitoring should not be a task at the expense of valuable IT talent and resources. Rather, your business must employ strategic, streamlined monitoring that provides real business value in and out of the data center.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.