Creating An Effective Data Warehouse Strategy

Alan McMahon works for Dell. He has worked for Dell for the last 13 years involved in enterprise solution design, across a range for products from servers, storage to virtualization, and is based in Ireland.

alan-mcmahon-tn

Every company has a stockpile of data – loads and loads of data. That data may not be that useful however, if you are unable to even access it without dedicating copious amounts of time and effort to the endeavor. That’s where an effective data warehouse strategy comes in.

Contrary to what some companies may still believe, effective data warehouse solutions do not have to be costly. Nor do they have to be complex or limited to a single size and scope.

What No Longer Works

In the so-called olden days, which in the high-tech world can be as recent as last year, data warehousing was attempted using two fairly common methods.

One was relying on external resources to cobble together a system as the company went along. Such systems could contain any number and types of servers, storage arrays and software. When combined, companies hoped such a collection would work as an effective data warehousing solution, although that has become less and less likely of being the case. Disparate units thrown together can create an increasingly complex system that is difficult to monitor, track or manage in an effective manner.

The do-it-yourself approach also runs into trouble for companies who have limited internal IT resources to dedicate to the creation and management of an effective warehousing system. IT resources may not be large enough or enjoy the availability to focus on implementing or managing a sprawling warehousing system.

Another old-school method of data warehousing was going for a system based on proprietary technology. While this type of system may offer the capabilities and technology to meet the needs of many businesses, the cost was typically high. Outlay was costly, as were the ongoing contracts required to ensure the systems would be continuously optimized and maintained. Reaching for proprietary systems could also often result in over-provisioning for the small and medium business. Smaller businesses would not necessarily need such an extensive system but were forced to pay for it anyway, believing it was the only available option.

The drawbacks of former data warehousing methods include high cost, low efficiency, and the simple inability to make any useful sense of the data being stored.

What You Can Do with a System That is Much More Effective

Instead of having vast amounts of unorganized and inaccessible data, an effective data warehouse strategy lets you access the data easily and rapidly for a number of uses. Reviewing various types of data allows you to track past and current trends, while predicting future trends and issue – resulting in meaningful business intelligence reports.

Vast amounts of data stored in an inefficient manner can result in drastically reduced system performance. As data volume increases, so can the amount of time it takes for data to load for even the simplest routine operations. Throw a few queries in there for an attempt to locate a specific item, and the system can lag even further as the system  attempts to sift through or process existing data. These time lags not only affect the employees’ productivity, but they can also affect the company as a whole if downtime or bottle-necked traffic results.

Extensive and ever-expanding data collections are a major challenge for today’s businesses. Internal and external source are constantly adding more data to the mix in a variety of formats and complexity levels. Duplications and redundant data are neither uncommon nor of any practical use.

Online Analytical Processing, or OLAP, can be a very handy application for mining data from different data bases, but it places an extreme workload and pressure on a system that may not be designed to handle anything as complex or large.

Effective data warehousing can also eliminate archaic data storage systems that have long outlived any useful purpose or free up other devices that are too stocked with data to perform additional functions.

What to Look for in an Efficient Data Warehousing System

Capacity and performance are the two big factors to review when choosing a data warehouse strategy. The framework should be capable of supporting and balancing the hardware and software comprising the system can contain important features that are vital to today's enterprises. These include:

Size Matters

The option to size availability is a must, to keep processing speeds high and cost low. Small, medium and large data warehousing options should be available to meet the specific needs of your business. Small and medium businesses, for instance, may do well with a 5 TerraByte (TB) platform consisting of a single server with internal storage. Slightly larger businesses may be able to create an effective strategy using a 10 TB platform with a larger server and internet storage array. The largest enterprises, by contrast, may want nothing less than a 20 TB platform based on a large server and fibre channel storage array that can handle the massive loads.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.