Five Data and Infrastructure Predictions for 2018

Gary Orenstein is Chief Marketing Officer for MemSQL.

1. The Data Lake Transcends Hadoop

Hadoop was born around 2007, and by 2017 it had come full circle. In fact this year, Hadoop completely dropped off of the Data Management Hype Cycle from Gartner.

@mikeferguson1 Gartner declares Hadoop distributions will be obsolete before plateau on their latest Data Management hype cycle.

Hadoop was an enterprise darling for nearly a decade, promising to loosen companies from the grip of pricey, non-scalable data management solutions. But expectations waned and Hadoop lost much of its sheen.

Donald Feinberg @Brazingo

At #GartnerDA many 1-1’s on this. Seems no one wants to use #Hadoop any longer! Maybe this predict will happen by 2019. https://twitter.com/brazingo/status/931487755611275264

Donald Feinberg @Brazingo

By 2020, 30 percent of data lakes will be built on standard relational DBMS technology at equal or lower cost than Hadoop. From @Gartner_inc "Predicts 2018: Data Management Strategies Continue to Shift Toward Distributed" http://gtnr.it/2hHTelD

However, one concept of Hadoop, specifically the Hadoop Distributed File System which provided the “lake” of low cost storage, lives on. This lake is now provided by the cloud object stores such as Amazon S3, Azure Blob Store, and others. Going forward architects are likely to turn to cloud object stores as the data lake of choice.

2. No Data Movement is the New Optimization Metric

The cloud has made it easy to build new applications that collect large volumes of data. But moving that data around can be expensive, and in particular, exporting data from a large public cloud provider can render some applications economically untenable.

Years ago in the on-premises world, we optimized for data center space and power. When large data centers moved to the hinterlands to collocate next to hydro-electrics we focused less on space and almost exclusively on power. Now with the cloud as an application destination, we will turn our attention to data movement.

Intercloud data movement will become the new optimization metric, with the goal to minimize or eliminate the data movement between clouds and on-premises infrastructure. This will impact application planning across the board, and spur a wave of creativity to minimize data movement costs.

3.  Telcos Obviate Edge Computing Push with 5G

As the big telecommunications companies largely lost the cloud race, they seek the promise of edge computing, where they can reclaim their glory as the infrastructure provider of choice. Yet at the same time that AT&T and Verizon promise large enterprises a one-stop shop to manage IoT devices and provide “edge” solutions, they are racing to deliver the next generation of 5G wireless technology.

The irony is that with faster and lower latency wireless connections, much of which is enabled by 5G, the telcos are facilitating the cloud as the go to place for data processing less so than the edge. Understandably a balance will exist, but watch these two forces play out in the coming year.

4.  Successful Enterprises Face the Amazon Gauntlet

There is saying about Amazon that if you are in an important business, they will come after you. And if they are not coming after you, then you must be in an undesirable business. By that measure, all successful businesses will face the Amazon gauntlet, and they will quickly have to plan a survival strategy.

Enterprises keen to extend their life will pursue a path of rapid innovation to stay ahead or outside of the gauntlet. From an infrastructure perspective, this will involve a combination of using pervasive AWS tools, but also a range of solutions that provide a degree of independence. Expect multi-cloud approaches and ownership of data solutions as key themes ahead.

5.  Real-Time Apps Set to Drive the New Economy

We live in a real-time world where no one likes to wait. Increasingly business will set themselves apart by their ability to communicate in real-time with customers, suppliers, and partners. The emergence of a human time channel that is fast enough to keep up with our demands will form the backbone of new data architectures.

Expect the larges industries consuming information technology to move first. This includes finance and the need for real-time dashboards, media and the need for personalized engagement, energy with deployments on new IoT infrastructure, and the public sector with a dramatic need for evolving database and data warehouse solutions to keep up with the times while remaining affordable.

Opinions expressed in the article above do not necessarily reflect the opinions of Data Center Knowledge and Informa.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.