Keeping the Video Streaming

Lance Smith is CEO of data virtualization company Primary Data.

YouTube, Netflix, Amazon and a host of other on-demand broadcast content providers have changed the way we consume Internet media, and video is now the vast majority of content we check out online.

Many technology advances improved and redefined media play, transfer and storage to support this culture shift. Let’s review how some of the breakthroughs we’ve seen in recent years will bring even more on-demand media innovation in the near future.

The Video Stream Becomes a River

Many of us stream video today, but we also access video content via standard broadcast (cable, satellite and over-the-air); capture (DVR); and disc (Blu-ray/DVD). Much more content is moving to the cloud, and we are increasingly watching video on social media platforms.

In fact, a recent Business Insider article notes that 63 million people in the U.S. watched a video on Facebook during the month of April 2013. These videos now come from more sources than ever before, as our smartphones can produce high-quality videos, and specialized cameras like the GoPro enable all of us to shoot professional-caliber footage.

Streaming

Keeping video streaming to all of our screens requires more innovation in IT.

The shift to social networks, networked screens, and mobile devices as our primary viewing platforms will push most video content to be streamed exclusively. Enabling offline and mobile viewing on all of these platforms will require a number of innovations in video delivery technologies.

Location, Location, Location

Supporting the increasing volume of streamed video data requires that we solve the hosting performance and delivery bandwidth challenges that currently frustrate online video viewers. These two factors play a significant role in providing high Quality-of-Service to the customer. Whether at the source or the last mile, insufficient bandwidth creates interruptions that result in dreaded buffering pauses. In addition, to ensure smooth performance, the content provider must provide enough processing horsepower to support sudden spikes in simultaneous users and user activity. This can be common when a beloved celebrity dies and suddenly makes archived films “must see” content.

Today, processing performance issues are resolved by using memory caches and low latency storage media like flash. Content providers address basic bandwidth challenges by compressing data and bring hot content closer to users by placing the data on edge servers at key geographical locations around the world. Telco companies are continually expanding en-route bandwidth with scalable solutions such as 4G and LTE for mobile devices. Households insulate against peak-hour congestion with Fiber Optic connections of up to 1Gbps of dedicated bandwidth.

While these solutions work today, they are not sustainable under our current growth rates. Flash storage is currently too expensive to host the massive content libraries of a leading entertainment provider. The manual and semi-automatic movement of data to edge stores is too slow to scale with an unexpected pop culture event.

The next five years will bring technologies that enable video content hosting and delivery to adapt dynamically to rapid changes through data virtualization. Intelligence applications will provide real-time analysis to predict what data is becoming hot, what is cooling off, and then automatically move the data to match resource supply with data demand. This will both improve the viewing experience and increase content providers’ ability to scale cost-effectively.

Making the Most of Metadata

While the bandwidth and storage infrastructure requirements for Full HD and Ultra HD (4k) video files are enormous, an even bigger challenge arises from the significant value that businesses can derive by analyzing metadata about the video and the viewer. Pulled from a variety of different sources, including Web pages, browsing history, search criteria, file attributes and social media, metadata allows companies to gain insight about viewers, market segments, or general trends. This information can then be used for ad-matching, recommendations and future program planning to ensure that content matches our evolving interests.

Making the most of metadata presents two significant challenges for content providers. First, significantly more server and storage resources are required because metadata volume is growing faster than the data itself. Second, conventional databases are ill-suited to handle metadata because it comes from many different sources. This has led to the development of unstructured databases such as Cassandra and MongoDB that scale performance and capacity quickly.

Still, much work remains ahead, and we’ll soon see databases that enable content providers’ applications to access unstructured data more easily. As discussed in the eWeek article, “Hybrid Databases Gaining Favor for Enterprise Big Data Analytics,” new database platforms will accept inputs from many sources, cleanse data to minimize redundancy and worthless data, format it for different applications and high-level databases, and present views for end users and applications. Data virtualization technologies will dynamically manage where data should be stored based on access patterns and other metadata attributes. Cold data will be immediately moved to low-cost storage, while active data will be moved to Tier 0 and Tier 1 media.

The Road Ahead

Over the next few years, the technologies that support online video will continue to be a key source of innovation. What’s even more interesting is how many of these advancements will transfer seamlessly to all data centers as data virtualization is widely adopted. With policy-based dynamic data mobility placing data on the optimal resource for an application’s performance, price, and protection requirements, data virtualization will disrupt conventional centralized architectures, improve customer service levels and accelerate time to market, all while reducing data center costs.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.