How Edge Data Center Providers are Changing the Internet’s Geography

As people watch more and more of their video online versus cable TV or satellite services, and as businesses consume more and more cloud services versus buying hardware boxes and software licenses, the physical nature of internet infrastructure is changing. Whether your drug is Bloodline on Netflix or Duck Dynasty on Amazon Prime, the web content companies go to great lengths to make sure you can binge in high def. The same goes for cloud services, for whom performance on the user’s end means everything in their busy, competitive market.

In recent years, the big push has been to improve the quality of these high-bandwidth web services to users outside of the top metros like New York, Los Angeles, or San Francisco. And the best way to do it has been caching the most popular content or web-application data on servers closer to the so-called “tier-two markets,” places like Phoenix, Minneapolis, or St. Paul. This push has created a whole new category of data center service providers that call their facilities “edge data centers.” These are facilities that quite literally extend the “edge” of the internet further from the traditional internet hubs in places like New York, Northern Virginia, Dallas, or Silicon Valley.

Examples of companies that describe themselves as edge data center providers include EdgeConneX, vXchnge, and 365 Data Centers. Building something that can be truly called an “edge data center” requires a different set of considerations than building your standard colocation facility. It’s about creating interconnection ecosystems in cities away from the traditional core markets.

That EdgeConneX went from zero data centers two years ago to two dozen today and growing, and that vXchnge bought eight Sungard data centers in tier-two markets in one go earlier this year illustrates just how quickly demand for edge data centers is growing.

The Edge is a Place

Ultimately, location is the main way for companies like EdgeConneX to differentiate from the big colo players like Equinix or Interxion. Edge data center providers are essentially building in tier-two markets what Equinix and its rivals have built in the big core markets: hubs where all the players in the long chain of delivering content or services to customers interconnect and exchange traffic. These hubs are where most of the internet has lived and grown for the bulk of its existence, and edge data center companies are building smaller hubs in places that don’t already have them but are becoming increasingly bandwidth-hungry.

Take for example CloudFlare, a Content Delivery Network and internet security services company. To host its infrastructure in all the top markets, CloudFlare uses the big colos, including Equinix, Interxion, and TelecityGroup, which Equinix recently acquired. But it also recently became an EdgeConneX customer, because EdgeConneX could extend its network to places where the likes of Equinix don’t have data centers, Joshua Motta, director of special projects at CloudFlare, said.

You don’t see Equinix facilities in places like Phoenix, Las Vegas, or Minneapolis, he said. The distinction isn’t as clear-cut as it may seem at first, however. While it’s true that Equinix doesn’t have facilities in those three markets specifically, it does have presence in other cities that can be considered second-tier markets: places like Boston, Denver, and Philadelphia. But the size of Equinix data centers in those markets is very small compared to its marquee facilities in Silicon Valley, New York, London, or Frankfurt. Compare, for example, the nearly half a million square feet across seven Equinix data centers in the New York metro to 13,000 square feet in a single facility in Boston, and you’ll get the idea.

Transport Versus Peering: a Question of Cost

Boston is also an example of a market without a good data center option for CloudFlare, Motta said. There are plenty of data centers in town, and there is even an internet exchange, but there isn’t a data center where transit providers (companies that carry traffic over long distances), access networks (the home or business internet service providers), and content companies come together and interconnect, he said. It’s possible that this problem is specific to CloudFlare, he warned, since the company prefers not to pay for peering with access networks.

A company like Comcast, for example, will gladly sell you transit services and deliver the traffic to the eyeballs, but it’ll do it for a fee. In CloudFlare’s view, both parties bring something of value to this relationship – the CDN provides the content, and the ISP provides the audience – so one company paying a fee to the other wouldn’t be fair, he explained. The company isn’t dogmatic about it of course, but if it costs less to use transit services to bring content over a long distance from point A to point B than it is to pay for peering at point B, peering doesn’t make a whole lot of business sense.

To be sure, ISPs like Comcast do want the kind of content CloudFlare provides. Its customer list includes Reddit, Gilt, OK Cupid, Riot Games, Goldman Sachs, and about 2 million others, ranging from small sites using its free services to big global internet companies.

Some ISPs, including Cox Communications and Charter Communications, don’t charge for peering as a matter of policy, and those are the kinds of ISPs CloudFlare prefers to work with. Both Cox and Comcast happen to be customers and investors in EdgeConneX, according to Clint Heiden, the data center company’s chief commercial officer.

What is an Edge Data Center?

While location is key, being in an underserved tier-two market alone does not make an edge data center. It’s a combination of that and presence of a critical mass of content and web services companies and access networks that truly extends the internet’s edge. In Motta’s opinion, EdgeConneX has made a lot of headway in solving this chicken-and-egg puzzle: you need content to attract ISPs, and you need ISPs to attract content.

Heiden defines an edge data center as a place that connects at least 80 percent of the internet’s content with at least 50 percent of all broadband users in a metro. “Because if you don’t do that, you haven’t moved the edge of the internet,” he said.

How do you get access to 80 percent of the internet into a single building? It’s hard, but it’s not as hard as it may first appear. Only three companies are responsible for moving “just shy of 75 percent” of traffic on the internet, according to Heiden. They are Google (read YouTube), Netflix, and Akamai, which operates the world’s largest CDN, he said. “You add Facebook, Apple, Microsoft, Amazon – you’ve now exceeded 80 percent of the internet.”

Netflix alone accounted for 36.5 percent of downstream traffic in peak evening hours in North America last year, according to a report by Sandvine, a Canadian networking equipment vendor. As reported by Quartz, YouTube’s share at the time was about 14 percent. Akamai wasn’t on the list, because it doesn’t provide the content itself. Like CloudFlare, its business is improving the quality of content delivery to users around the world.

Gangnam Style Serves as Reality Check for Phoenix Internet

Companies like Netflix, Google, and Facebook use CDNs and edge data centers to enhance quality of user experience and to save money – a lot of money. In a recent study conducted by ACG Research and paid for by EdgeConneX, analysts estimated that caching content locally in a metro with a population of about 1 million can save about $110 million in backbone transport costs over five years, which is a 50 percent savings. ACG is an analyst and consulting firm focused on the telco industry.

EdgeConneX executives like to use the Gangnam Style video on YouTube by the South Korean rapper Psy to illustrate this. Before the company brought online one of its data centers in Phoenix, the closest place where a copy of the video was stored was a carrier hotel in Los Angeles, Heiden said. According to him, Cox, which provides internet access in Phoenix, was paying tens of millions of dollars in transport fees to move all the traffic generated by users in the metro who wanted to see the video, which broke all viral-content records on the web when it appeared in 2012. It was the first YouTube video to generate 1 billion views, and today it’s at around 2.4 billion views. The ISP was paying for transport of the data all the way from Los Angeles every time somebody in Phoenix wanted to watch the video, Heiden said.

A Cox spokesman declined an interview but sent us a statement saying that exchanging traffic with content providers in edge data centers, including EdgeConneX facilities, was an “important part of our strategy to better manage our network.”

Now that there is an EdgeConneX data center in Phoenix, the most popular content is stored locally, reducing costs for everyone involved and improving the quality of user experience. To Heiden, the story about Gangnam Style is “representative of what the internet was and what it is today.” Before the EdgeConneX site, there was “no internet in Phoenix,” he said. Sure, you could connect to the internet, but for most online content that connection would be over a very long distance.

And that’s essentially what’s changing about the internet’s physical makeup. Demand for local content caching in places like Phoenix is skyrocketing, and more and more internet hubs come to those markets, moving the edge of the internet outward. ACG predicts that within five years, tier-two markets will have the traffic capacity tier-one markets have today.

Building at the Internet’s Edge

Since it announced the launch of its first edge data center in Houston early last year – like Phoenix, Houston had been an internet no man’s land, served largely out of Dallas – EdgeConneX has built in 24 markets around the US, Heiden said. The company plans to bring up to six more locations online this year.

It buys Class C office buildings, guts them, and converts them to data centers it owns and operates as opposed to leasing from commercial data center providers. It wouldn’t be able to maintain healthy profit margins if it was paying rent to the likes of Digital Realty and re-leasing the capacity to its clients, Heiden explained. “It really requires that we build our own infrastructure,” he said.

As demand dynamics change, the design and expansion strategy for EdgeConneX data centers evolves. The company went from building 1.2-megawatt facilities when it started to facilities that can scale up to 10 megawatts in 2-megawatt chunks. These are also very power-dense designs, averaging 15 kilowatts per rack, Hedien said.

A typical customer starts with about 30kW to cache content that’s particularly popular in a specific market. Their requirements grow over time, and once they reach about 50kW, they combine the caching infrastructure with a network node of their own. Once a customer has their own node in a facility, they start looking at 100kW deals with the provider that can eventually grow into 200kW deployments, Heiden said.

To expand at such velocity is an incredibly expensive undertaking of course, and the company has had to raise a lot of money through a number of avenues. It started with venture capital, but eventually started taking on “strategic investors,” or companies that also use its services. In addition to Cox, Comcast and Akamai were among such investors, according to Heiden. Eventually, EdgeConneX also took private equity funding from Brown Brothers Harriman and Providence Equity Partners.

According to Heiden, the company has now invested about $500 million in infrastructure around the US. “And they’re not empty,” he said about EdgeConneX data centers. “They’re’ all turning a profit.”