Building Out the Edge

If you’ve checked out what’s happening in the edge computing space, you’ve probably noticed that there’s not even a glimpse of anything resembling a common taxonomy of players and use cases. It’s a new space, and most people in it have been making it up as they go along.

Absent a playbook, one of the best ways to understand an emerging market is to look at what specific players are doing and why. StackPath, the three-year-old startup founded by Lance Crosby – whose last company, SoftLayer, sold to IBM for $2 billion in 2013 – makes for a telling case study in edge computing. It’s both a consumer of edge computing infrastructure and a provider of services this kind of infrastructure enables, so it has visibility both up and down the food chain.

We recently sat down with Crosby to get a better understanding of the strategy behind his company’s ongoing edge infrastructure buildout. As of late September, StackPath had hardware in 45 locations – all colocation data centers that host major internet exchanges – and it’s adding more all the time. In the next phase of buildout, Crosby said, he’ll be going out to cell towers.

What Is It Good For?

So, what is he building this infrastructure for?

When StackPath came out of stealth back in 2016, it billed itself as a provider of security as a cloud service – a Salesforce for security, if you will.

Security still plays a big role, but the company now offers much more than that. It’s made six acquisitions since launching publicly two years ago, adding infrastructure monitoring, DDoS mitigation, Web Application Firewall (WAF), and VPN capabilities, as well as a content delivery network consisting of two companies’ infrastructure.

One example of what it can do now is helping software vendors push out updates to users faster and cheaper. A software company with a data center in Silicon Valley, for example, doesn’t have to send a copy of an update to every user across the country and pay the associated data transport costs. Instead, it can send a single copy to a StackPath site in every major metro. StackPath will then deliver it to users within those metros. Users in Atlanta will get their updates from an Atlanta data center; users in Dallas will get theirs from Dallas.

This straightforward CDN use case is only an early-stage capability in a much bigger vision Crosby has for the future of the platform. That vision revolves to a great extent around 5G, the new wireless standard whose arrival on the market, he expects, will unleash new opportunities.

Want to learn more about what StackPath and other early movers in edge computing are up to? Most importantly, are you wondering whether enterprise IT/data center leaders should even care about any of this? We at DCK think they should, but don’t take our word for it. At Data Center World Global in March, StackPath founder and CEO Lance Crosby, Packet CTO Ihab Tarazi, and Vapor IO founder and CEO Cole Crawford will join us for a panel to take a deep dive into the earliest deployments of edge computing infrastructure and to talk about what it all means for the future of enterprise technology. Register here

The Internet’s Biggest Intersections

Crosby left IBM in 2015 (he’d been running the company’s enterprise cloud business since the SoftLayer acquisition) to build what he called a “secure platform at the edge.”

To him, the edge back then was in facilities that hosted internet exchange points, and that’s guided StackPath’s current site selection strategy. They’ve been going into buildings that have at least three carriers and/or 500 Gigabit per second of internet traffic being exchanged.

“There’s only about 120 of those in the world,” Crosby said. The current goal is to get into all 120 by sometime next year.

StackPath is already in all the top data center markets – places like San Jose, Ashburn, Amsterdam, Hong Kong, and Sydney – so the focus now is on secondary markets, such as Seattle, Atlanta, Madrid, and Milan.

The size of a single StackPath deployment varies from city to city. Its footprint in New York’s 111 8th Avenue and 60 Hudson Street, for example, is 20 racks each. The company goes into a tier-two market like Seattle with 10 racks, and into a tier-three city like Portland with two to four racks.

In Comes 5G

When the 5G opportunity became clear, StackPath changed its definition of edge. “5G came along, and we were like, OK, this is going to be the new edge,” Crosby said.

This new edge is at cell towers that will connect to end users via 5G, so StackPath is planning to follow its buildout at internet exchanges with expansion into the small data centers several vendors are starting to deploy at cell towers in preparation for the ultra-fast wireless standard’s arrival.

The startup is currently working with three of those edge data center specialists – Vapor IO, EdgeMicro, and BitBox (owned by Compass Datacenters, which also owns the edge data center builder EdgePoint) – as it drafts the plans for its next phase of expansion, Crosby said. Its first such deployment went live several months ago in an EdgeMicro test container in Denver.

Geographically, StackPath’s “new edge” buildout will follow big telecommunications companies that are using the same edge data center vendors for their 5G infrastructure builds. In Texas, for example, the initial locations the telcos are going into are Houston, Austin, and San Antonio, Crosby said.

One thing those three cities have in common is lack of a major internet exchange. That’s why the telcos are going there first, he explained. Today, most internet traffic reaches those cities via Dallas, home to Equinix’s Infomart Dallas, one of the most network-dense buildings in the US. Placing compute infrastructure in those cities means the carriers won’t have to backhaul all the traffic associated with 5G applications back to Dallas.

Crosby predicts that the carriers will stick to this strategy elsewhere: going to tier-two and tier-three cities that depend on big remote interconnection hubs before they get back around to the big cities. And companies that want to build internet services at the edge – companies like StackPath – will go where they go, Crosby said. “People like me, we’ll chase the towers.”

Ultra-Dense Hardware

Going inside those colocation data centers and eventually more edge data centers at cell towers is custom hardware StackPath designs on its own. One design goal is to pack as much compute, memory, storage, and network as possible while staying below 10kW per rack.

10kW is “just the physical limit,” Crosby said. “When you break that, then you have to start doing tricks.” By “tricks,” he means things like hot-aisle containment or more expensive options, such as the various liquid-cooling technologies. “That all costs big money. We want to stay under 10kW, so I can go into any data center in the world if I need to and be a fungible footprint.”

Currently, a standard StackPath server has 72 processor cores, a terabyte of RAM, 20 2TB SSDs, and four 25G NICs for a total of 100G per server. Everything in StackPath’s platform is 100G or higher. Each of the NICs has processing power of its own, sporting its own Arm processor and RAM. “It’s a server inside of a server almost,” Crosby said.

The company makes its own rack switches, using merchant silicon and its own network operating system, but uses Arista’s 7500 series switches for its Clos fabric. Once Arista releases 400G switches – which Crosby expects next year – he plans to go to four 50G NICs per server.

StackPath also has servers with GPUs and FPGAs. One use case for 5G and edge infrastructure services Crosby expects to be big is renting the infrastructure to security vendors. Those security firms, he said, can use GPUs or FPGAs, depending on their applications, to offload encryption, so they can free up CPU resources for other things.

Security vendors are going to build global software-defined WANs for clients that will connect their data centers, their cloud providers, office buildings, and mobile workers. An ultra-fast wireless connection means a mobile worker can connect straight to the servers at the base of the cell tower nearest to them and have the client’s policy decide at the tower whether to push the traffic to a data center, an intranet, the public internet, or elsewhere – all fully encrypted. Essentially, it’s corporate VPN minus the terrible user experience.

Global Infrastructure at $50 a Server

Crosby expects media, gaming, and security to be the biggest sectors that will use edge computing services like StackPath’s. Media and gaming companies want the best possible user experience for their customers without the astronomical costs of delivering that experience over long distances; there’s the $100 billion enterprise security market that’s transitioning to the cloud. There are also hundreds of software startups building DNS services, SD-WANs, VPNs, load balancers, firewalls, application accelerators, and so on.

“And they all have terrific software, but they don’t have a way to get to a global platform, so it’s hard for them to compete against the big guys,” Crosby said. “And now you can come to our website, and you can buy 50 VMs for 50 bucks each,” getting a global platform for $2,500.