Three Internet Things That Could, But Didn’t Go Wrong in the Pandemic

In March, when much of the western world stayed home instead of going to work or school, Avi Freedman, a self-described former “internet plumber,” worried about three things that could bring the internet to its knees and make the pandemic even more disruptive than it had already been.

For starters, while he knew that network architects tend to overbuild to absorb unanticipated spikes in demand, he wasn’t sure how much they had overbuilt, and whether it would be enough to handle the massive change in global traffic patterns. It was.

“I thought it might only be 10 percent; it turned out it was more like 25 to 40 percent,” he told DCK in a recent interview for the Data Center Podcast (scroll down to stream, download, or read the transcript).

Now CEO of Kentik, a startup he founded to give people who build and manage networks modern tools for the job, Freedman has watched the internet grow up from the front row over a span of several decades. He was one of the key architects who built the infrastructure of Akamai, one of the world’s largest CDN providers; ran technology strategy of the bare-metal cloud provider ServerCentral as CTO; built, ran, and later sold a global Usenet provider called Readnews; and created Philadelphia’s first ISP in his basement.

The internet handled the pandemic just fine, and while it didn’t go completely without hiccups, there was nothing that approached the kind of disruptions that were the norm in the nineties, Freedman likes to say.

With extra capacity apparently aplenty in the world’s networks, his second concern at the start of the crisis was whether supply chains for infrastructure components could handle its global and ongoing nature. Typical disaster recovery modeling exercises anticipate issues that are temporary and limited to specific geographic areas.

Freedman wasn’t sure most operators’ DR scenarios prepared them for “this patchwork up-down, up-down,” he said. “With these interconnected supply chains, will you get your optics? Will you get your line cards? Will you get your routers and switches?”

As DCK has reported, supply chains, which worried many across the industry, held up. Delayed shipments did become commonplace for some time, but most companies appear to have adapted their planning to the longer equipment lead times.

“Supply chains have definitely gone up this year but not as much as feared,” Freedman said.

Finally, he worried that cybercriminals would take the opportunity to attack network infrastructure more vigorously, since the shift to remote work made corporate networks more vulnerable.

“I think the reason we haven’t seen this is because the criminals make money over the infrastructure,” he said. “You could take the infrastructure down while it’s being more vulnerable with all this activity, but I just think the incentive has not been there. As the whole world moves digital, so do the criminals.”

Listen to the new episode of the Data Center Podcast for a lot more of our conversation with Freedman, where we talk about the early days of Akamai, the shift to hyperscale-like infrastructure by traditional enterprises, “edgification of everything,” the rise of the “internet bypass,” and the sorts of business lessons one learns from playing poker.

·

Podcast episode transript:

Yevgeniy Sverdlik:
Hey, everybody. Welcome to the Data Center podcast, after a year long hiatus, we're coming back. Coming back strong with Avi Freedman, he's the founder and CEO of Kentik, which is a San Francisco SaaS startup that provides network monitoring tools. Hi, Avi. Thank you so much for taking the time.

Avi Freedman:
Hi, Yevgeniy, thank you very much. I read everything on DCK and I'm thrilled to be a guest and chat about what's going on in the data center and networking world.

Yevgeniy Sverdlik:
Thank you for reading it. So Avi, you're a self described former internet plumber. You also do a lot of podcasts, you enjoy this medium. Do you consume a lot of them?

Avi Freedman:
You know, I consume transcripts. I read much faster than I listen but there are some podcasts that I listen to also. I'm just not spending a lot of time driving and it's been a busy year, surprisingly busier than even last year, so don't listen to as many as I do read.

Yevgeniy Sverdlik:
Avi can be frequently seen on Poker Night in America on CBS Sports, you did before the pandemic. He's wearing blue Kentik polo shirt, gray Kentik baseball hat. So Avi, is poker your plan B if the whole startup thing doesn't work out?

Avi Freedman:
Poker is an interesting distraction. I find it's useful to put your brain in another mode and be learning and thinking and observing. There's also some good business lesson in poker and yeah, I was in the World Series of Poker wearing [COMPT 00:01:51] hat, so people were like [COMPT 00:01:53]? What is that thing? So, I try to represent networking and of course, now Kentik when I go out in the world. I haven't made any interesting customer connections at the poker table but definitely some good diversion and you can make some money too as well.

Yevgeniy Sverdlik:
What sort of things have you learned about business from poker?

Avi Freedman:
Well, it's interesting because in business you never really have all the information that you need to sort of maximize but you can study and keep repeating and learn. And in poker table, it's very similar. You're operating with incomplete information and also sometimes, as you saw, the hand that was on Twitter, sometimes you just get very lucky and you don't want to confuse that with repetitive success. So the two concepts of expected value, which is, is thing generally going to be successful? And then variance, which is, is short term is luck really playing? There's a lot of parallels and you also have to be really self aware. Earlier in our careers, we just do things because we're interested in them. And when you're running a business, you need to be a little bit more thoughtful and say, "Is this good result the result of good planning and strategy and execution or did we just get lucky," which is absolutely allowed and in poker of course, encouraged just as it is in business.

Yevgeniy Sverdlik:
So don't mistake luck for...

Avi Freedman:
Exactly. Exactly.

Yevgeniy Sverdlik:
... for just you being great.

Avi Freedman:
Yeah, and I've bootstrapped companies before where it's my company, I get to do what I want and if I want to go write code or build product, you could do that. But in larger business, when I was at above at net Akamai or at Kentik, you sign up from the beginning to say, anytime we can think of ways to do one or 2% better, we want to be thinking and challenging ourselves to do that. And poker is very much like that. If you play the same game for 20 years, you're going to be stale. So it's helpful sometimes to go into another medium and see the same lessons and try to bring them back to the core thing that I do.

Yevgeniy Sverdlik:
One interesting thing from your biography is you started the first ISP in Philadelphia, which is you were born and raised there, right? In Philly.

Avi Freedman:
Yeah.

Yevgeniy Sverdlik:
Tell us about that first ISP. It was in your own apartment?

Avi Freedman:
Yeah, it was in the basement. It was 215664 Unix, was the dial in number. I always wanted to run a multiline VBS. I got into computers in the... I was really fortunate. I was eight at the time in 1978. So it was a little bit early for eight year olds to be getting in but I'm very fortunate. My father had a wizard working with him, who had worked at Bell Labs and got me into Unix in C. I wanted to do multiline VBS couldn't but I administered lots of suns and did multi-user dungeon games and online Unix machines at Temple University. And when I was getting ready to leave school, there was no way to buy dial up access. And so I was like, "What is this lease line thing? What is this internet..." I mean, I knew what the internet was but I had to... How do you actually... What is it, how do you get connected?

Avi Freedman:
So I decided to start the first internet provider. For awhile, if I didn't want to sell to you, you couldn't get internet access. You had to be a member of a school in Philadelphia. And I thought, how hard could it be? I'll just pick up some modems. It was a lot harder than that. I had worked in some businesses and thought I knew what was going on but it was a good education. And that was how I fell into internet working.

Yevgeniy Sverdlik:
What did that set up look like? What sort of equipment did you have then?

Avi Freedman:
At first, I was like, "Oh, I buy and sell sun workstations. Let me put some serial port cards in." Sonos and serial port cards were not as reliable as one needed. So the setup was... Actually, I'll send you a picture of it. Or if you go to my homepage, there's a gentleman standing in front of the basement where it grew into. So you can see basically Sonos sun blade servers, like sparks on their side and stuff like that. So originally, it was just some modems sun for 110 with a card. And then eventually it became a class fare, all running hot on a 10 megabit hub until that broke. NFS mounted shell accounts, had a BBS that looked like... that was a Unix program that looked like a BBS.

Avi Freedman:
So, when we started, it was actually gofer email used net IRC, because WWW was not really a very usable. And then in '94, of course, mosaic, trumpet wind sock and all that. But in '92, the internet was shell not... Who thought everyone would want IP addresses at their home only, that was a company needs IP addresses [inaudible 00:07:11]. So the world has come a long way since then. Sometimes I still... I haven't been to the data centers since February. But I remember last year, we have a CEO cabinet and I actually have a catalyst, a [inaudible 00:07:28] seven 23 VXL catalyst. For some reason, I still had a 48 by a hundred megabit card in there for out-of-band.

Avi Freedman:
And I looked at that, I'm like, "Oh, that's a T1 one card, that's not useful." And I was like, "Wait, no, actually it's a lot faster than T1s." But that's how old I am. I was like, "Oh, it's the slow, useless card." And now 10 gig has become this little useless thing. So it's a pretty fun.

Yevgeniy Sverdlik:
Avi was a key architect behind Akamai's network. And when I spoke to Jonathan Seelig, one of our commodities founders, he said, "Once I hired Avi and started working with him, I realized I'm not needed anymore." Or like, "This guy is so much better at what he does. He needs to do my job basically." So tell us about that. What were you tasked with doing at Akamai? What was your role?

Avi Freedman:
So I was running above net. I had sold my ISP. I was running above net. And I was really frustrated with the state of networking in the late 90's, the fiber SOC, the fiber wasn't where people said it was. You'd have outages and the routers vendors [inaudible 00:08:41], just the internet. We didn't have enough capacity for what was happening. And I have a computer science background. And so, I went and I saw a talk at [inaudible 00:08:54] from Peter [Danzig 00:08:55]. And they talked about... I call it the magic packet transporters. So that's what I call the CDNs, because to network people, these packets show up, they don't listen to BVP, it's all this overlay stuff. And Peter [Danzing 00:09:09] was talking about how Akamai could map, which is how they figure out where to send a user, what server and network to send a user to, intelligently.

Avi Freedman:
And I thought, "Well, that's cool." And actually, even look at a random mapping of a little bit of traffic and CICD they're mapping code to say, "Are we getting better or worse than random?" And I was like, "Wow, I would love to be... That's really cool." If there's a problem with a router here, you can avoid it. It's sort of routing up in software. I thought that was really cool. So I reached out, I asked him afterwards and he said, "Oh, no, no, no, we're about to go public." And I said, "No, I'm really interested." So I met them, joined. I was going to be leading a group called Network Architecture in the engineering group, because I thought, "Well, let's look at BGP churn because that's a predictor of performance. Let's do more science-y things."

Avi Freedman:
And then, yeah, after I started, Jonathan... By the way, Jonathan set up this group and in six months got to a thousand networks. And the structure of that group is almost the same as it is today, 22 years later, where another Freedman, my brother has my old job running that group. So Noam, my brother, runs that group now. It's I think five or 600 people. So Jonathan is very humble but the structure of go get contracts, think about deployments, think about architecture, offer benefit to providers, think about strategy, that's all still there. But they were paying more than market. I'll just say so. And there was things that a network person would point out that, which is like, "Hey, some provider in Latin America wants to charge us $10,000 per megabit per second but we're deployed in these universities and these universities are tend to one inbound to outbound, let's buy their unused outbound bandwidth." Right.

Avi Freedman:
It's a network architecture evolution was what that group contributed and some of the business side. And then Jonathan, even Tim Weller, who's now CEO of Datto. He was the CFO. We all partnered together. So it's not like Jonathan stopped working on it but some of the more network-y stuff we took over and network architecture moved out of engineering and into this combo group. But the beautiful thing, because Akamai was at the time, in some sense, the world's largest non-network, is that network people don't get paged. If something goes down, it's a software problem. The design of Akamai was that the network people could get to do networking things but not get paged. So I thought, "Wow, that's pretty cool."

Yevgeniy Sverdlik:
So let's fast forward to Kentik. Let's start with the basics. What's your quick elevator pitch for the startup?

Avi Freedman:
Sure. Yeah, we're the network observability platform that the people who build the internet and the people that use it, use to see plan, run and fix their infrastructure. So we take live streaming, telemetry and data traffic routing from all the different kinds of network elements and all the different kinds of networks, the ones that you build and things like cloud and SaaS and things that you don't own. And then it's a SaaS platform, we do run it on-prem sometimes for people, if that's how they need to have it in their network but it's a managed big data platform. And then there's all sorts of actions people take. So we have a UI, we have an API, we just announced a partnership with New Relic today. We have a fire hose, so we can send things into data lakes and other observability tools.

Avi Freedman:
And then we linked to automation. Today that's largely DDoSs, load balancers, CDNs, things like that. Like, what kind of action do people want to take around it? And so, we sell to networking professionals generally in the service provider world sometime or the cloud world that could actually be a product person making an analytics portal or an SD-WAN or sassy analytics portal or DDoSs service. And sometimes we actually have sales people that log into Kentik to say, "Hey, I'm not getting paid for this traffic that goes to that pier. Maybe I should sell to their customers." And then network engineers, architects operations. So we're six years old. We have few hundred customers, probably seven of the 10 biggest data center providers, half of the biggest cloud providers, five of seven or eight of the biggest CDN providers.

Avi Freedman:
And then a whole bunch of traditional enterprise. So you think about it in an aquaponics type ecosystem or QTS or data founder, anyone that you want to list on the data center list. The people that are there to sell services, they're our customers, the data centers themselves and then the enterprises that are there because of all the connectivity and services we sell to them to.

Yevgeniy Sverdlik:
How did the idea behind it come about it? It wasn't Kentik at the start, right? CloudHelix.

Avi Freedman:
Yeah. We called it CloudHelix. Yeah. And the original idea was, so I left Akamai. I spent eight years failing as a product marketer, trying to convince them to create or get into cloud infrastructure.

Yevgeniy Sverdlik:
To convince Akamai to get into cloud infrastructure, which they're doing now. Sort of.

Avi Freedman:
Sort of. Well, what's even more embarrassing was what we were calling it... what Jonathan Seelig and I were calling it, it was native edge computing, which is a thing now. Is not actually-

Yevgeniy Sverdlik:
Wait, when was this?

Avi Freedman:
This was, I mean, 2000. I mean, 2001, 2002. We did in the network group, we actually were... Basically, when Danny Lewin was alive, it was like an Israeli company. So you couldn't get fired by taking initiative. So I was just like, "You know what? FedEx, mercury interactive, you want some servers? Awesome. I have some servers. You want to order it, we'll give you a global network." So we were doing that kind of thing. We called it platform sharing program. And we said, "Let's do this." VMware was expensive. I said, "Let's use user mode Linux." I mean, let's do this kind of stuff. But at the time, I was less sophisticated at explaining some of the business outcome. And it seemed obvious to me but it was going to be very CapEx intensive. Remember, I'm this ISP guy that comes in and it's like, "Oh, I'll be saying we should do ISP."

Avi Freedman:
But I could have done a better job at explaining and corralling. And it is lower margin than where Akamai came from. But if you look at what [inaudible 00:15:52], all these services, each one of them is a super version of something like an Amazon service. It's just not the direction that Akamai wanted to go. But they are a player in obviously content delivery, in edge computing, in security, in connecting people. So, obviously it's still a major player there. So I left Akamai and was doing a few things. I had a Usenet company, if people know what that is, that's collaboration infrastructure, very big data nowadays.

Avi Freedman:
I was CTO of a cloud company called ServerCentral, where we did some very high performance bare metal cloud. But I also started building network sensors for some federal folks that I had worked with when I was at Akamai. And everyone that bought these sensors said, "What do we do with all this traffic? I mean, it's 10 gig, 40 gig, a hundred gig. I can't put it in Hadoop, Arbor and all these NetScout, these appliances. I can't [inaudible 00:16:45].

Yevgeniy Sverdlik:
Sorry. Traffic from the data censors?

Avi Freedman:
Yeah, traffic from the censors.

Yevgeniy Sverdlik:
Data coming from censors.

Avi Freedman:
Network traffic packets, wire data. And they said, "How do we actually run a modern network? Right? If it's an appliance, we can't see the cloud, we can't put packet copies everywhere. And even if we could, I mean, all this data from NetFlow, BGP routing, it's all too much. And I thought, "Well, that's a really interesting problem." So I was running a Usenet company and I had petabytes of servers. I was in Equinix. I had a hundred gigabits of traffic. I thought, "Well, I could see if I could solve this problem for myself." So I did that, showed it to people and said, "Hey, would you send me your traffic so I could..." I assumed people would want it in their network. And everyone's like, "No, no, no. I want a cloud thing. Yes. We want this. We're tired of Arbor. We want this, go make that."

Avi Freedman:
And then I had a decision to make, which is, do I bootstrap, right? I had a profitable, using a company going. Do I do it in that or wow, this looks like a real problem. The last was in the early two thousands in network analytics. And that's why I decided to move out to the Bay area, getting to the venture ecosystem. Kentik, which was originally called CloudHelix. I actually applied to YCombinator. You were talking about the show Silicon Valley. And I was living that. Like we had just raised our seed round when the first season ended and the guy that led our seed round was actually one of the people that was in the last episode. It was all very surreal. Yeah. And I just thought it was a big market, there all these operational things that are separate, all these silos of data analytics. All these telemetry silos, network silos. And it's time to bring it all together. That was the goal with Kentik and that's what we're doing now.

Yevgeniy Sverdlik:
And there are others... Who are your main competitors? There's ThousandEyes that Cisco recently bought. I think they do a similar thing. Talk about the market that you deal with.

Avi Freedman:
If you look at our historic competitors, we've really been... When we started, there were two, it was DIY. So people trying to build their own. Or Arbor networks, which is owned by NetScout, looked at the traffic and understood also the internet side. It didn't know about cloud and it did roll ups only. So nowadays we talk about observability and it basically threw away all the data and gave you some canned reports. And so, that was what we started replacing around the internet edge side. Then we started getting seeing ThousandEyes but a little bit more complimentary because we didn't do network testing when we started and ThousandEyes doesn't know what your network actually is or what it's doing. It looks at tests. So it's trace route, it's paying its HTP tests but when you get an alert from a pure... they call it DEM, digital experience monitoring system.

Avi Freedman:
You don't know whether there's actually any traffic on that path. So you need to put it together and that's what we're doing now. And along the way we picked up Riverbed, NetScout, some of the appliance-based guys and girls, the vendors as competitors. And now we're actually seeing partnership and in some sense competition with some of the observability players who they come down to inside the computer. But if you look at them, they don't go beyond the ethernet. So coming from the network infrastructure world, our customers are saying, "Well, that's nice." I do need to see inside the Kubernetes cluster but I also need to see the switches and routers and SD-WAN and internet and all those things."

Yevgeniy Sverdlik:
I see.

Avi Freedman:
So, yeah. But we started off and we saw ThousandEyes a lot because they were internet monitoring. We were doing a lot of internet monitoring. As we've broadened, yeah, we still see them... It was awesome they were bought by Cisco because now they're one of 17 monitoring products that Cisco has, none of which worked together. So they've got StealthWatch, StealthWatch cloud, Tetration, they resell Harbor, they've got AppD, DNA center... I'm forgetting and thousands. I think they got a bunch... Prime, don't forget Prime. And they get a lot of them. So, yeah.

Yevgeniy Sverdlik:
Have you seen a lift in business from the pandemic? People are worried about the network health more.

Avi Freedman:
The network people are really realizing it's critical and a lot of things have really surprised me about how this year went and businesses have been doing great. We've moved from all of a sudden we can't see people but we had a big enough brand and a big enough customers and people are still changing jobs and saying, "Okay, I need Kentik." That business has been going well. What's really interesting is some of the people that I thought in the beginning of the year, things would have even accelerated with have been great but I still see some web companies that are still saying, "Ooh, we don't know what's going on, although." A lot of people are really firming up their 2021 plans and saying, "Yeah, things actually look pretty good."

Avi Freedman:
But what's really surprising is all of our travel customers are renewing or expanding. So they're really bullish too, which makes me-

Yevgeniy Sverdlik:
Travel.

Avi Freedman:
... happy. Yeah. And we-

Yevgeniy Sverdlik:
Travel customers like airlines or...

Avi Freedman:
Yeah. Like airlines, airline booking companies, transportation companies. They're all... Some of them have adopted food delivery. Some of them are getting into longer term vacation stuff. They've all found a way to adapt their business. And for us, where we started, we came from service provider cloud. Some of the digital companies that were very... The people like CDNs that invented orchestration that everyone's trying to get towards. Kentik was built for that, right? Feed us a constant stream of all your DHCP, your DNS, your IPM, your [inaudible 00:22:48] SBSS. And we'll put it together with all your telemetry and then learn and show you what you need to do. We were built in that world and our first adopters were customers that couldn't use appliances, because like, well, I don't want to ask about an IP address. Before Kubernetes, I'm deploying applications, all I care about is, is this application and that user having a problem? And so, we were built to be able to do that. Now, we're seeing traditional enterprise, that's the thing which has really accelerated for us this year.

Avi Freedman:
Hedge funds, E-commerce, retail. We had all these as customers but a lot of them are now saying whether it's the sassy, the edge at the people's homes, if that doesn't work, our business doesn't go or just the internet retail, E-commerce... Their company's revenue is flowing over the internet. And people really realize that now, if the stores are aren't shut and your business keeps going, well, obviously that's all supported by the network. There's definitely some application people like, "Oh, it's all API. There is no network anymore." But what we're seeing is both the lines of business and the IT shops in traditional enterprise are looking for more modern solutions that are the equivalent of what they've been buying with Datadog and New Relic and Dynatrace and the observability side for the DevOps group. It's like why can't network people have nice things too? That's what Kentik is about.

Yevgeniy Sverdlik:
Right. So it's not like just dismissing all the complexity because it's been abstracted or they do want visibility into the infrastructure, the lower infrastructure. They just need it presented maybe in a nicer way.

Avi Freedman:
Someone needs to figure out underlay overlay. When something doesn't work and it's not the application, someone needs to figure it out who you're going to call? Network people.

Yevgeniy Sverdlik:
Yep. And your first customers were some big, big names, Zoom, Yelp, Dropbox, other customers, you have Uber, I believe, Verizon Media.

Avi Freedman:
Yeah. We have-

Yevgeniy Sverdlik:
One thing... Sorry, go ahead.

Avi Freedman:
No, we were honored really early with some critical players in digital and economic infrastructure becoming customers. And our first trial customer was Cogent, who sent us all their backbone data. So we cut our teeth on the largest scale and prove that we could do it. So that really helped. And obviously our networking world contacts helped gain people's trust.

Yevgeniy Sverdlik:
And you did an interview recently and you mentioned interesting thing that... So these customers that we just mentioned, they operate these scale networks, especially the Dropbox or Verizon and Zoom. You said now traditional enterprises they're realizing they need to operate like these international infrastructure companies. Can you explain that a little bit? Why do enterprises need to do that now?

Avi Freedman:
Well, again, it's something COVID has made people really realize, right? Servers are cattle, not pets. So our network elements, right? If you don't want to log into Cat three, because you think that it's got this weird issue but you go and reset the Ethan at every so often and it's happy. Like in a COVID world, that doesn't really work. It's harder to travel. But all the automation initiatives have really picked up. And the other thing we've seen is just an acceleration of the cloud. So I don't want to say everyone's turning off all their infrastructure but they're definitely more cloud dependent, more SaaS dependent. And so, the traditional enterprise are struggling with some of these traditional appliances or windows software that feels like playing Epic video games if you use them, which are the traditional network world that we've all used this stuff in the past and saying, "Okay, well, what are the people that were cloud native, that grew up in the digital where there was no synchronet, what are they doing? And let's think about that."

Avi Freedman:
But they also have initiatives, app modernization, cloud migration, observability, streaming data buses of telemetry. When they go out and look and say, "Okay, what do I need to do?" They need to have more modern solutions and they have that on the application side. And so, that's why we've seen a lot of these people and also word of mouth. You get to scale, you get one hedge fund, they talk to other people, you get five more, same thing with retail, same thing with a lot of the streaming companies, things like that. So we're going to be doing a lot more education around telemetry, data platforms, just generally network observability and we think that'll help a lot of people and maybe that'll help people get into the industry, which is a big problem for us, which is a whole separate conversation, the aging of network folks and lack of diversity.

Avi Freedman:
But we also think it'll attract people that are studying these things to us. And I think there's a lot of burnout from vendors saying, "We now have self-driving networks or closed loop automation." But that really is, if you look at the presentation, it's write it in Python instead of logging in CLI, we're not at the magic land. So we try to be in the middle of saying like, "Here's really where we are and here's what you can do next year." Right. Now you can automate your device and interface turnip next year, routing next, whatever. And so, that's another way we engage is trying to find practitioners and help show them what we're seeing across our customer base.

Yevgeniy Sverdlik:
And so, [inaudible 00:28:26] survived the pandemic, as we've talked about several times. As a former internet plumber, what were your biggest fears about connectivity when the crisis started? Did you have any or did you think it's fine, it's built to handle all this?

Avi Freedman:
Yeah. There were a few things I was concerned about. One was, how much people had really overbuilt. I thought it might only be 10%. It turned out it was more like 25 to 40%. Excuse me. So people overbuilt and that was good. They've overbuilt by more than I was concerned they might.

Yevgeniy Sverdlik:
They'd had more bandwidth in there [crosstalk 00:29:07].

Avi Freedman:
Yeah. They had more bandwidth available. Another thing that I was concerned about, which is a concern that people are watching is supply chain, because our DRM modeling, our scenario modeling assumed... I mean, from back from when I did it but our customers say the same thing. Assumes either something happens in one geo and then it's better or maybe not. Or something happens worldwide and that it's better but not this patchwork up down, up down. And with these interconnected supply chains, will you get your optics? Will you get your line card? Will you get your routers and switches. So supply chains have definitely gone up this year but not as much as [inaudible 00:29:54]. But a lot of our customers are now using more Kentik for capacity planning to know further and advance order further and advance.

Avi Freedman:
And the third thing was, I wasn't really worried about large scale sustained outages but it would have been a good opportunity for people to up the infrastructure attacks. Now, I think the reason we haven't seen this is because the criminals make money over the infrastructure so that you could take the infrastructure down while it's being more vulnerable with all this activity. But I just think the incentives has not been there as the whole world moves digital so do the criminals. And from our perspective, all of our performance data, what I've said is there's been some vendors that have made a lot of noise, ThousandEyes is one of them, about, "Oh, there's all these outages with COVID."

Avi Freedman:
The way I explain it is, if you just take the internet growth and speed it up by 30%, you got 30% more micro outages. That's what happens. Sometimes you plug in a thing and something happens, there's little things but really the internet has worked marvelously well through this. And I've been very happy as an internet plumber, as a former plumber-

Yevgeniy Sverdlik:
A lot of the outages that ThousandEyes were tracking and they showed these spikes, these were minor outages deep in the network that didn't really materialize for the users.

Avi Freedman:
Yeah. I would say they're localized. I mean, again, our customers at Zoom, we have other video conferencing customers. We have five gaming customers. We have 10 ad tech customers. When I look across them, were there problems in some providers, some places? Was there a century link, a big problem? Yes. Every year, there's two or three big problems. But when I look at the micro outages that happens when you turn up capacity, all that, that's the kind of thing that we're seeing, localized to a region, a network, a network region. And overall, the Internet's been performing pretty well. We actually we'll be launching next month... We've started doing continuous monitoring, right? Because your traffic on your network is continuous social your measurement.

Avi Freedman:
So we're doing every second monitoring. It's because we have a lot of service providers, they use this thing called TWAMP, it's actually spelled T-W-A-M-P but people don't want to pronounce it the way you would. So they say TWAMP, Two-Way Active Measurement Protocol. And service providers are used to doing one to 10 or 20 packets a second. And we've brought that into the enterprise world to say, "Let's just do continuous testing." And you know what? You do see a lot of micro outages and sometimes it's worth routing around them. But again, when you look in the aggregate, things are working pretty well, generally.

Yevgeniy Sverdlik:
So let's switch gears here a little bit. When your PR person reached out to me recently, they pitched a conversation with Avi Freedman about his three big trends to watch. And I do want to talk about them because they are pretty interesting. And some of them are a bit confusing to me, at least the explanation. So hopefully you can clarify.

Avi Freedman:
Sure.

Yevgeniy Sverdlik:
So the first one is edification of everything thanks to 5G. Now, where I'm a little bit confused here is the connection between 5G and edge computing-

Avi Freedman:
I don't know how the... I think there might've been..

Yevgeniy Sverdlik:
There's been...

Avi Freedman:
Oh.

Yevgeniy Sverdlik:
Sorry.

Avi Freedman:
We're doing CSMA/CD, Carrier Sense Multiple Access with Collision Detection now. I don't know this 5G [crosstalk 00:33:40].

Yevgeniy Sverdlik:
There's been investment in edge computing, you cited investments in edge compute infrastructure by enterprises and telcos. Omdia analysts, what they've seen is edge compute investment is happening but it's happening with or without 5G. 5G is just the transmission technology at the end and telcos are investing primarily for caching and to virtualize their networks into investing into the edge compute. Infrastructure enterprise they're investing in industrial IOT but also it's like smart retail, et cetera. So what are you seeing? How do you connect 5G and edge compute and why is this a big trend in your opinion for 2021?

Avi Freedman:
So I think edification is a big trend going on right now. And I think 5G is a part of that. So, I think maybe we might have elevated 5G a little bit in the translation of what I see a lot of activity on, on both the enterprise and service provider side for next year. As I said earlier, I think people are realizing that the biggest distributed edge is actually well, your house, my house, it's where everybody actually is. Now, when COVID is over, thankfully hopefully soon, 5G will be a part of that but I would just say wireless. What is unique about 5G is the 5G infrastructure itself is being built and deployed cloud native, which makes it even easier to justify putting the kind of architecture, the edge compute and delivery architecture is that the people can run the applications.

Avi Freedman:
Now, I would say 2020 was a lot of prep for that. There's a lot of, how do we get the right open ran architecture? How do we do what, whether it's geo or Rakuten, when you're looking at some of the things that people are doing that are next gen and how do we prep to have footprint, whether it's towers or other kinds of data centers. But the applications that we track across our customers that are actually delivered to edge, I mean, it's not 1%. Now, you could argue CDNs are edge, that's 90%. If you think about magic packet transporters but I'm talking about delivered right next to the cell or to the user. That's still really early but I think carriers committing to a cloud native architecture, enterprises being all about cloud transformation, is going to make some of the edge promise a little bit more real.

Avi Freedman:
So maybe I wouldn't describe it as edge in 5G in 2021 but 5G is definitely a contributor and I think we're going to start to see not quite at the hype level. So we'll still be behind the great promise but I think we are going to start to see some of these cloud native functions go. It won't be exactly like VNS and service chaining and all that but some things that are deployable. Now, we still, as an industry and application have a lot of education to do about how you actually take advantage of the network, how you build these applications. A lot of these applications really require a state. So you need a micro cloud architecture. It's not just a proxy that you need. And you can see that innovation on the CDN side. So, what I would say is I think we're going to go from almost all hype to first instantiation in 2021. And that'll be really interesting to see which areas race ahead of which.

Yevgeniy Sverdlik:
I see. And the second trend was the rise of the internet bypass. And you mentioned in the brief description, it's been around, it's not a new concept. The idea of serving content locally instead of over long distance over the internet. Media companies have been investing in local caching, digital media companies, Netflix, et cetera.

Avi Freedman:
Yeah.

Yevgeniy Sverdlik:
CDNs have launched edge compute platforms, which you brought up. They seem to be primarily pushing serverless for developers. It seems like something they see as promising, where they can offer some value that maybe AWS can not. So explain this new generation internet bypass, if you will, from your perspective.

Avi Freedman:
Sure. So what's really interesting when we look at our service provider customers who a lot of what they're interested in is OTT, right? Because as I described Akamai and the other CDNs are magic packet transporters. If you're used to networking and you say, "Oh, I advertise a route. And then based on that, people send me traffic. And if I don't want the traffic there, I move it out there and the traffic comes that way." Well, that's not really how CDNs work. They use the principle that you can control your outbound arbitrarily. And what we've seen for a decade now is, when I look at my service provider customers for most of the eyeball networks, less than 10% of their traffic or I would say more than 90% of their traffic comes from overlay networks, right? Non single AAS based content network.

Avi Freedman:
So everyone knows Netflix but there's a whole bunch of others. And it used to be, you had to go hire a whole bunch of people like Akamai I did. And then the other CDNs to do this kind of edge architecture but Netflix, Google, Facebook, were three of the first that did the same thing. In fact, Netflix took a really elegant solution because they put some of the CDN mapping in the player, which simplified actually getting going and building to pretty big scale. And now, we are seeing a growth in that, right? It's interesting to think, is it possible that less than 5% of the... if the traffic to a service rider like Comcast might come from traditional from the cloud or from web service sitting somewhere or from an end video conferencing but we might actually get there.

Avi Freedman:
And so, the reason I call that internet bypass is it's the backbone bypass. You're basically trying to serve around the bottlenecks, which are generally between networks, because between networks is where politics and cost comes in. It's generally cheaper inside network. So there's much more capacity. One interesting thing since I collaborated with the PR agency and we talked about what might be happening next year is, did you see the Disney announcement with Quilt? And is it called the Open Content Alliance?

Yevgeniy Sverdlik:
No. I haven't seen it.

Avi Freedman:
But... So Disney is a big player, obviously. They've got a lot of traffic and they've partnered in a lot of areas of the world with Quilt, which has done cashing but in an open way. So, whereas Netflix, Akamai, Fastly, Facebook, when you get the caches from these servers and they could be active, they could be running applications. It only serves that one provider. So there's been multiple attempts before to have boxes that can be deployed deeper in network, that could serve multiple kinds of content but respect the policy that the service provider wants. And so, there's some innovation there too, which I'll be interested to see how it goes but it seems to be helping Disney and some service providers, both. So, it'll be really interesting to see how that goes.

Avi Freedman:
But this trend towards... People think of the internet as all these providers connecting to each other and that's how all the traffic flows, that's actually by traffic volume not actually been what actually happens from the service provider viewpoint. And I think that's going to accelerate and there are some new models that are coming into play that are consistent with this push to the edge. So I think it'll be exciting to see what happens with that next year.

Yevgeniy Sverdlik:
I'm curious about your thoughts on this trend of CDNs, getting into serverless computing. The way I see it, they have the... or at least perceive this as a natural advantage where they have already invested in this infrastructure that's close to end users close to as many end users as possible, the eyeballs. And so, the hyperscalers haven't built infrastructure quiet that distributed. And so, what the CDNs are seeing as an opportunity here is to offer these serverless services, like edge cloud but only serverless to the developers. So what are your thoughts on that? Is that a promising area?

Avi Freedman:
Yeah. No, look, I definitely think it is. I was an advocate for it 20 years ago before even serverless. At first, before I answer, I've actually really amused last week, I think was AWS. The year before when Reinvent was in-person. And they announced outposts, I took a bunch of pictures of myself standing with my hand on an outpost. And then I bubbled serverless pointing to the rack of servers, which you could deploy to run serverless. I know that [inaudible 00:43:42] functions...

Yevgeniy Sverdlik:
It's magic.

Avi Freedman:
Yeah, exactly. Ignore the service here. There's no service, it's all serverless. It still cracks me up but... Yeah, I think that it's promising but early days. I'll give you an example. We went to CloudFlare and Fastly and said, "Awesome, you have all this deployment. We're deploying synthetic. We have hundreds of agents we're going to be growing to really..." We actually call our project internally regionally called it project trillion eyes, because there's more than a thousand networks. You clearly needed to be in more than a thousand places and thousands of very small. We have trillions of data points per day to come into Kentik. We want the maximum edge ultimately down in the individual edge, which, in the COVID world we've seen is really important.

Avi Freedman:
But for example, today, I couldn't with any of the edge services today, actually say, run it there. So you're still subject to the CDN architecture of try to run it in some general place. But for Kentik, we need to know it was this network. We need to be able to say, "We need to measure from this network and this geography." So it's very early. It is edge. I think what'll be interesting is, how deep do they go into storage? Because this is something we saw at Akamai. If you can't store a session state, then there's a limit to how much you can do. Now, some of them like CloudFlare might apply their overlay technology to persistent reach back into the cloud services.

Avi Freedman:
But I still haven't seen from any of the serverless side CDN marriage. What is that architecture of? Are they going to be building all the storage stuff? I know CloudFlare just announced persistent objects of some sort. But there's a lot still to be done. And yeah, you can generate shipping labels. There's some things you can do with stateless edge computing and distribution lower latency. There's a lot of areas that that has advantages but ultimately developers still need a lot more principles... a lot more foundational components that are there now. So it'll be again, interesting to see how that goes. Actually, you mentioned Jonathan Seelig, he's actually doing a distributed native edge computing, which to me is actually much more interesting because then I can run whatever I want.

Avi Freedman:
It's containers in that case but with persistent storage to company called Ridge. And he's doing it by partnering with data centers to use their compute resources and then build a Federation. So it's like what Quilt is doing on the caching side but on the compute side. And I wonder whether we'll see some of the CDNs do that. Limelight actually used to do that. I used to buy transit from them. They used to sell bare metal. IBM is one of our large customers... That was one thing SoftLayer did. So I think the CDN bare metal serverless, I expect to either see the CDNs add more cloud services or add a distributed persistent container or bare metal to support a wider range of these applications that people would have run.

Yevgeniy Sverdlik:
I see. So flesh it out beyond the basic.

Avi Freedman:
Yeah. It needs to go in one direction that

Yevgeniy Sverdlik:
Simple functions that they have right now.

Avi Freedman:
Yeah. You do need... Ultimately, serverless is cool but forward deploying serverless, there's a limit to what you can do without some sort of storage data, queuing all the other things that the centralized cloud. It's funny to call AWS centralized with dozens of locations but they've less distributed clouds do.

Yevgeniy Sverdlik:
Yeah. And then the final trend was the convergence of NetApps, DevOps and SecOps. I don't know enough in this area to introduce this part any further. So what do you mean here by convergence between the three?

Avi Freedman:
There's three distinct cultures that we see in and around Kentik and who we work with. And in the 90's, when I grew up, there was just nerd, right? You were a nerd, you played with systems, you wrote some software, you were interested in the network and you were an admin. You were a hacker, you were whatever. But from the 90's, there was this specialization. So I'm a security person, I'm a storage person, I'm a network person, I'm an application operations person, I'm a developer. And you've got at least three cultures which we interact with. You've got the core networking and there are subcultures there but network ops, right? And that could be planning, monitoring, fixing, automating but you take a network infrastructure view of the world and then, oh, [inaudible 00:48:40] excellent. It's just...

Avi Freedman:
You either think of it as a tunnel or as an extra tag, whichever way you think of it, you have a model. Like cloud, oh, it's just other people's tumbles and stuff. I mean, and you can put it together. So that's the way you look at the world. You've got the application operations view of the world. So the system in that went up into the application. It's like, well, there's APIs for the network. If it's not my problem, it's their problem but I don't really need to see that, historically. Or they feel like they're not empowered. They don't really understand it. And then you've got security groups, who also operate but have a different mentality, a different language. And in a cloud native world, where the network person needs to look at the server because it's a virtual IMEX or it's a networking plane on a containerized system. The application operator needs to look because it might be their company's infrastructure, cloud infrastructure or an internet problem.

Avi Freedman:
And the security people, like, what are these crazy youths doing turning stuff up and down? How could we possibly, whatever. They don't know what your Coobernetti's and your other systems are doing, they might think it's an attack and shut it down. And then the operations people don't know that they just did that. So in theory, you would think the world would be ready. The same tools get used for all three of those things. And that's not been the way that it's worked because of different language, different cultures, different things that people look for. So I think that in 2021, we're going to see an acceleration but not the Nirvana of these three groups coming together more.

Avi Freedman:
We're seeing it with the operations side. We're seeing that... As I said, we just announced a partnership with New Relic today, where we can stream the network data into their telemetry data platform and we're building an app for it. So that application owners can see, where is this application sitting and who should I talk to if there's an infrastructure problem? It's something we focus a lot of product energy on is, how do you make it usable for people that don't know what networks are to understand network problems. And we're seeing security people also.

Avi Freedman:
Now, there's a lot of people that say, "Oh, network observability, network monitoring, it's not a security thing." But we're seeing that as infrastructure is getting more dynamic and applications are a lot of security, people are saying, "Oh, we really do need to know that or we're going to waste all this time separately." So I would say three years ago probably had 5% of our customers had, I'll call it DevOps observability folks using Kentik. Now, it's probably 25%. And I think it's going to go higher next year, especially we've built libraries for not just API but libraries about how to pull data in and out. On the security side, it was probably started higher because of our DDoSs work.

Avi Freedman:
But that's also growing and still less than 50%. But I think it's going to increase next year. And I also think education is something that all three groups need to work on, because we're not really going to be happy until all these groups speak same language and understand each other's problems better than we do today. But it's trending in the positive direction, which I view as very promising.

Yevgeniy Sverdlik:
It's trending in the positive direction for the industry and for Kentik.

Avi Freedman:
Yes, for both. We're very excited. We're doing a lot of hiring. If you're looking for a job, please go to careers page. If you are a pure network architect, we don't really run big massive infrastructure, but we have hundreds and hundreds of customers and I'll be happy to point you into our customer base if you're just generally in the networking world. And if you're early on, we really need to encourage early folks. And we'd love to hear from people passionate about networking and wanting to learn. We're going to try to do more education around that too. But yeah, for the industry too, I think there's been a lot of visibility of the importance of networking and networkers. And there's going to be continued to move to really complex hybrid infrastructure and everyone wanting to be faster, which is going to be exciting for the industry.

Avi Freedman:
In many ways, it's like being an ISP in the 90's, you just wake up and there's this new idea and now everyone needs to react. I love it. It's exciting.

Yevgeniy Sverdlik:
Avi, thank you so much for a great conversation. Thank you for your time.

Avi Freedman:
Oh, absolutely. Thank you. I always enjoy it. I look forward to next time.