Sandy's Impact: The View from the Network

renesys-sandy_US_outages

Stepping a little further back than the data center, now it’s time for a broader picture of how the Internet held up during SuperStorm Sandy. Monitoring from Keynote, Renesys and RIPE provided a high-level analysis of how the Internet managed the storm, allowing us to paint a broader picture than the facility outages we’re tracking in our data center coverage.

Renesys saw carriers scrambling for alternate paths outside of New York, and Keynote saw end users lose connections for both landline and mobile connections, and retail sites faring better than news sites under extreme traffic. RIPE delivered some preliminary results on what the superstorm did to the Internet in general, measuring connections from the impacted East Coast with a location on the West coast and one in Germany.

The overall picture shows how resilient the Internet truly is, with impacts remaining local to the storm ,for the most part. "The Internet is highly resilient and is designed in such a way as to allow for its operation despite the impact of natural or man-made disasters," said Shawn White, Keynote’s Vice President of Operations. "This doesn't mean impacts won't be felt locally by those directly in the line of a storm or in those areas where a majority of Internet traffic flows. This is definitely the case with Hurricane Sandy's impact in the New York City area.”

Renesys

Renesys provides an Internet Health Portal, which provides detailed reports about the health of internet from any region. It had its eyes on the storm and provided a summary of its findings. New York happens to be a major hub of international telecommunications, so during the storm Renesys found traffic shifting away from the city as carriers scrambled for alternate paths. The company’s data center in Portsmouth, New Hampshire experience multiple power outages, though it was able to deliver services without interruption throughout. This video from Renesys shows how Internet outages spread across New York and New Jersey.

RIPE

RIPE creates a collection of live internet maps on its RIPE Atlas. It measured the performance of connections within NY, NJ and Philadelphia going to both the West Coast and overseas to Germany. It utilized the measurement of 14 probes located in these 3 East Coast cities and how they performed going to one location on the West Coast and one in Germany. The company stopped receiving data from all probes located in New Jersey and a majority of probes in New York city around Midnight UTC during the storm. The stoppage was attributed to power outages. Philadelphia stayed up, but saw increased round trip times (RTTs) to Germany and the West Coast.

Keynote

Keynote is a mobile & web cloud testing and monitoring company based in San Mateo, California. Keynote was watching the performance of leading mobile and connected websites closely for the duration of Hurricane Sandy and its aftermath. Keynote’s global measurement infrastructure consists of web performance testing & monitoring agents in over 100 locations, and mobile monitoring agents & network operator probes in over 250 locations. The company emphasizes end-point testing and monitoring, or monitoring where your actual users are located and how they connect, be it mobile or web browser.

There were a lot of Internet connectivity problems in New York. In terms of web sites, retail sites came away fairly unscathed thanks to experience with traffic spikes and subsequent use of technologies to address sudden bursts of traffic. Interestingly enough, some performance issues were related to third-party content such as ads slowing down performance, as was seen with the New York Times. Finally, the company points out the fragility of the mobile web in highly dense populations, as even a single cellular tower going down can wreak havoc.

Keynote findings in detail  

Its monitoring showed that almost everyone in and around NYC had full scale or sporadic performance issues with internet connectivity, for both high-speed landline and mobile connectivity. The company’s measurement computers in New York experienced outages for basic internet requests starting at 10:49pm EDT, lasting throughout the night. The data centers Keynote uses in NY also experienced sporadic power outages, including one incident where on-site generators failed due to water flooding generator areas. The mobile agents also experienced significant slow-downs and service interruptions, due to cell towers affected by power outages and voice and data traffic surges.

The top web sites impacted were news and media sites, which typically get traffic surges during these types of events. The most severe traffic spike-related slowdowns were seen at CNN, Weather.com, The New York Times, the Wall Street Journal (NYT and WSJ both dropped paywalls during the storm), Bloomberg, and The LA Times. Weather.com was most affected between 1:42pm EDT and 4:33pm EDT, during which load times slowed to a crawl or the page was completely unavailable. The New York Times faced a similar slowdown in the same period (1:38pm-4:33pm) but Keynote found this was the result of 3^rd party content embedded on the homepage, not just the traffic load on company owned servers. MarketWatch went down completely and posted a notice.

Retail sites fared better. More accustomed to traffic spikes, particularly during holiday seasons, and more sensitive to latency – customers are quick to abandon e-commerce sites because of slow performance – these sites are architected with traffic spikes in mind. Keynote’s Retail Index showed no significant impact related to Sandy.