Internap Data Center Outage Takes Down Livestream, StackExchange

Internap experienced a power outage at its data center located in the Google-owned building at 111 8th Avenue in New York City. There appears to have been a utility power outage, followed by a problem with uninterrupted power supply system that supported the Internap facility. The outage affected about 20 customers, according to Internap.

The outage  brought down a few notable Internap customers, including Livestream, a popular streaming video platform, for most of the morning Friday. It also affected the StackExchange network of sites, popular among developers. So far, the customers have been more vocal about the outage than Internap has been.

Livestream first reported issues at 3:49 a.m. EST. The company continued to report problems both on its status page and through live tweets throughout the morning. As of 11:30 EST the company reported that power was restored and systems were being brought back online.

“We have experienced a power outage with our data center provider Internap. We are still working to restore power as quickly as possible,” Livestream said in an update. The data center engineers are trying to restore power.

StackExchange reported issues on its status page as well as its Twitter feed. The company first failed over to its Oregon data center, but incurred further problems when migrating back to the main New York data center. "We are in a holding pattern since the UPS system serving both our primary and backup power feeds (yes, we’re very excited to learn this too) is still being worked on," Stack Exchange Network said.

After the hard power outage, the problem appeared to be with the UPS system, something that has affected Internap in the past. Utility power to the building has been restored, but Internap said engineers were still working with its electrical vendors to stabilize UPS power.

In 2009, Internap suffered an outage in Boston that resulted from poor battery maintenance. The outage caused finger pointing between Internap and VoIP provider Ooma that resulted in some bad publicity.

How Internap handles this morning's situation will be integral in minimizing the negative impact of the outage. Livestream is a sizeable player and can raise a ruckus if the situation is not handled correctly.

Outages are never a good thing, but transparency, frequent communication and taking responsibility goes a long way in retaining customers. If the provider does not come clean about causes of an outage, chances are a customer will be more vocal about it.

In addition to 111 8th Avenue, Internap's New York metro data centers include a facility at 75 Broad Street in New York and another one across the Hudson River at 1 Enterprise Avenue North in Secaucus, New Jersey.

UPDATE: Internap issued the following statement: On Friday, May 16 at approximately 3 a.m. ET, Internap experienced a power outage in a portion of its data center located at 111 8th Avenue in New York City. Utility power was restored at approximately 6 a.m. ET. The outage impacted approximately 20 customers using a combination of colocation and IP connectivity services. We are still working to determine the root cause of the disruption, but the site has been returned to normal N+1 redundant configuration. We have been providing customers with hourly status notifications since the beginning of the event, and our technicians are onsite 24x7 to assist them. Customers can contact [email protected] with questions.