Internap Completes Move Out of Google's Manhattan Building

Internap has successfully moved its New York data center out of the Google-owned building at 111 8th Avenue in Manhattan and into its brand new Secaucus, New Jersey, facility.

Google, which uses the building for office space, has not allowed data center providers there to renew their leases since it bought it four years ago. Internap was one of the casualties. The building is one of the most desirable data center locations in the world because of the huge amount of carrier networks it is connected to. It is one of a handful of the top carrier hotels in Manhattan.

Internap has been working non-stop to move its New York data center customers to New Jersey. One of the customers, Outbrain, moved more than a thousand servers over a rainy Memorial Day weekend this year.

Other data center providers that still have active leases in the building include Digital Realty Trust, Equinix, and Telx. Different providers have different lease terms in the building. Telx CEO Chris Downie, for example, told us earlier this year the colocation company's lease there would not end for several more decades.

An Undesired but Necessary Project

Internap built the Secaucus data center with plans for long-term growth, the company's senior vice president Mike Higgins said.

“We went through quite an undertaking starting in 2013 – something in the data center space that no one likes to do,” he said. Internap operated out of 111 for over a decade, but its lease was up. “Customers had been with us a very long time, running production environments. The migration took a tremendous amount of effort for both Internap and the customers, but we’re happy to say the project was successful.”

Apart from the physical move -- complicated by Google employees moving in as Internap was moving out -- network configuration was a big part of the puzzle. Customers couldn't afford downtime, and the provider was somewhat between a rock and a hard place. The company put in temporary infrastructure so customers could keep network connections live.

“Many customers took a maintenance window, a number of customers did not,” said Higgins. “We had other customers that needed to lease temporary infrastructure. Those customers took advantage of bare metal cloud.” Bare metal cloud is one of Internap's services.

For some tenants the move meant conducting inventory of their applications. “Some customers were one, two, three CIOs removed from their initial Internap deployment,” said Higgins. “A number of customers had to get external project management support to get a handle on the applications they were running.”

Many customers didn’t migrate with Internap because their technical expertise was in Manhattan. Still, the company managed to migrate enough customers to Secaucus for the move not to affect revenue, Higgins said.

Outbrain's 1,000-Server Migration

Content discovery platform provider Outbrain operates from three data centers in the U.S., one of which is now the Secaucus facility. The company's infrastructure serves more than 72,000 links to content every second.

“We were cognizant of customers with larger environments,” said Higgins."We built a migration plan around customers like Outbrain to make sure we had the resources to accommodate everyone. All hands were on deck, we provided full NOC support. We mapped out the weekends how those customers would fit.”

“There are two approaches to migration,” said Orit Yaron, head of operations, Outbrain. "The first is to build everything new on the new side and slowly migrate it over the network. We took the second approach, the big bang approach. We moved very fast over the weekend, meaning more risk but less intrusive.”

Outbrain performed a lot of project management and risk mitigation prior to the move. The company wanted to migrate over a long weekend, and Memorial Day was the only option. “We didn’t have any infrastructure in Secaucus,” said Yaron. "It was a very aggressive timeline with a lot of cross communication between us and Internap needed.”

The company used multiple trucks and planned what infrastructure specifically would go in what truck. “We did this for redundancy,” said Yaron. “We wanted to split the nodes on different trucks, so if one were to get stuck, we could continue to operate. People thought we were paranoid, but we took a lot of precautions.”

The company uses commodity servers that all look the same, so each one was tagged three times just in case a tag went missing.

Outbrain uses a continuous integration methodology, meaning there are lots of production deployments on any given day. It was very important for them to maintain that during the move, said Yaron. The company performed lots of tests to make sure the monitoring and test systems remained active and ran a test simulating 111 8th being taken down to prepare. Scripts were plugged in to prevent a flood of alerts during the move and to bring the system back to normal following the move.

The weekend of the migration, it was raining hard, but the move was completed in 36 hours rather than the planned 48.

“We were prepared for almost every scenario,” said Yaron. “The location of the cabinets, what equipment goes into what truck, printing out the phone numbers, we went into deep-level details in the planning.”

'Most complex migration'

Internap's New York data center is now out of the Manhattan building, and the company is ready for its future in Secaucus.

“This was by far the most complex migration I’ve been a part of,” said Higgins. “Think about New York, the number of customers in that multi-tenant environment. I’ve been a part of some enterprise migrations, but none this complex.”