Major Migration Outages for NaviSite

For the second time this year, a data center migration following an acquisition has led to major customer downtime. Many customers of Alabanza say their sites were offline for up to 72 hours as they were migrated to a data center at NaviSite (NAVI), which acquired Alabanza in August. NaviSite was moving customer accounts from Alabanza's main data center in Baltimore to a NaviSite facility in Andover, Mass. See the end of this article for an update.

Alabanza is one of the largest hosting reseller specialists, and customers say a large chunk of the company's 175,000 sites have been offline. The migration problems for NaviSite follow a similar outage in late July when Hostway moved ValueWeb servers from Miami to Tampa. Hostway said later that more than 500 servers suffered hardware failures when they were restarted in the new facility.

Accounts posted by customers at Christian Web Host (a major Alabanza reseller) and Web Hosting Talk indicate that the move was planned to take place via a data transfer to hardware at Andover, with no physical relocation of servers. But network latency problems created problems with rsync (a popular file transfer tool) and NaviSite decided that it needed to physically move servers from Baltimore to Andover, a trip of about 420 miles.

The data transfer effort began at 5 am Eastern time on Friday, according to posts by Lance Odeneal at Christian Web Host's customer information site. Some of the server transfers were completed using rsync, but others encountered latency problems, leading NaviSite to load servers on trucks for the seven-hour drive to Andover. By Sunday evening, frustration was evident.

"I am honestly just at a loss at this point as I would understand each of you being very upset at us," Odeneal writes in a post to customers at 7:50 pm Sunday. "We have been put in a position with several other hundred hosts around the world that has each of our hands tied in this." He reported at 6 am this morning that servers were coming back online.

But as of 8 a.m., the main Alabanza site remains down, as do ChristianWebHost.com and LinuxWebHost.com, also an Alabanza customer. See IP Democracy and Light Reading for additional coverage.

UPDATE: As of 7:30 pm Monday, many sites remain offline. ""We are really hoping this is something we'll be able to address by a very reasonable amount of time, if not by tonight, then by early tomorrow morning," chief marketing officer Rathin Sinha told the Register. An Alabanza status page says Navisite has "over 90% of the server operating systems up and running in our Andover, MA datacenter," but also says it is "continuing to work on the name servers to enable the availability of the individual hosts." The page offers contact customers who remain offline, but says that "we anticipate that our updates will continue throughout the evening."