Virginia IT Systems Lack Network Redundancy

The state of Virginia's new state IT system is experiencing downtime in key state services because of a mind-boggling oversight: the state apparently neglected to include network redundancy as a requirement in a 10-year, $2.3 billion outsourcing deal won by Northrop Grumman.

Here's what George Coulter encountered when he took over as the state's chief information officer in August: "The first thing I noticed was that the network that Northrop Grumman rolled out didn't have redundancy, backup," Coulter told the Richmond Times-Dispatch. "The contract does not call for redundancy in carriers . . . in the network. Why that wasn't put into the network, I don't know. This is a service we have to have."

The oversight is taking its toll. The Richmond paper reports that in just five weeks this fall, the Virginia DMV suffered 12 computer system outages, putting individual offices out of business for a total of more than 100 hours. "The problem of no-redundancy . . . accounts for 90 percent of our outages," DMV chief information officer David Burhop told the Times-Dispatch.

The paper says the lack of redundancy also hampered communications services for the Virginia Department of Transportation when a state of emergency was declared during heavy rains and flooding from the Nov. 11 Northeaster.      

Here's another to this story. The state's $2.3 billion outsourcing deal with Northrop Grumman was signed in 2005, shortly before the appointment of Aneesh Chopra as Virginia's Secretary of Technology. In April of this year Chopra was named as the Obama Administration's CTO for the federal government.

Since he arrived after the deal was signed, Chopra isn't liable for the oversight in a network redundancy requirement for the outsourcing contract. But in nearly four years as the state's Secretary of Technology, it appears Chopra either never realized or never addressed the scope of the problem.

The issue is relevant given the Obama administration's ambition to shift much of the government's IT operations to a cloud computing model. If problems emerge as federal apps move to the cloud, will Chopra and his team be able to identify and resolve them? This is especially critical with requirements for redundancy and uptime. Let's hope the Obama team can improve on the experience in Virginia.

Coulter is calling for an emergency meeting of the state's Information Technology Investment Board in the first week of December to deal with the lack of redundancy in the state's IT system.