Arc Flash: the Real Data Center Security Hazard is Just a Spark

What good is protecting your data center from every possible incursion, from any known or unknown source, on account of any known or foreseeable vulnerability, if the greatest threat it faces today is a spark of electricity?

An arc flash study could become at least as valuable to your data center as a vulnerability assessment or a penetration test, says Joe Furmanski, the veteran facilities director for the University of Pittsburgh Medical Center.

“An arc flash study looks at all the electrical components, from the source at the power company, the whole way through to the plugs that you plug into your IT equipment,” Furmanski told us in an interview.  “They look at how all the circuit breakers are set up — it’s called a coordination study — and they look at the power going through.  They punch in all these formulas to figure out, will these breakers move fast enough if there’s an electrical short, or will they move too slowly and let the capability of an arc flash be created?”

The typical electrical safety document [PDF], such as the kind published by the US Occupational Safety Hazard Administration, reads like a brochure the fire chief would pass out during lecture time in junior high.  Reads one passage, “There exists a number of ways to protect workers from the threat of electrical hazards,” prior to a list which includes remembering to turn things off.  It’s been difficult for OSHA, the National Fire Prevention Association, and their peers to find the right attitude with respect to addressing data center professionals.

Furmanski will go into detail on the topics of prevention, remediation, and mitigation of power events in a comprehensive presentation at the Data Center World conference in New Orleans this September.

The NFPA tightens its rules every year, Furmanski told us, and OSHA maintains regulations stipulating that data centers must adhere to the NFPA rules.  Since 2013, this turning up the heat has had a positive effect on the conversations that data center engineers have among each other, he said.  But for some reason, that discussion has not yet translated to their clients.

Joe-Furmanski

In the eight years that Furmanski has spent with UPMC, he’s had three serious data center incidents on account of arc flash.  One, he admits, was preventable:  As a new electrical box was being put it and energized, it was rated for a lower voltage than what it was being fed.  “Luckily, our folks were on the other side of a wall when they flipped a breaker to turn on that panel,” he said.  “It blew the panel apart, but no one was injured.”

In another incident, a fellow cutting wires through the ceiling dropped some snippets onto the grid below.  Not immediately, but a while later, one of those snippets crossed two electrical circuits.  “As one of my guys walking down the aisle towards it said, it looked like a volcano spewing big hunks of orange out the top of it.”

See also: Electromagnetic Pulse: Can Space Weather Kill the Cloud?

Detecting electrical incidents such as these cannot be done, well, entirely electrically.  Electrons travel at one speed, even when there’s an explosion behind them.  So any signal that an incident is happening probably won’t beat the incident itself to the signaling mechanism.  At least some sensors for power distribution units need to be optical, because photons stand a much better chance (let’s say, about 100 percent of the time) of outpacing electrons, giving the mechanism the opportunity to trip breakers before the power event catches up.

Surprisingly, said Furmanski, few equipment vendors in the field actually sell these optical sensors.  New data centers don’t require these retrofits, and some enterprises are simply moving to new facilities, with designs that are more practical and more safety conscious.  But healthcare is among the sectors of the economy that don’t always have that luxury.

“Transformers are the killer of all things,” he remarked.  “In the old way of distributing power, you’d put transformers out on the data center floor and from there you’d wire them to your cabinets where the IT equipment was.  That, in many cases, is the source of the problem.”  Modern designs separate transformers further from the equipment and have adopted much safer power distribution equipment such as Starline Track Busway.

Joe Furmanski will go into much further detail on the topics of prevention, remediation, and mitigation of power events at 9:10 am Central Time Thursday, September 15, in Room R215 at Data Center World, presented at the Morial Convention Center in Downtown New Orleans.  Data Center World is presented by AFCOM, the association for educating data center and IT infrastructure professionals worldwide.

Register for Data Center World today!