New Data Center Design Drives Efficiency Gains for Dupont Fabros

dft-acc7-build

An aerial view of the huge ACC7 data center under construction in Ashburn, Va., where DuPont Fabros Technology is implementing a new design. (Photo: DuPont Fabros)

ASHBURN, Va. - Data center developer DuPont Fabros Technology has overhauled its facility design, seeking to bring the benefits of hyperscale server farms to companies leasing third-party data center space. The prototype for the new design is the massive ACC7 data center at the company's campus in Ashburn, Virginia.

DuPont Fabros (DFT) has refined its approach to how it cools and powers its data centers, and has shifted from a raised-floor to a combination of a hard floor and a hot-aisle containment system to house customer cabinets. The end result is a data center that maintains DFT's emphasis on reliability while delivering big improvements in energy efficiency and ease of maintenance. ACC7 will also allow customers to expand their capacity by housing server-filled containers on cement pads next to the data center.

“We recognize that in this industry, things change and evolve,” said Scott Davis, Senior Vice President of Operations for DuPont Fabros Technology (DFT). “We sat down with the design group and looked at the trends (n data center design). We took all those trends and came up with goals. The end result is (a data center that's ) cheaper to build, requires lower maintenance, and has an industry leading PUE. We never save at the cost of reliability or resiliency.”

The company expects annualized Power Usage Efficiency (PUE) UE to be below 1.14 at 75 percent capacity, and below 1.13 at 100 percent utilization. That's an improvement on DFT's current design, which delivers a PUE of 1.28 at other facilities on the Ashburn campus.

With the new design, DuPont Fabros has gone bigger, taking advantages of economies of scale as well as a complete tech overhaul to bring efficiencies to new heights, without sacrifice of redundancy or resiliency. In approaching the design, Davis and the DFT team weighed three goals:

Building Bigger, But in Smaller Phases

The ACC7 data center is 446,000 square feet in size and has a total power capacity of a whopping 41.6 megawatts. The building includes 28 large computer rooms, with a standard critical load of 1.486 megawatts each, and the ability to increase density to offer up to 2.1 megawatts each. Each data hall can accomodate approximately 378 standard cabinets.

Dupont Fabros is building out the entire shell, finishing eight computer rooms in the first phase of construction, while the other rooms will be empty shell space. In the past, DuPont Fabros has built out as much as 18 megawatts of finished space at a time. With the new design, they'll start out with 11.9 megawatts of customer capacity, and build out the building in smaller phases as it leases up.

In a base computing room there are a pair of a+b Power Distribution Unit (PDU) transformers capable of carrying a 1.5 megawatt load. The computer room air handling (CRAH) units are on the perimeter of both walls.

Dupont Fabros is not going to have a raised floor in this design, with all cabling running overhead. Each room has a ceiling height of 13 feet, leaving plenty of room for overhead cabling.

This approach allows Dupont to sell space in smaller chunks. There's an option to deploy a still mesh fence in the computer rooms to break them up, or to isolate the air handlers from the server area for security reasons.. The PDUs and distribution panels are out of the room to increase the footprint for IT equipment.

Making Greater Use of Free Cooling

The new facility capitalizes on the greater opportunity for free cooling. The company calls its new approach “water side economization plant with chiller assist.” This means that outside air will cool water for the cooling system, using a plate and frame heat exchanger, which is expected to be the primary cooling source for 75 percent of the calendar year. Chillers will kick in on days with warmer temperatures. In the company's previous design, the chiller plant did all the work. Less use of chillers and pumps leads to lower energy bills.

On the data floor, DFT is now using a hot aisle containment solution to optimize for energy efficiency. "A form of 'hot aisle' containment is a requirement, either full aisle or cabinet chimneys," said Davis. "This is a necessary element to provide greater control (and separation) of hot and cold air so that there is no mixing. This is key in being able to pursue ultra low PUE values."

CRAH units supply the cold air to inlet side of the servers, which is then exhausted through chimney racks or via full aisle containment, and returned back to the ceiling plenum to the CRAH units.

The company is tapping into Loudoun County's new reclaimed water loop, in addition to white water and well. The reclaimed water loop will be in ACC3, 4,5,6 and the new ACC7. It's an extra source of water at a cheaper rate, and part of the effort to go green.

Electrical System: Medium Voltage Used

For the first time, the company is employing medium voltage technology for power distribution within the building. “The advances in medium voltage technology caught up in terms of reliability numbers,” said Davis. Before, the company didn’t see medium voltage (MV) as reliable enough, but the latest technology has made MV more reliable and resilient than low voltage, bringing significant cost savings, increased reliability and increased flexibility.

Medium Voltage also reduces feeders by a factor of 7x, meaning one feed does the work of 7. This allows them to build smaller duct banks, saving serious money because square footage here isn’t cheap. “It’s cheaper to build, there’s fewer terminations, and it’s easier to maintain,” said Davis. Clients also save money on build out and it’s a greener technology in the sense that it uses less material (copper, PVC, and steel).

Power comes out of distribution panels, and it’s possible to put A+B down every row. While not entirely new to the company, It’s using all overhead solutions as it believes that underfloor power and network distribution is being phased out. Busways are dominating RFP’s, according to Davis. The company has been eliminating raised floors this way, and again, the overhead solutions make for easier and more flexible maintenance.

Innovation at the PDU Level

DuPont Fabros is phasing out 120 volt single phase circuits and bringing in 230 and 240 volt for a 2x power boost to busway products

The company is introducing Medium Voltage Power Distribution Units (MVPDUs), which offers several advantages. The PDUs are 1.5 megawatts each and extremely compact. They've also moved.

“With legacy (design), you want to locate PDUs close to rows, but to us that’s not the most desirable location,” said Davis. “People want to put up racks in that space.” Now the PDUs are outside the room in pairs for immediate 2N redundancy.

“The PDU design is disruptive,” said Davis. The new medium voltage PDUs are custom built for DFT and oil-based, and tested as 99.6 percent efficient versus 97.5 for "dry" PDUs.

"Traditional PDU’s use dry-type transformers and use air as the transfer medium to conduct or remove the heat generated," said Davis. "Oil-filled units use an environmentally friendly mineral oil bath that the transformer windings are submersed in. This is a better conductor of heat and therefore leads to greater efficiency."

The oil-filled units are also more compact. "Dry-type tends to be for low-voltage applications, as air is a relatively poor insulator and thus the physical footprint of a same rating unit would be considerably larger for a dry type unit than the oil-filled units," said Davis. The technical term for the oil product is Type FR-3 biodegradable dielectric fluid.

The company is very excited about the advantages to medium voltage. It means greater safety with lower chance of arc flash incidents, a smaller footprint, higher reliability, fewer terminations and greater flexibility in the data center layout such as longer feeders without huge costs.

Facility to Offer More Power Choices

The typical room in the facility is 8,500 square feet with a baseline power of 1.486 megawatts, with the ability to go to 2.1 megawatts. However, the design is easily and rapidly expandable, and rooms are easily subdivided and rooms support a wide range of densities

“We want to appeal to a broad spectrum of variable room densities,” said Davis. “Several rooms are targeted to segment down as well. We can segment the power down to 375 kW, and smaller even with container support.” This is not the first time that the company has cut up a room to serve a customer. In Chicago and New Jersey, the company has done this for a few customers, with at least one instance of a 284kW deal.

However, Davis emphasized that they’re not setting out to undersell. The flexibility is partially for clients with low and high density needs and its deals will average 1.5 megawatts. Unlike Chicago and New Jersey, Ashburn tends to be a market for Internet clients seeking large footprints.

ACC7 also has provisions for containers and hook ups. Customers can use containers outside of the core facility space for incremental growth, disaster recovery or client preference. It’s just one more option built in to make ACC7 as flexible as possible.

Additional Customer Flexibility

There are also a number of smaller touches that go a long way. While not new, the facility reduced CRAH expense. It uses variable speed fan horsepower for 60 percent savings. DuPont Fabros will use LED lighting with motion sensors, with the light distributed where you need it, onto the components. The aisles will look darker, but what you need to see is lit up. It’s implemented all the touches to go for LEED Gold certification as well.

“As a proof of concept, we are going to install 200 cabinets and 10-15 kilowatt load bank,” said Davis. “We’ll prove that the room can handle it.” The company has reached out to equipment manufacturers to lend out equipment for the proof of concept and will document its efforts.