Blue Waters Supercomputer Debuts At 11.6 Petaflops

npcf-470

An exterior view of the National Petascale Computing Facility at the University of Illinois, home to the Blue Waters supercomputer (Image: NCSA/University of Illinois)

Less than two years after a reboot to switch vendors, one of America's most ambitious supercomputing projects has gone live. The $350 million Blue Waters supercomputer, which was dedicated last week at the University of Illinois' National Center for Supercomputing Applications (NCSA), was envisioned as the new frontier in American supercomputing technology.

Officials at the NCSA said the completed machine has delivered on that vision, NCSA spokeswoman Trish Barker said that peak performance for the supercomputer operates at 11.6 petaflops, or 11.6 quadrillion calculations per second. Had Blue Waters submitted this performance data for the November 2012 Top500 list of the most powerful supercomputers in the world, it would have ranked third.

"Blue Waters is a truly extraordinary computing system that will enable the nation's researchers, as well as our innovative companies, to achieve breakthroughs in a broad range of science and engineering," said Thom Dunning, director of the NCSA. "Blue Waters is also a unique resource for the University of Illinois and the state of Illinois, enhancing the path-finding role that Illinois has played in supercomputing for more than 25 years."

Illinois Governor Pat Quinn joined the National Science Foundation, officials from the University of Illinois and other state and business leaders to celebrate the debut and success of the NSF-funded project. Governor Quinn declared March 28th, 2013 Blue Waters Supercomputing Day in the State of Illinois.

Starting with a NSF grant in 2007 NCSA partnered with the University of Illinois to build the National Petascale Computing Facility (NPCF) to power Blue Waters -  a set of infrastructure, including a data center, plus computation, networking, and storage and code. After the data center opened, and an initial setback when IBM pulled out, the project awarded a multi-year project to Cray.

The Cray version of the now installed Blue Waters supercomputer is made up of over 300 cabinets, with 25,000 nodes, over 380,000 cores, aided by more than 3,000 NVIDIA GPU’s, and has over 25 Petabytes of usable storage. NCSA also selected the Black Diamond X8 switch from Extreme Networks to support the 25 petabytes of storage transported by the Ethernet network.

NSF - Transforming Scientific Process and Discovery

The NSF has invested heavily in supercomputers - inaugurating a system at NCAR last fall, and Stampede and Blue Waters this month. Researchers will utilize Blue Waters for science and engineering topics from earthquakes to astronomy and the molecular mechanisms of disease. Thanks to a remote system researchers can perform work 365 days a year from their workspace.

Brian Jewett, an atmospheric research scientist at the University of Illinois at Urbana-Champaign, is studying tornadoes, with the goal of better understanding the factors that can lead to dangerous twisters. "We already know from storm chaser reports over decades that mergers can either kill off a storm, or result in one that is much stronger," he adds. "Understanding this and other factors that influence the merger could help in future prediction and warnings."

Ten research projects have been awarded access to the Blue Waters supercomputer. The computing and data capabilities of Blue Waters will assist researchers in addressing questions of biology, nanoelectronics, ecological and economic impacts of climate change, and more.

bluewaters-ecophlex

The piping for they ECOPhlex cooling system developed by Cray, which uses both water and refrigerants. The ECOPhlex (short for PHase-change Liquid Exchange) cooling system uses two cooling loops, one filled with a refrigerant (R-134a ), and the other with chilled water. (Photo: NCSA)

bluewater-cabs

Blue Waters is composed of nearly 300 cabinets of equipment, divided between Cray XE6 and Cray XK7 supercomputer with NVIDIA Tesla Kepler GPU computing capability. (Photo: NCSA)

bluewaters-team

At the March 28 dedication ceremony, U.S. Rep. Dan Lipinski, NCSA Director Thom Dunning, University of Illinois Trustees Karen Hasara and Ed McMillan, and NCSA's Bill Kramer stand with the massive Blue Waters supercomputer behind them. (Photo: Brian Stauffer/UI News Bureau)