Immersion Cooling Comes to the Hosting Market

Submerging servers in fluid isn't for everyone. But immersion has proven effective for cooling high-density server configurations, including high-performance computing clusters for academic computing, seismic imaging for energy companies, and bitcoin mining.

Now CH3 Data is making immersion cooling available to the broader web hosting market. The service provider just completed work on a new data center in Austin, Texas, featuring servers submerged in a dielectric fluid. CH3 Data was founded in 2011 as Midas Green Technologies, and has since split into two companies, CH3 handing data center services, while Green Midas focuses on commercializing its immersion cooling technology.

In addition to supporting extreme power density, immersion cooling has the potential to produce significant savings on data center infrastructure, allowing users to operate servers without a raised floor, computer room air conditioning (CRAC) units or chillers.

Chris Laguna, data center manager for CH3 Data, provides some additional details on CH3's use of immersion cooling and the company's new facility. Here's a Data Center Knowledge Q&A with Laguna:

Data Center Knowledge: What prompted you to adopt immersion cooling for your business?

Chris Laguna, CH3: Our CTO Chris Boyd started looking for cooling alternatives back in the mid-2000s. During this period, our electric provider increased rates substantially, so Boyd started thinking about ways to improve our data center efficiency. Better heat transfer was an obvious way, so he looked into liquid-to-the-server solutions. These didn’t exist in a useful form at the time, but gamers had liquid to the chip systems available in the market. However, these did not fit the ideal 1U (1.75”) or less form factor, so deployment of massive water-cooling systems for individual servers didn’t make financial sense. In addition, “dripless” connectors were not guaranteed to be dripless for enough cycles of server installation and refresh over the life of a data center to make deployment practical.

In March of 2011, we formed the company Midas Green Technologies in Austin, Texas, and became the first commercial data center in the world running servers immersed in tanks of mineral oil (Immersion cooling has been used as far back as the 1940s, more recently in the Crays of the 1980s, but never in a hosted data center form). Over the first eighteen months of operation, we found changes we felt were imperative to make the system a proper data center solution. We decided if we were going to improve things, we would need to do it ourselves and use our team’s extensive knowledge and experience in data center operations to better the product. To accomplish this, we have worked with several engineering and legal firms and simply put, we believe we have a better way to build a data center.

Midas Green Technologies LLC filed its IP and today has 1 patent pending with over 28 claims. That design has been realized and is currently deployed in CH3 Data's new facility where we continue to run servers and other devices submerged in our tanks.

DCK: What are the advantages and challenges in using this approach?

Laguna: The advantages are quite abundant. During the summer of 2014, we were able to maintain a PUE of 1.09 in Austin, Texas during 95F+ heat and relative high humidity. Our GPU and Mining clients are especially in love the immersion-cooled environment for the densities and over-clocking benefits it provides.

As for challenges, we do not yet have a broad range of immersion-certified servers and networking gear, but this is very rapidly changing. We primarily user Super Micro servers, but Dell has some exciting opportunities that will give us and our customers many more options in the near future.

ch3-enclosures

Several of the immersion cooling tanks being used at CH3. Each is filled with a dielectric fluid, which is similar to mineral oil. It transfers heat almost as well as water, but doesn’t conduct an electric charge. (Photo: CH3)

DCK: We don't see many examples of immersion being used in multi-tenant or hosting environments. How has the use of immersion cooling approach been accepted by customers? Is education an important component of your sales process?

Laguna: Our customers love it, but surprisingly not for the reasons you would expect. When searching for a reliable data center solution, many factors are included: Price, SLAs, Security, etc, and we can check all those boxes. We have a great customer base who use us because of the environmentally-friendly nature of immersion cooling. However, with majority of our customers, it ultimately comes down to the bottom-line. It is a divide that you can almost be cleanly drawn between Main Street and Wall Street. We are unique in that we offer a “green” solution that will actually cost you LESS.

DCK: Does immersion cooling create any considerations on the staffing front? Is it easy to find staff with experience with this equipment, or is it primarily a training issue?

Laguna: At the end of the day we are still performing the same functions as a traditional data center (except we never have to change out server fans because we do not use them!), our staff does have a small amount additional training to work with servers in the immersion environment. That training is mostly handling and processes, performing any server repairs remains the same.

A look at some of the piping that supports the liquid cooling system at CH3 Data in Austin. (Photo: CH3)