Bitcoin Mining Arms Race Boosting Interest in Liquid Cooling

bitcoin-mining-boards

Densely-packed chips performing Bitcoin calculations are immersed in cooling fluid, which bubbles as it boils, removing the heat from the ultra-high density "mining" operation. (Photo: Allied Control)

There's a computing arms race going on in the world of Bitcoin. Interest in the digital crypto-currency is driving the development of specialized hardware chips, which are selling out almost as fast as they can be built. This is boosting interest in data centers using immersion cooling, in which high-density hardware is dunked into fluids similar to mineral oil.

This new frontier in high-performance computing can be seen in Hong Kong, where a bitcoin mining company called ASICMiner has created an unusual data center. Within the facility, rows of rack-mounted tanks are filled with Novec, a liquid cooling solution created by 3M. Inside each tank,densely-packed boards of ASICs (Application Specific Integrated Circuits) run constantly as they crunch data for creating and tracking bitcoins. As the chips generate heat, the Novec boils off, removing the heat as it changes from liquid to gas.

These systems are part of a huge computing network driving global payment processing for Bitcoin, which uses processing power to verify transactions. Participants in the network - which includes individuals, corporations and mining collectives - are rewarded with the issuance of new bitcoins, which happens about every eight minutes. There's a wrinkle: Over time the algorithms make it progressively harder to earn new bitcoins. If processing power remained static, it would take much longer to generate new bitcoins. The solution: more computing power!

Bitcoin mining can be done with traditional CPUs or GPUs, but most serious players have graduated to specialized chips such as FPGAs (Field Programmable Gate Arrays) or ASICs that can be optimized for specific workloads. This has led to the emergence of a new class of hardware vendors selling custom hardware for bitcoin mining.

There are two key elements in bitcoin mining: computing power and electric power. Miners must make the most of both resources.

Hardware Power

The state of bitcoin mining hardware is perhaps best expressed by Ravi Iyengar, who left a position as lead CPU architect at Samsung to launch CoinTerra, a startup building custom ASIC hardware for the bitcoin market. "I've been in arms races throughout my career - AMD, ARM, Intel," Iyengar told Reuters. "But none of them match the intensity of Bitcoin mining. Each month in Bitcoin mining is like a year."

Austin-based CoinTerra launched in August after closing a $1.5 million seed round of financing. It quickly sold out its first batch of units, and a similar sellout was reported by HashFast, a Bay Area startup featuring alumni of Xerox PARC and Engine Yard. Both CoinTerra and HashFast are building ASICs featuring state-of-the-art 28nm chip designs. Other early leaders in the Bitcoin hardware market include Avalon, KnCMiner, BitFury and Butterfly Labs.

The performance benchmark for Bitcoin hardware is gigahashes (GH) per second. The hash rate is the number of bitcoin calculations that hardware can perform every second. Tools have sprung up to help aspiring miners evaluate hardware performance and the economics of mining, including The Genesis Block and Decentralized Hashing.

NEXT: Bitcoin Mining Facility Design

These racks are filled with high-density servers immersed in cooling fluid. The installation in Hong Kong was created by Allied Control using 3M's Novec fluid. (Photo: Allied Control)

Energy Efficiency Drives Profitability

Mining profitability calculations aren't useful unless they account for the cost of electricity. That's why energy efficiency is crucial to the economics of bitcoin mining, as reducing the electric bill to operate the hardware has a direct impact on the profitability of the mining operation.

"The standard few KVA per 19-inch rack on electricity and cooling capacity of the average data center have long been exceeded by students mining in their dorms or basements," said Alex Kampl, VP of Engineering for Allied Control, a Hong Kong firm that builds high-performance hardware and data centers. "Mining pools and mining consortiums were organized over the Internet. Purpose-built mining farms started to operate. Investments became quite substantial, with professional people getting involved in chip design and bigger facilities being built around the world. We are talking about 1 megawatt to 5 megawatt facilities."

As it becomes more difficult to earn new bitcoins, miners boost their computing horsepower, and energy efficiency becomes more important.

"Whatever the hardware, there will be the day where the electricity cost to run it and cool it will be higher than the income it can generate," said Kampl. "That's when it's time to replace it with the next generation. If you are paying close to nothing for cooling, you can extend this by a few months, which is an eternity in the world of Bitcoin mining. Lifespans are becoming shorter and shorter. We might be talking about months or weeks in the near future."

Last month we profiled the Hong Kong data center project developed by Allied Control using a two-phase cooling technique known as open bath immersion (OBI). Allied Control does not disclose its clients, but the operator of the facility was recently revealed when photos of the data center were shared at Bitcoin Talk. Chinese entrepreneur Xiaogang Cao was given a tour of the facility by ASICMiner, which sells ASIC hardware and also mines for its own accounts.

Kampl says immersion cooling tanks are ideal for Bitcoin mining because they can support rapid hardware refresh cycles. The backplanes housing the ASICs can be designed to support multiple generations of chips, and immersion cooling can support much higher densities than current hardware requires. Allied Control has created a data center container design for HPC users, which provides the scalability and speed-to-market for mining operations.

Build Once, Upgrade Often

"The logical next step for (miners) is to use a universal system," said Kampl. "You build the enclosure and infrastructure only once and just replace the hardware inside. The system we have deployed at this moment is only scratching on the surface in terms of capacity."

Another leading provider of immersion cooling, Green Revolution Cooling (GRC), also offers a containerized system, which currently supports the world's most efficient supercomputer.

“We have had an extremely high amount of interest from this community," said Matt Solomon, Marketing director at Green Revolution. "It makes a lot of sense for them and they are catching on to this fact."

Some hardware builders are adopting a different approach to liquid cooling. CoinTerra is working with CoolIT, which provides cold plates and heat pipe technology to bring liquid cooling directly to the chip.

"Bitcoin mining systems carry a unique thermal profile, which require the most advanced, custom cooling solutions," said Iyengar, the CEO of CoinTerra. "We chose CoolIT because they have the best technology and track record in this space."

“Working with an industry pioneer like CoinTerra is an important step towards liquid cooling becoming an industry standard,” said Geoff Lyon, CEO of CoolIT Systems.