Direct-to-Chip Liquid Cooling: Optimizing Data Center Efficiency

The integration of artificial intelligence (AI) and advanced cooling technology is creating significant shifts in data centers. Foremost among these is the rise of direct-to-chip liquid cooling, a sophisticated solution redefining thermal management in high-density GPU clusters.

With the deployment of NVIDIA H100 GPUs finally overcoming previous delays, 2024 is shaping up to be a pivotal year that marks a significant leap forward in AI and advanced computing capabilities.

Direct-to-chip (D2C) liquid cooling is transitioning from a luxury performance boost to an indispensable requirement for fueling high-density clusters in colocations, edge computing, and federal applications.

AI’s Role in Cooling Technology Innovations

With the escalating demand for AI and its power-intensive computations comes a significant thermal challenge: efficiently managing the cooling of AI workloads. Because of this, there's an anticipated surge in the implementation of liquid cooling solutions, particularly single-phase direct-to-chip cooling, to manage the extreme heat produced by densely packed server racks and high-density GPU clusters.

The global data center liquid cooling market, which accounted for $2.25 billion in 2021, is projected to grow at a CAGR of 25.8%, reaching approximately $31.07 billion by 2032, a direct reflection of the market’s urgent need to cool CPU and GPU clusters, which are increasingly pivotal in AI and High-Performance Computing (HPC) applications. This evolution contributes to enhanced system performance and more effective power management strategies.

Related:Direct-to-Chip Cooling: Everything Data Center Operators Should Know

The Emergence of GPU Pods and Modular Data Center Solutions

As generative AI adoption escalates – highlighted by a KPMG survey where 72% of U.S. CEOs consider it crucial – the urgency for sophisticated cooling technologies, especially direct-to-chip liquid cooling, is at an all-time high. The investment in premium technologies, such as NVIDIA H100 GPUs costing over $30,000 each, underscores the essential need for efficient cooling to protect these investments.

Responding to this need, companies are developing GPU pods and modular data center solutions designed for today's complex server environments, including cloud computing and high-density setups. These innovations emphasize efficiency and leverage direct-to-chip cooling to enhance server performance, mitigate overheating risks, and improve GPU utilization.

In 2024, the move towards strategic partnerships with liquid cooling and infrastructure providers offers streamlined, effective management of dense computing clusters, making modular data centers a scalable and efficient option for data center operators. This approach not only meets the growing demands of advanced computing but also promotes operational efficiency, reduces energy consumption, and facilitates rapid expansion.

Related:The Pros and Cons of Wind Power for Data Center Sustainability

Direct-to-Chip Liquid Cooling Growth

Single-phase direct-to-chip liquid cooling is emerging as a front-runner in the cooling sector this year. Utilizing microchannels or microjets, which are part of the cold plate systems, to efficiently dissipate heat. In the realm of cooling technologies, direct-to-chip and immersion cooling methods have been in a competitive race for dominance.

In the context of single-phase immersion cooling, there is ongoing development to enhance its capability for handling higher power densities. This approach now includes integrating fans and additional power mechanisms into the oil tank.

Recent developments have not achieved more than 1,000W of cooling capacity, and the operational conditions have not yet met the compliance standards set by chip manufacturers. For instance, these experimental setups often result in chip temperatures exceeding 80 degrees Celsius and require high flow rates, around seven liters per minute. Despite increasing performance at the cost of efficiency, immersion cooling has its use cases for the market.

On the other hand, two-phase liquid cooling is becoming more popular, spurred by the demands of AI-driven applications. This method faces its own set of regulatory hurdles, as the industry moves toward environmentally friendly and operationally efficient cooling solutions.

Regulators Step In: The Environmental Impact

For two-phase liquid cooling, direct-to-chip or otherwise, under scrutiny is the use of chemicals in two-phase immersion cooling systems, particularly,  polyfluoroalkyl substances (PFAS), often referred to as "forever chemicals." These substances, while effective in cooling, are being investigated for their environmental and health impacts. The market experienced the effects of regulatory changes firsthand when 3M, a major PFAS manufacturer, closed its primary plant in early 2022. In a time like 2024, there is an expectation that regulatory measures regarding these chemicals will tighten, reflecting a global commitment to sustainable practices.

In response, enterprises are likely to gravitate toward single-phase direct-to-chip (D2C) liquid cooling manufacturers to ensure long-term sustainability and compliance. The push for sustainable cooling methods is integral to supporting the next generation of advanced computing, underscoring the industry's commitment to innovation and environmental responsibility.

Liquid Cooling Expands into Multi-Tenant Colocation Facilities

In 2024, the reach of liquid cooling is expected to extend beyond individual enterprise data centers and take over multi-tenant colocation facilities. As enterprises continue to prioritize performance and efficiency, colocations will play an influential role in offloading and managing diverse workloads.

Multi-tenant facilities often house varied workloads from different clients, each with unique cooling requirements. However, as the demand for computing power intensifies, colocation facilities are expected to incorporate preliminary liquid cooling solutions into their offerings to remain competitive.

Leading colocation providers such as Equinix and Sabey Data Centers are already taking proactive steps to diversify their cooling solutions. Equinix, for instance, is rapidly deploying liquid cooling solutions, including single-phase direct-to-chip and rear-door heat exchangers, to support enterprise AI workloads.

Meanwhile, Sabey Data Centers saw a significant 13.5% reduction in power consumption through a liquid-assisted air cooling implementation, as demonstrated in a recent case study. This reduction enables their customers to deploy more high-density IT servers while utilizing less space. This not only expands Sabey's facility capacity but also allows them to offer competitive electricity rates due to enhanced cooling efficiency.

As liquid cooling solutions gain favor for AI and HPC clusters, colocations that embrace these solutions can accommodate more computing power within a smaller footprint, increasing utilization while catering to a diverse customer base. Endorsing water-based liquid cooling enables colocations to make their global campuses more efficient in a safer manner than two-phase liquid cooling.

Liquid Cooling is the Solution for Data Center Infrastructures

In conclusion, the rapid evolution of artificial intelligence and the corresponding need for advanced computing capabilities are pushing the boundaries of traditional thermal management solutions. Direct-to-chip liquid cooling, particularly single-phase systems, stands at the forefront of this revolution, offering an efficient, sustainable solution to the thermal challenges posed by high-density GPU clusters and AI workloads.

As the global data center liquid cooling market is projected to grow significantly, reflecting the industry's shift towards more energy-efficient, high-performance computing infrastructures, the adoption of these advanced cooling technologies is not just a trend but a necessity. With regulatory bodies stepping in to ensure environmental compliance and the expansion of liquid cooling into multi-tenant colocation facilities, the future of data centers looks promising.

This shift towards direct-to-chip liquid cooling encapsulates a broader movement towards innovation, sustainability, and efficiency, marking 2024 as a transformative year for data center infrastructure and the computing industry at large. 

Bernie Malouin, CEO and Founder of JetCool.