As the AI boom gains momentum, Amazon Web Services (AWS) has unveiled a new custom-built cooling system to support Nvidia’s next-generation AI chips. These Nvidia AI chips, which power many large language models and AI workloads, consume massive amounts of energy and generate significant heat, pushing traditional data center infrastructure to its limits.
To meet these evolving demands, Amazon developed the In-Row Heat Exchanger (IRHX), a proprietary cooling system designed specifically for use in both existing and new AWS data centers. Unlike typical liquid-cooling methods, which would require large-scale overhauls and excessive water usage, IRHX is plug-and-play and scalable to Amazon’s extensive infrastructure.
Amazon’s VP of Compute and Machine Learning Services, Dave Brown, emphasized the need for a solution designed for hyperscale deployment, as existing systems would consume too much floor space or increase water usage, while maintaining efficiency and sustainability at the scale.
With this innovation, AWS has launched new P6e instances, enabling customers to tap into unprecedented levels of compute performance. These instances are powered by Nvidia’s GB200 NVL72, which houses 72 interconnected Nvidia Blackwell GPUs in a single rack—ideal for training and deploying massive AI models.
While companies like Microsoft and CoreWeave have already provided access to GB200 NVL72-based clusters, Amazon’s IRHX sets it apart by enabling dense compute capacity with minimal environmental impact. AWS remains the global leader in cloud infrastructure and is now reinforcing its position by solving one of AI’s most pressing infrastructure challenges—efficient cooling.
Amazon’s in-house hardware innovation strategy, including custom AI chips, storage, and networking solutions, aims to reduce dependency on third-party suppliers and improve profit margins. In Q1 2025, AWS recorded its highest operating margin since 2014, demonstrating its commitment to in-house development.
Not to be left behind, Microsoft has also followed a similar path, developing its own AI chips and corresponding cooling systems, such as Sidekicks for its Maia chips, in a race to power the next generation of AI.
As the demand for generative AI applications continues to surge, Amazon’s investment in cooling infrastructure for Nvidia AI chips signals a strong commitment to staying ahead in the competitive cloud and AI infrastructure landscape.
