NVIDIA Co-Design Enhances AI Factory Memory Efficiency and Throughput

NVIDIA has unveiled a co-design strategy aimed at maximizing the efficiency of high-bandwidth memory (HBM) in AI factories. By optimizing how memory is utilized, the company has achieved significant throughput gains, with the NVIDIA Vera Rubin NVL72 delivering up to 30x the throughput per megawatt compared to the GB300 NVL72, despite both GPUs maintaining the same 288 GB HBM capacity. This improvement highlights the importance of maximizing the useful work each byte of memory performs, rather than merely increasing capacity.

Source: Blockchain.News