AMD Updates Instinct Data Center GPU Line

During Computex 2024 in Taiwan, AMD unveiled its latest AI processing card, the Instinct MI325X GPU, which boasts significantly more high-bandwidth memory (HBM) than its predecessor.

Computex 2024 is a major event for hardware announcements, where OEMs and parts suppliers showcase their newest innovations. Nvidia, for instance, introduced its Blackwell systems and announced the availability of its Spectrum-X Ethernet stack designed for AI workloads.

AMD introduced the Instinct MI325X GPU, featuring a faster processor and a substantial increase in HBM compared to both the MI300X and Nvidia’s Blackwell processor. The MI325X is equipped with 288 GB of HBM3e memory and offers 6.0 TBps of memory bandwidth, while the MI300X has 192 GB of HBM3 memory with 5.3 TBps bandwidth.

According to AMD, the Nvidia Hopper H200 accelerator’s top performance includes 141GB HBM3e memory and 4.8 TBps of memory bandwidth. However, Nvidia’s upcoming Blackwell generation, expected this fall, boasts up to 384GB of HBM memory and 16TB of bandwidth.

Despite the competition, the MI325X excels with peak theoretical throughput for 8-bit (FP8) and 16-bit (FP16) floating-point operations reaching 2.6 petaflops and 1.3 petaflops, respectively—30% higher than the H200’s capabilities. Moreover, AMD claims the MI325X can handle a 1-trillion-parameter model entirely, doubling the capacity of the H200.

Both the MI325X and MI300X utilize AMD’s CDNA 3 architecture, derived from the RDNA architecture used in gaming graphics cards but specifically optimized for data center applications like generative AI and high-performance computing.

Looking ahead, AMD plans to launch the CDNA 4 architecture in 2025, introducing the Instinct MI350 series. This new series is expected to offer up to 35 times the AI inference performance of the current MI300 series. By 2026, AMD aims to release the Instinct MI400 series, built on the forthcoming CDNA “Next” architecture.

Brad McCredie, AMD’s corporate vice president of data center accelerated compute, emphasized the company’s commitment to innovation and performance, saying, “With our updated annual cadence of products, we are relentless in our pace of innovation, providing the leadership capabilities and performance the AI industry and our customers expect to drive the next evolution of data center AI training and inference.”

The MI350X will feature 288 GB of HBM3e, similar to the MI325X, but with enhancements including a shift to a 3-nanometer manufacturing process, down from the 6nm nodes used in MI300 chips. It will also support new floating-point data formats, FP4 and FP6, with FP6 being unique to AMD, unlike Intel and Nvidia who have adopted FP4 and FP8.

While AMD’s Instinct line currently trails Nvidia’s Hopper GPUs, it is rapidly gaining traction with key partners. AMD has highlighted demand from Microsoft for its Azure OpenAI services, Dell Technologies for enterprise AI workloads, Supermicro with various solutions using AMD Instinct accelerators, Lenovo offering Instinct in its ThinkSystem, and HPE integrating them in HPE Cray