AMD introduces MI300x AI chip as ‘Generative AI accelerator’

Advanced Micro Devices CEO Lisa Su unveiled a chip in San Francisco on Tuesday that is a centerpiece in the company’s strategy for artificial intelligence computing, which will use its massive memory and data throughput for so-called generative AI tasks such as large language models. claims to.

The Instinct MI300X, as the part is known, is the follow-up to the previously announced MI300A. The chip is actually a composite of several “chiplets”, individual chips that are joined together in a single package by shared memory and networking links.

Su, on stage for an invite-only audience at the Fairmont Hotel in downtown San Francisco, referred to the part as a “generic AI accelerator” and said the GPU chips contained in it, a family known as cDNA3, has been “specially designed” for AI and HPC (high-performance computing) workloads.

The MI300X is the “GPU-only” version of the part. The MI300A is a combination of three Zen4 CPU chips with multiple GPU chips. But in the MI300X, the CPU is swapped for two additional cDNA 3 chipsets.

Too: Nvidia unveils new type of Ethernet for AI, Grace Hopper ‘superchip’ in full production

The MI300X increases the transistor count from 146 billion transistors to 153 billion, and the shared DRAM memory is increased from 128 gigabytes in the MI300A to 192 gigabytes.

Memory bandwidth has been increased from 800 gigabytes per second to 5.2 terabytes per second.

“Our use of chiplets in this product is very strategic,” Su said, because of the ability to mix and match different types of computation without swapping out CPUs or GPUs.

Su said the Mi 300X will deliver 2.4 times the memory density and 1.6 times the memory bandwidth compared to Nvidia’s H100 “Hopper” GPU.

“Generic AI, the big language model has changed the landscape,” Su said. “The need for more computation is growing exponentially, whether you’re talking about training or inference.”

To demonstrate the need for powerful computing, Sue showed that she is currently working on the most popular large language model, the open source Falcon-40B. Language models require more computation because they are built with greater numbers of what are called neural network “parameters”. Falcon-40B has 40 billion parameters.

The MI300X, he said, is the first chip powerful enough to run a neural network of that size entirely in memory, rather than ferrying data back and forth to and from external memory.

Su demonstrated the MI300X while composing a poem about San Francisco using the Falcon-40B.

“A single MI300X can run models with up to about 80 billion parameters in memory”, he said.

“When you compare the MI300X to the competition, the MI300X offers 2.4 times more memory, and 1.6 times more memory bandwidth, and with that extra memory capacity, we really have an advantage for larger language models because we Can run large models directly in memory.”

To be able to run the entire model in memory, Su said, means that, “for the largest models, that really reduces the number of GPUs you need, especially for inference.” speeds up, as well as lowers total cost of ownership.”

“By the way, I love this chip,” Sue said excitedly. “We love this chip.”

“With the MI300X, you can reduce the number of GPUs, and this will become even more important as the model size increases.”

“With more memory, more memory bandwidth, and fewer GPUs needed, we can run more inference tasks per GPU than before,” Su said. That would reduce the total cost of ownership for the larger language model, she said, making the technology more accessible.

Too: For AI’s ‘iPhone moment’, Nvidia unveils a big language model chip

To compete with Nvidia’s DGX systems, Su unveiled the “AMD Instinct Platform”, a family of AI computers. The first examples of this will combine eight of the MI300X with 1.5 terabytes of HMB3 memory. The server conforms to the industry standard Open Compute Platform specification.

“For customers, they can access all this AI compute power in memory in an industry-standard platform that goes directly into their existing infrastructure,” Su said.

Unlike the MI300X, which is only a GPU, the current MI300A is going up against Nvidia’s Grace Hopper combo chip, which uses Nvidia’s Grace CPU and its Hopper GPU, which the company announced was in full production last month .

The MI300A is being built at the El Capitan supercomputer under construction at the Department of Energy’s Lawrence Livermore National Laboratories, Su noted.

The MI300A is currently being shown as a sample to AMD customers, and the MI300X will begin sampling to customers in the third quarter of this year, Su said. Both will be in volume production in the fourth quarter, he said.

You can watch a replay of the presentation For news on the website set up by AMD,