Nvidia's H100 AI GPUs cost up to four times more than AMD's competing MI300X — AMD's chips cost $10 to $15K apiece; Nvidia's H100 has peaked beyond $40,000: Report (2024)

According to Citi's price projections for AMD's MI300 AI accelerators, Nvidia currently charges up to four times more for its competing H100 GPUs, highlighting its incredible pricing power as a shortage of H100 GPUs continues. We contacted AMD to confirm Citi's pricing projection, but an AMD representative told Tom's Hardware the company doesn't share that pricing publicly.

AMD has formally started volume shipments of its CDNA 3-based Instinct MI300X accelerators and MI300A accelerated processing units (APUs), and some of the first customers have already received their MI300X parts, but pricing for different customers varies based on volumes and other factors. But in all cases, Instincts are massively cheaper than Nvidia's H100.

Citi (viaSeekingAlpha) estimates that AMD sells its Instinct MI300X 192GB to Microsoft for roughly $10,000 a unit, as the software and cloud giant is believed to be the largest consumer of these products at this time (and it has managed to bring up GPT-4 on MI300X in its production environment). Other customers have to pay around $15,000 for an Instinct MI300X GPU for artificial intelligence (AI) and high-performance computing (HPC) applications. Both prices are massively lower than Nvidia charges for itshugely popular H100 80GBAI and HPC GPU.

Just like AMD, Nvidia also does not officially disclose the pricing of its H100 80GB products as it depends on numerous factors, such as the volume of the batch and overall volumes that a particular client procures from Nvidia. But over the recent quarters, we have seen Nvidia's H100 80GB HBM2E add-in-card available for$30,000, $40,000, and evenmuch more at eBay. Meanwhile, the more powerful H100 80GB SXM with 80GB of HBM3 memory tends to cost more than an H100 80GB AIB.

In general, the prices of Nvidia's H100vary greatly, but it is not even close to $10,000 to $15,000. Furthermore, given the memory capacity of the Instinct MI300X 192GB HBM3, it makes more sense to compare it to Nvidia's upcoming H200 141GB HBM3E and Nvidia's special-editionH100 NVL 188GB HBM3 dual-card solutiondesigned specifically to train large language models (LLMs) that probably sell for an arm and a leg.

Given that tons of AI applications and workloads are optimized for Nvidia's CUDA software stack, demand for its compute GPUs is overwhelming, which is why the company can sell its Hopper-based products at a huge premium. Meanwhile, AMD is trying to attract clients to its CDNA 3-based Instinct MI300-series products, so it might have decided to sell them at a relatively low price.

AMD expects sales of its data center GPUs — which includes MI300-series devices — to exceed $3.5 billion, and the company says it has some supply still available, which stands in contrast to Nvidia's rumored 52-week wait times. In either case, analysts from Citi deem AMD's $3.5 billion in sales an underestimation.Christopher Danely, an analyst with Citi, believes that AMD could generate $5 billion on data center GPUs this year, and $8 billion in 2025.