Member-only story
Trainium and Inferentia: Amazon Takes On NVIDIA
Amazon’s AI chips are significantly cheaper than NVIDIA’s GPUs. And these are the chips Anthropic uses to train (and maybe run) Claude.
Amazon’s Trainium chip has been making headlines lately. Amazon Web Services (AWS) has been aggressively pitching this chip to customers, offering server rentals with computing power comparable to NVIDIA’s H100 GPUs but at a steep discount (posts on X suggest it’s just a quarter of what NVIDIA charges).
It’s a bold move, and there’s a mix of strategy, economics, and market dynamics driving it.
NVIDIA’s H100 GPUs are, as you might know, the go-to for AI training, powering everything from large language models to image/video generation. But they come with a hefty price tag. Last I checked, running an EC2 P5.48xlarge instance costs around $98.32/hour (on-demand pricing). Sure, it’s a beast of a setup, with 8 H100 GPUs, 640 GB of VRAM, and 2 TB of RAM, but that price still stings.
Besides, they’re always short on supply.
The supply shortage is well-documented. TSMC’s 4nm production process just can’t keep up with the crazy demand for H100s. And it’s not for lack of effort. Yields (the percentage of chips that actually work) take a serious hit…
