Amazon is looking to take on Nvidia, the dominant player in the AI market, by purportedly undercutting its pricing by 25%.
An anonymous AWS cloud customer has revealed that the cloud giant pitched it on renting servers powered by its purpose-built Trainium chips, offering the same computing power as Nvidia’s highly sought-after H100 chips at three-quarters of the price.
Coincidentally or not, the news came out as Nvidia hosts its flagship event, Nvidia GTC, this week.
“By offering access to their Trainium chips, Amazon is mitigating the constraint of access to GPUs for AI and keeping interest in AI in the cloud,” said Forrester Senior Analyst Alvin Nguyen. “This also mitigates their dependency on Nvidia as a supplier for a critical service.”
The most focused and aggressive of the large CSPs
Nvidia’s architecture is highly sought after, but expensive and difficult to come by. Thus, the company is facing growing competition, even from its own customers, including Google, which is working on the next generation of its Tensor Processing Units to rival GPUs, Microsoft, which offers its custom-designed Azure chips, and, of course, Amazon.
Other players such as AMD, Broadcom, Qualcomm, Intel, and Marvell have also taken aim at Nvidia.
All of the large cloud service providers (CSPs) have developed dedicated silicon, designed in house, to offer as an alternative to commercially available chips at a “healthy discounted rate,” explained Matt Kimball, VP and principal analyst for data compute and storage at Moor Insights & Strategy.
For Amazon, that’s the Trainium, the Graviton processor, and the Inferentia chip. AWS has been able to monetize its homegrown chips quite effectively and has been “wildly successful” with Graviton, he noted. It has been the most focused and aggressive of the large CSPs in chasing the enterprise AI market with its own silicon.
“What AWS is doing is smart — it is telling the world that there is a cost-effective alternative that is also performant for AI training needs,” said Kimball. “It is inserting itself into the AI conversation.”
AI inferencing geared to the everyday enterprise
While Nvidia continues to raise the bar on AI performance with its latest chips, it’s like a set of golden handcuffs, said Nguyen. That is, the company doesn’t have much incentive to create lower-end chips for smaller workloads. This creates an opportunity for Amazon and others to gain footholds in these adjacent markets.
“This serves to fill underneath,” said Nguyen. “Purpose-built training chips are a nice to have for the open market.”
These chips can make chain-of-thought (CoT) and reasoning models, in which AI systems think out each step before providing an output, more accessible to more companies, essentially democratizing AI, he said.
“AWS will find a strong interest in the enterprise that is new to high performance computing and AI: the commercial customer,” Kimball agreed.
The result may not necessarily be production-ready AI, Nguyen emphasized, but it at least allows enterprises the ability to experiment. Many haven’t been able to do so due to access issues; one company he consults with, for instance, had to wait 18 months to even get access to an Nvidia GPU.
“That’s a long time for any business to wait,” Nguyen noted.
Conversation-starter or vendor lock-in?
Training cost is highly dependent on many factors — the number of GPUs, token count, model size, and frameworks — all of which need to come into play when enterprises consider one chip over another, Kimball emphasized.
This move from Amazon will undoubtedly be a conversation starter, and the AWS AI stack can be a great fit for some enterprises “due to its (relative) ease of use, cost and overall performance,” he said.
However, enterprises used to working with Nvidia’s compute unified device architecture (CUDA) need to think about the cost of switching to a whole new platform like Trainium. Furthermore, Trainium is only available on AWS, so users can get locked in.
“Whereas,” Kimball noted, “I can shop for the best price on the Nvidia front.” That said, however, “I’m still locked in to Nvidia, and from a capex perspective, it is more costly,” he said.
Nvidia will still dominate for the foreseeable future
Analysts agree that, despite increased competition, Nvidia will remain on top.
“AWS can provide better performance efficiency, just like others who are targeting AI inferencing at a lower cost,” said Mario Morales, group VP for enabling technologies and semiconductors at IDC.
However, he said, “Nvidia is launching next-generation architectures today that no other company, including AWS, comes close to in terms of overall performance.”
Nguyen agreed. “No one’s displacing them anytime soon. At this point, Nvidia is doing everything right; they’re controlling everything they can control,” he said. “They have the top fabs. You can’t get a GPU that’s more powerful.”
Source:: Network World