Compute Exchange, which describes itself as an open marketplace for AI infrastructure, has launched Pricing Intelligence Calculator (PIC), a tool it says, “delivers real-time, historical, and forecasted GPU compute pricing data.”
The introduction of PIC comes just under seven months after the company announced what it called “the world’s first auction-based exchange for AI compute resources” in which companies with excess compute capacity can connect directly with companies who need that capacity.
The new service, a release states, leverages information from Silicon Data, a source of compute market intelligence, and provides “live price benchmarks across hyperscalers, neoclouds and independent providers, customizable configurations from single-GPU requests to 2048+ B200 clusters [and] Service Level Agreements and availability matching to workload needs.”
The tool, it says, “makes it easy to compare GPU hardware, discover vendors, and reserve resources through an open, market-based approach that shows the actual prices at which compute deals are being closed.”
Compute Exchange said that over the coming months it will extend PIC’s capabilities to include forward-looking pricing trends to support reliable budget forecasting, interactive visualizations to help guide planning and procurement, and developer access for programmatic integration into R&D, DevOps, and procurement workflows.
Simeon Bochev, its CEO and co-founder, said in a statement to Network World that the firm “is the first and only neutral market for compute in the world, with over 80,000 GPUs in supply across dozens of providers.” It uses Silicon Data’s pricing index to inform market pricing on the exchange, and to drive the new pricing intelligence calculator.
Compute Exchange and Silicon Data, Bochev added “are also working on developing clearer benchmarks for the compute market, and will have more details to share on that in the coming weeks.”
PIC ‘should serve to keep suppliers honest ..’
Scott Bickley, an advisory fellow at Info-Tech Research Group, said he views the offering “as a way for enterprises to source short-term GPU capacity and possibly get a deal, especially if it is stranded capacity from the neocloud providers.”
This, he said, “would also help to benchmark costs when purchasing this capacity in general, so it’s good, but it is also straightforward in terms of the value proposition.” He also noted that most companies are not buying GPU capacity directly; “This is for those that are building their own models or deploying their own AI applications atop existing models.”
Bickley added, “it should serve to keep suppliers honest to some degree in terms of the floors and ceilings of the price to access GPU capacity.”
Soon after Compute Exchange first launched in February, Matt Kimball, VP and principal analyst for data center compute and storage at Moor Insights & Strategy, described the GPU compute situation as “pretty dire. This is driven by what most view as a single supplier (Nvidia) selling GPUs before they can even be made to a market that has an insatiable thirst.”
On Tuesday, following the announcement, he said that the concept of PIC is appealing: “I really like the idea of PIC as a tool for customers and seeing the compute exchange become an arbitrageur of sorts. This delivers a real value to [anyone] who is looking to utilize AI infrastructure,” he said.
“What would be fantastic [would be] to see Compute Exchange overlay something like MLPerf benchmarking data, to provide me with a more precise ‘performance per dollar’ kind of measurement to help aid in my decisions,” he added. “Just because, say, AWS and Azure may both employ the B300 for training doesn’t mean my training job will perform the same on both. Certainly not at the same price.”
By providing some kind of benchmarking tool to deliver this finer measure of true cost, Compute Exchange would have a service that would prove to be invaluable, said Kimball.
Also, he said, “Silicon Data provides a bit more data that I think Compute Exchange could leverage moving forward. Tools that enable me to better understand where my training job will have the least amount of environmental impact would be a good feature, especially for those customers that are under more stringent regulatory requirements in the EU.”
Huge GPU demand and limited supply
The importance of having accurate benchmarking tools for the GPU market was evident in a Forrester Research report released in June, in which its authors stated that one of the key investment choices facing tech leaders is selecting the chips used to build their AI infrastructure.
“The infrastructure needed to perform the latest genAI work is a major expense,” they wrote. “The core of this has been the AI chips used — most notably, the data center GPUs, which reached prices of more than $40,000 per unit. The choice of AI chips also impacts energy costs, data center choices, and personnel skills, which may reduce your return on investment (ROI).”
Source:: Network World