Inference Economics Drive Enterprises Toward Edge Deployment and Open Models
As AI inference costs surge, companies are rethinking where models run and which ones to use, with open-source alternatives gaining ground over expensive frontier models in just three months.
The enterprise AI sector is entering a phase dominated by cost discipline, with hybrid infrastructure becoming the battleground where spending decisions get made. Rising inference expenses are forcing business leaders to make critical choices about model deployment locations, which AI systems to adopt, and how much proprietary data they are prepared to share with external vendors.
Cost pressures are driving purchasing decisions across multiple sectors, particularly in financial services where companies seek to convert disconnected data into immediate insights. Dell reports that more than 6,500 enterprises have deployed the Dell AI Factory with Nvidia updates. A dramatic transformation in token economics is reshaping customer requirements, according to Kyle Dufresne, senior vice president of sales at Dell.
It's incredible to see just over the last three months where we saw 80% of the tokens being generated from frontier models. Three months later, 80% of the tokens in the world are being generated by open models. Why? Because of cost. And everybody talks about tokenomics.
Kyle Dufresne, senior vice president of sales at Dell
Dufresne shared these insights during an interview with theCUBE Research's Dave Vellante at the Dell Technologies New York Symposium, broadcast on theCUBE, SiliconANGLE Media's livestreaming platform. The discussion covered token economics, hybrid infrastructure strategies, and Dell's approach to helping organizations plan their AI spending.
Inference costs push hybrid infrastructure toward the edge
Inference workloads are now expanding at a faster pace than training activities, though both are accelerating significantly, Dufresne noted. This shift is altering the geography of AI operations, with Dell observing inference tasks executing in multiple locations—public cloud environments, on-premises data centers, and increasingly on individual PCs.
We're seeing the business owners saying, 'Oh my gosh, these costs are getting really high for inferencing. How do we reduce those costs without curbing innovation?' We're finding it's happening across that continuum but more and more right out at the far edge.
Kyle Dufresne
The economics of inference also introduce a governance dimension centered on data ownership and competitive advantage. Organizations must determine whether to retain control of their data and proprietary models or delegate these assets to cloud providers, and they are now calculating AI expenses based on infrastructure location, model selection, and data residency. Dell's latest AI Factory offering with Nvidia updates emphasizes enabling customers to achieve results using infrastructure they control.
We're seeing every single [chief financial officer] in the world that has implemented these structures saying, 'Gosh, my token costs are exploding.' 'How do we drive costs out?'
Kyle Dufresne
Supply-side constraints present another challenge. Memory pricing is driving up technology costs rather than reducing them, and Dufresne projected that approximately 20% additional memory capacity will enter the market next year against roughly 200% more demand. Dell is addressing this mismatch through collaborative forecasting, directing customers toward components with better availability and providing advance notice of upcoming price movements. Many organizations find that their cost of capital is lower than the rate of component inflation, prompting them to accelerate purchasing timelines.
https://www.youtube.com/embed/qKnzlMX5TTI?feature=oembed
It's not only just being able to provide the technology. But also to be able to share with customers what's going to happen in the future so they can make better decisions today on the use of their capital.
Kyle Dufresne