Big Tech

Enterprise AI Moves Into Production: Cost Control and Governance Emerge as Critical Challenges

At Dell's AI Leadership Symposium, technology leaders discussed the practical obstacles facing organizations deploying artificial intelligence at scale, from managing token expenses to securing autonomous agent behavior.

5 min read
Three insights you might have missed from theCUBE’s coverage of Dell’s “AI Leadership Symposium”

Bringing enterprise AI systems from pilot to production deployment presents substantial hurdles for information technology leaders. The primary concerns revolve around controlling expenses, expanding the underlying infrastructure to handle growing workloads, and implementing safeguards to manage autonomous systems while protecting sensitive information.

These twin pressures—cost and governance—dominated conversations at Dell Technologies Inc.'s "AI Leadership Symposium." As organizations scale their AI operations, infrastructure bills and model licensing fees climb rapidly. Simultaneously, the introduction of autonomous agents raises fresh concerns about system oversight, data protection and regulatory compliance.

Companies are simultaneously experimenting with varied approaches to running AI systems, ranging from private cloud setups to models executing directly on user devices. This diversification is generating fresh demand for enterprise infrastructure solutions and vendor partnerships capable of supporting multiple deployment architectures.

No one is asking whether to adopt AI; that's pretty much a done deal. They're asking who controls it, where it runs and whose law it operates.

John Furrier, executive analyst at theCUBE Research

During the symposium, Furrier conducted conversations with technology leaders from Dell, Deepgram Inc., Singulr AI Inc., MisaLabs Inc., Intel Corp. and other organizations. These discussions examined the computational resources, vendor relationships and infrastructure requirements necessary to expand AI deployments.

https://www.youtube.com/embed/A-0d6dwvYEo?feature=oembed

Token Expenses Force Enterprises to Reassess AI Spending

The cost of cloud-based AI tokens represents a significant strain on enterprise budgets. Organizations are seeking strategies to reduce this financial pressure, according to Helen O'Sullivan, AI business development manager and solutions specialist at Dell.

A lot of our customers I support in the Bay Area, they're dealing with this real immediate struggle of running through their token budget in a matter of weeks that they had allotted for months. You still use those frontier models, but you use the open source and infrastructure on-prem to at least eliminate some of the token stress and token maxing going on right now.

Helen O'Sullivan, Dell

Enterprises may also gain efficiency by leveraging unused capacity within their existing data center infrastructure when planning AI technology investments. Dell and Intel are assisting customers in evaluating these possibilities, according to Tim Wood, sales director for the Northwest region at Intel.

https://www.youtube.com/embed/Xg7XBk0Nweo?feature=oembed

A lot of enterprises already have [central processing units] in their data center. There may be existing resources and headroom in the infrastructure they already have that's capable of doing some of these next-generation apps and workloads.

Tim Wood, Intel

Rapid AI Advancement Demands Stronger Agent Oversight and Security Controls

As organizations deploy growing numbers of AI agents, they require improved mechanisms to track agent activities and verify that those actions remain aligned with organizational policies and objectives. This requires implementing governance systems that monitor and evaluate agent behavior during real-time execution, according to Shiv Agarwal, co-founder and chief executive officer of Singulr AI Inc.

[It's] as simple as an action where it reaches out to the internet to get some data or sends out an email to a private account or something. That could be deeply hidden inside an agentic action.

Shiv Agarwal, Singulr AI Inc.

Effective governance also requires establishing restrictions on where models can retrieve data and the methods they employ to process it. Many CIOs favor isolated virtual private cloud environments to maintain sensitive data within secure boundaries, according to Sri Viswanath, founder and chief executive officer of Sycamore Labs Inc.

They're also trying to architect in a way that all of the data stays there. They don't trust sending it outside because then we don't know where it goes.

Sri Viswanath, Sycamore Labs Inc.

https://www.youtube.com/embed/ElKU0iRM630?feature=oembed

The accelerating pace of AI innovation complicates governance implementation. Organizations must maintain system reliability and predictability even as AI technologies advance, according to Victor Jakubiuk, co-founder of MisaLabs Inc.

AI has no limit to the speed of development, speed of progress. As an enterprise, as that AI lead, you want to be able to take the latest and greatest and stay on top.

Victor Jakubiuk, MisaLabs Inc.

Custom-Built Solutions Reshape Enterprise AI Infrastructure Strategy

https://www.youtube.com/embed/mFpqDnWRBU4?feature=oembed

Deepgram is collaborating with Dell and Fortanix Inc. to move its voice-based AI models beyond cloud environments and deploy them directly onto customer hardware. Potential use cases span from voice response systems for small enterprises to AI-powered computing devices for educational users, according to Nick Mann, staff technical program manager at Deepgram.

I want our models on those, on the hardware, to run natively. Air-gapped. That's where we want to get to. That's where AI as a whole needs to start meeting people where they are and how they're going to use it.

Nick Mann, Deepgram

https://www.youtube.com/embed/s_I3bGW1AN4?feature=oembed

This initiative reflects a wider movement toward creating AI solutions customized to individual customer requirements and deployment preferences. In Silicon Valley, where technically advanced organizations frequently prefer developing proprietary solutions over purchasing off-the-shelf offerings, Dell is broadening its partner network to meet this demand, according to Sean O'Connor, regional manager of ENT West, large enterprise and acquisition, at Dell.

https://www.youtube.com/embed/videoseries?list=PLcDFYmuiJhfE

For us, that notion of ecosystem is starting to change. Today we're here with Intel [Corp.], we're here with SHI [International Corp.], we're here with Microsoft [Corp.]. But that ecosystem also now is starting to bleed into private equity and venture capital.

Sean O'Connor, Dell

Source: SiliconANGLE · Reporting supplemented by The Silicon Ledger staff.