Red Hat Charts Course for AI Agents on Kubernetes Infrastructure

For years, Red Hat LLC has worked to simplify Kubernetes operations for enterprises focused on running applications rather than managing infrastructure. Now the company is applying that same philosophy to a new class of workloads: AI agents that not only consume computing resources but also execute commands, invoke external tools, interact with business systems and take actions with real consequences. Managing these autonomous systems requires both enforceable security controls and efficient resource allocation.

Red Hat is presenting its expanding toolkit at KubeCon + CloudNativeCon North America in Salt Lake City this week. The company's approach combines OpenShift as its Kubernetes foundation, a partnership with Nvidia Corp. on OpenShell for agent execution and security, and the llm-d project, which reached version 1.0 and handles the context-heavy inference workloads that agents generate. Together, these components enable enterprises to run autonomous AI through open, Kubernetes-based systems.

Autonomous agents differ fundamentally from traditional applications. They can generate their own commands, activate external tools and distribute work across multiple systems. This autonomy creates powerful automation possibilities but also demands robust access controls and continuous oversight.

Security isolation becomes central to agent deployment

The growing use of large language models has intensified focus on how to secure agents safely. An agent capable of reading files, executing shell commands and making network requests requires clearly defined permissions and strict isolation to operate securely in enterprise settings.

Red Hat's strategy centers on its work with Nvidia around OpenShell, a runtime that confines agents to isolated execution environments and enforces policy-based access restrictions. Nvidia created OpenShell; Red Hat is collaborating with Nvidia and the broader community to incorporate it into Red Hat AI and its open hybrid cloud platform. Red Hat characterizes this effort as part of a larger initiative to bring security-conscious agentic AI to enterprise infrastructure.

OpenShell employs sandboxing, process-level policy enforcement and restrictions on network access and credential handling. Red Hat has also outlined how OpenShell integrates with OpenShift's sandboxed containers, layering application-level controls within a virtual-machine boundary. These overlapping protective layers constrain agent actions and separate the spaces where agents run, as Red Hat explains in its technical documentation on layered sandboxing.

A critical distinction: sandboxing alone does not make an agent trustworthy. Rather, it constrains the harm an agent can inflict when its instructions, generated code or external inputs lead it astray.

Pete Cheslock, principal AI community architect at Red Hat, discussed how OpenShell builds on Kubernetes capabilities in a recent conversation with theCUBE: "The agent sandbox is one of them, so that you can deploy these agents in a secure way using Kubernetes native APIs." This observation highlights the core challenge: enterprises cannot grant authority to agents without first defining and enforcing boundaries on that authority.

Red Hat Chief Technology Officer Chris Wright underscored the necessity of agent isolation in a May interview with theCUBE. Agents capable of performing actual business operations need sandboxing and least-privilege access controls, he noted.

You've got to give it the right sandboxing. You've got to put protections around the agent, give it least privileges so it doesn't think about read versus read-write — very big difference.

Chris Wright, Red Hat Chief Technology Officer

Beyond sandboxing, enterprises must oversee agent identities, data access permissions, tool availability and authorization rules. Paul Nashawaty, practice lead and principal analyst for application development, modernization and cloud native at theCUBE Research, reinforces this point in his September analysis of AI agent governance. His research distinguishes between monitoring agent actions and verifying what agents were permitted to do. The analysis identifies runtime enforcement, delegated authority and auditable proof as essential requirements as autonomous systems enter production.

This creates an opportunity for platforms that can unify application development, infrastructure orchestration, security and observability rather than adding another layer of disconnected tooling.

Paul Nashawaty, theCUBE Research

Inference infrastructure must adapt to agent workload patterns

Agents introduce distinct demands on inference infrastructure beyond security. An agent might query a model, fetch information, call a tool and return to the model with fresh context. Each step consumes processing power, memory and bandwidth, while the system must retain sufficient context for efficient continuation.

Red Hat's llm-d project addresses this challenge. Established by Red Hat, IBM Corp., Google Cloud, CoreWeave Inc. and Nvidia, and now overseen by the Cloud Native Computing Foundation, llm-d employs Kubernetes-native components to spread inference across accelerators and route traffic between them.

On Oct. 20, the project released version 1.0, introducing session-aware routing, turn-priority scheduling, multi-tenant quality-of-service controls and high-availability capabilities. These features keep multi-turn conversations near cached context, prioritize low-latency requests and minimize disruption during updates. Red Hat's release announcement and Google Cloud's technical overview detail the release.

The design gives operators more flexibility in coordinating inference workloads and allocating accelerator resources within Kubernetes.

Brian Stevens, Red Hat Senior Vice President and AI Chief Technology Officer, highlighted one of llm-d's key architectural choices in SiliconANGLE's coverage of Kubernetes inference: "We can independently scale the processing of the input from the production of the tokens."

Other vendors are tackling related challenges. ScaleOps Labs Ltd. concentrates on GPU utilization and resource optimization across Kubernetes workloads. Vultr Holdings LLC supplies GPU infrastructure and Kubernetes-based model deployment, while Broadcom Inc. positions VMware Cloud Foundation as a private-cloud platform for Kubernetes and AI workloads. These solutions reflect the expanding range of options enterprises have for resource optimization, accelerator capacity and private-cloud operations.

An operational gap persists, however. TheCUBE Research's "Day 0 Build, Test Survey Research Report," based on responses from 375 IT, development, operations and platform engineering professionals, shows that cloud-native adoption is widespread but maturity varies. Inconsistent automated testing, scattered developer tools and missing rollback procedures continue to complicate production rollouts.

Ninety-two percent of organizations are integrating AI into at least one stage of the software development lifecycle, while 79% expect full-scale AI agent adoption within the next three years.

Paul Nashawaty, theCUBE Research

Nashawaty added that roughly 30% of organizations still encounter operational readiness obstacles.

The broader industry is moving toward production-ready infrastructure. The CNCF's 2026 annual cloud-native survey found that 82% of container users operate Kubernetes in production and 66% of organizations running generative AI workloads use Kubernetes for some or all inference. These numbers underscore Kubernetes' growing importance as enterprises adopt AI workloads.

Open-source contributions accelerate AI infrastructure development

The open-source landscape is evolving alongside agentic AI. On Nov. 5, Google announced it was donating Agent Substrate, gVisor and OpenRL to the CNCF after their acceptance into its Sandbox program in October. These projects address agent execution, isolation and reinforcement-learning workflows. Agent Substrate isolates agent sessions and identities from active compute, allowing idle environments to pause and resume; gVisor creates an additional isolation layer for untrusted code; and OpenRL enables reinforcement-learning post-training on Kubernetes. Google's announcement describes the projects and their transition into CNCF governance.

These contributions align with broader efforts within the Kubernetes community. Red Hat co-founded llm-d and is incorporating Nvidia's OpenShell into its enterprise AI platform, while Google is contributing technologies for runtime and training. The shared focus on open infrastructure creates pathways for interoperability as enterprise AI systems mature.

Broadcom marked another milestone on Nov. 9, announcing that its VM API project had been accepted into the CNCF Sandbox. The project aims to enable organizations to manage virtual machines through Kubernetes application programming interfaces across multiple infrastructure providers. This extends Kubernetes' reach to managing virtual machines alongside containerized workloads.

Project Lightwell represents another dimension of Red Hat's enterprise approach. Unveiled with IBM in May, the $5 billion initiative seeks to use AI-assisted vulnerability detection and validated fixes to secure open-source software supply chains. The initiative broadens security efforts to the software dependencies that underpin production systems.

Sam Weston, industry analyst for enterprise applications, development and modernization at theCUBE Research, points to the infrastructure landscape enterprises currently navigate.

With our data showing 61.8% of enterprises run hybrid deployments and an additional 10% operate across multi-cloud environments, Red Hat's hybrid-first platform strategy aligns closely with where enterprise infrastructure lives today.

Sam Weston, theCUBE Research

Red Hat's existing OpenShift presence offers a foundation for connecting these initiatives. As enterprises deploy more autonomous workloads, the company is positioned to merge Kubernetes orchestration, agent isolation and distributed inference within its broader hybrid cloud strategy.

The opportunity transcends any single project. OpenShift, llm-d and OpenShell demonstrate how Red Hat is advancing an open infrastructure approach that serves both conventional applications and emerging AI agents.

For enterprises, the goal is to deploy AI workloads securely, efficiently and manageably while retaining control over how autonomous systems behave. Red Hat's contributions across the Kubernetes ecosystem show how that foundation is being constructed.