Google Open-Sources OpenRL to Simplify Reinforcement Learning on Kubernetes
Google's Kubernetes team has donated OpenRL to the Cloud Native Computing Foundation, a framework designed to separate the complexities of infrastructure from the demands of reinforcement learning model fine-tuning.
Reinforcement learning model fine-tuning represents an increasingly taxing workload for AI infrastructure. The process intertwines training, inference and isolated agent execution within a single workflow, frequently with experimental code embedded directly into underlying systems.
Google has addressed this challenge through its Kubernetes engineering division, recently contributing OpenRL to the Cloud Native Computing Foundation. The GKE Labs team arrived at this solution by first developing an AI agent for Kubernetes operations, then creating a benchmark to evaluate model performance on those tasks, according to Sunil Arora, senior software engineer at Google Cloud.
If you look at reinforcement learning, I think of it as a kind of a super app of AI because it involves training. It involves sampling. It involves agents actually doing things in a sandbox and sometimes kind of evaluating the work. It's kind of a very diverse workload and it is, I think, pushing the boundaries of our infrastructure.
Sunil Arora, senior software engineer at Google Cloud
Arora and Justin Santa Barbara, a developer at Google, presented their work at KubeCon + CloudNativeCon NA during an exclusive broadcast, discussing model fine-tuning and GKE Labs' approach of making experimental projects available to users at an early stage.
Model fine-tuning without the infrastructure tangle
OpenRL launched as a research preview in June 2026, providing a self-hosted training API for organizations running their own Kubernetes clusters. The framework separates the reinforcement learning loop from the underlying compute infrastructure, allowing infrastructure engineers to manage scaling independently while researchers maintain control over their experimental work, Arora explained.
With OpenRL, one of the things that we are trying to do is kind of abstracting away the infrastructure as kind of core primitives so that researchers and AI engineers can just focus on their RL loop.
Sunil Arora
Sandboxes play a critical role where the reinforcement learning loop intersects with agent infrastructure, particularly when reward calculation requires executing millions of isolated operations. Google's Agent Substrate increases the density of these sandboxes on finite compute resources, according to Arora. Santa Barbara described how this need emerged during development.
https://www.youtube.com/embed/nKRVFNHsnaw?feature=oembed
We built kubectl-ai, and in theory, kubectl-ai could have formatted my hard disk. I would quite like a sandbox for this so that if it formats a hard disk, it formats that sandbox and no damage.
Justin Santa Barbara, developer at Google