Storage emerges as the critical constraint in enterprise AI infrastructure
As organizations exhaust GPU capacity, data delivery has become the primary bottleneck limiting accelerator performance. Google Cloud is repositioning storage as a foundational layer of AI infrastructure rather than a backend utility.
Two years of enterprise AI spending have centered on a single priority: acquiring compute resources. Companies pursued GPU allocations aggressively, finalized infrastructure agreements and deployed machine learning systems rapidly. Yet many encountered an unexpected reality: the constraint they believed they were addressing was not the only obstacle. Data movement has become the limiting factor constraining the very accelerators organizations worked to secure.
In a recent AppDevANGLE podcast episode, Ronen Schwartz, vice president of products for Google Cloud Storage at Google LLC, discussed how storage has emerged as the defining limitation in enterprise AI infrastructure and what a purpose-built system for AI operations actually requires. The discussion addresses a concrete, quantifiable challenge that remains largely overlooked in most AI planning conversations.
Storage just got promoted. Instead of being at the backend as an enabler, it is moving into the front as a critical part of accelerator execution.
Ronen Schwartz, Google Cloud Storage
The hidden tax on GPU investment
The financial implications are straightforward. Accelerators represent an enterprise's single largest infrastructure expense. Every moment they sit idle waiting for information represents direct, measurable financial loss rather than theoretical inefficiency. However, when teams finally obtain access to their TPUs or GPUs, the infrastructure supporting data delivery frequently has not received equivalent investment as the compute infrastructure itself.
I speak to many customers, and what I'm hearing is the team spent a long time fighting to get their TPUs, their GPUs, to get their hands on accelerators. And once they got them, they're actually not always able to get the output they needed. Very often it actually goes back to not enough thought put into the data delivery path.
Ronen Schwartz
This represents a widespread pattern rather than an isolated case. Schwartz outlined how organizations typically discover storage limitations only after they manifest—when GPU utilization falls below expectations and project schedules slip. At that stage, the expense of inadequate storage design is already embedded in operations.
Training, inference and agents: Three distinct storage problems
A significant takeaway from the discussion is that AI encompasses multiple distinct workload types, and addressing them uniformly creates architectural gaps. Schwartz identified training, inference and agentic operations as having fundamentally different operational patterns with unique data movement requirements that demand specialized approaches.
Training workloads require extensive sequential data reads, but checkpoint preservation represents an equally critical challenge. Training clusters must pause approximately every 15 minutes to write multi-terabyte checkpoints protecting completed work against system failures. Schwartz noted that suboptimal storage performance at this stage paralyzes the entire GPU cluster. Google's Rapid object storage technology, according to Schwartz, delivers checkpoint writes more than three times faster and restoration more than five times faster compared to conventional object storage solutions, a distinction that becomes significant at enterprise scale. Torin, an autonomous vehicle developer, exemplifies this requirement: the company manages millions of sensor readings flowing into storage to train vehicle models, where checkpoint efficiency is not peripheral but fundamental to operations.
Inference presents a different performance requirement: real-time, latency-critical interactions where response speed directly influences user experience and token efficiency. Agentic systems introduce a third category of demands: collaborative environments, state management across multiple agents, rapid data retrieval and support for both temporary and persistent agent processes that may suspend and resume operation.
AI is not one workload. Really, the movement of data looks completely different. It's like different heartbeats depending on where you are in the lifecycle.
Ronen Schwartz
Colossus-derived architecture as enterprise infrastructure
Google's competitive positioning rests on infrastructure experience accumulated through years of internal hyperscale AI operations before commercializing these capabilities. Schwartz cited specific performance metrics: 15 terabytes per second throughput on Rapid object storage and 10 terabytes per second on Google Cloud managed clusters, figures he characterized as verified industry-leading benchmarks.
The underlying architecture includes zonal colocation, positioning data physically adjacent to GPU and TPU clusters to reduce data transmission time to accelerators. Schwartz described this as a first-to-market feature requiring fundamental reconsideration of conventional storage assumptions, where multi-zone or multi-region redundancy was standard practice. For AI systems, accelerator proximity takes precedence. Google's Rapid Bucket technology streamlines the storage processing pipeline, allowing a single bucket to achieve more than 15 terabytes per second throughput while handling 20 million queries per second at sub-millisecond response times.
It is because it is built on the Google infrastructure, which has been used internally longer than anyone else to support accelerators and their performance. But it is also because Google is working with nine out of the ten world-leading AI frontier model companies.
Ronen Schwartz
A storage system that demonstrably increases GPU utilization enables faster development cycles through reduced idle compute time and lower per-training-run expenses.
Making unstructured data agent-ready
The agentic component of the discussion highlighted an often-overlooked gap in enterprise AI readiness. Agents require more than web search and database access; they need large-scale unstructured data availability, encompassing images, video, documents and PDFs that constitute the majority of most organizations' actual data holdings. Without mechanisms to make this content semantically discoverable, agents operate with incomplete and frequently unrepresentative access to organizational information.
Google's approach centers on what Schwartz termed a Smart Storage Initiative, built around a newly released Object Enrichment feature. When files or images arrive in a storage bucket, Google AI models automatically extract content and attach structured metadata to the object—accomplished in seconds at scale. This transformation makes previously unstructured material queryable by agents without manual processing or separate data pipelines.
An agent can now interact with what used to be called unstructured data because there is actually a very good understanding of the content.
Ronen Schwartz
The storage layer integrates with Google's broader agentic data cloud and borderless lakehouse platform, enabling agents to access data residing outside Google's infrastructure as well.
For large-scale agent deployment, Schwartz highlighted dedicated Filestore agent volumes directly integrated into Google Kubernetes Engine and the agent sandbox environment, engineered specifically to address the expense, latency and data-sharing demands of operating thousands or millions of agents concurrently.
Google Cloud Storage for AI: The bottom line
The underlying shift occurring in the market is architectural rather than incremental. Enterprise AI systems have been constructed sequentially (compute first, then frameworks, then data) with storage frequently treated as a purchasing decision rather than a performance factor. This approach is proving expensive. As GPU utilization targets become measurable business metrics and AI operations budgets face increased pressure, storage is receiving reassessment not seen in years.
Google's positioning demonstrates that operational expertise at the storage layer translates into concrete enterprise benefits—not as marketing rhetoric, but as measurable improvements in checkpoint performance, throughput, latency and agent responsiveness. Organizations that have operated frontier model training and inference at scale have already solved these technical challenges; enterprise teams are encountering them for the first time.
For infrastructure engineers, ML operations teams and enterprise architects, the takeaway is direct: storage architecture choices made during initial AI platform deployment will either constrain or enhance performance for all subsequent workloads. Addressing these decisions after bottlenecks emerge proves substantially more costly than incorporating them during the design phase.
The industry is moving toward a state where storage occupies a primary position in AI infrastructure discussions rather than serving as an afterthought. Organizations recognizing this transition promptly will realize substantially greater returns from accelerator investments already committed.