AI clusters break free from single data centers as power and cooling constraints reshape infrastructure
As artificial intelligence systems grow beyond what individual facilities can support, the industry is developing new networking technologies to connect computing resources across multiple locations and regions.

The expansion of AI clusters is forcing computing infrastructure to exceed the constraints of standalone data centers, driving demand for networking solutions that can link computational resources across multiple sites and geographic areas. Researchers from The Ohio State University presented findings at Supercomputing 2026 (SC26) on how high-performance networking has evolved to meet these demands. Their work underscored that connecting workloads across separate data centers in different regions requires far more than simply increasing network speed, emphasizing the need for coordinated scaling across multiple domains.
Massive systems call for new interconnects
Contemporary AI clusters house well over 100,000 AI accelerators alongside potentially millions of CPU nodes, straining network bandwidth, latency, and congestion handling capabilities. The computational output of accelerators now grows faster than their communication abilities, meaning that more powerful AI GPUs do not automatically translate to better overall application results. The performance of network infrastructure therefore becomes the limiting factor in how efficiently large-scale AI systems can leverage their computing power.

High-performance computing has traditionally depended on two interconnect categories: scale-up and scale-out. Scale-up networks bind processors and accelerators within unified systems using technologies such as AMD's Infinity Fabric, Intel's UPI, Nvidia's NVLink, and newer options like UALink. Scale-out networks, commonly based on Ethernet, InfiniBand, HPE Slingshot, or the newer Ultra Ethernet, link individual servers or rack-level systems into broader clusters. Each type addresses distinct needs around bandwidth, latency, and scalability, serving different architectural purposes.
Scale-across interconnects are (almost) here
The unprecedented scale and power draw of current AI clusters have created demand for a new category: scale-across interconnects. Distinct from scale-out networks that typically operate within a single data center, scale-across systems deliver high-performance connectivity spanning different buildings, campuses, or even distant regions.
This model allows operators to pool resources from multiple locations to meet performance targets rather than concentrating vast numbers of accelerators in one facility, which may prove impractical or impossible given power availability, cooling needs, or spatial constraints.

Scale-across deployment has begun moving from concept to reality. Google rolled out its Virgo network architecture this year to handle AI workloads that surpass what individual data centers can accommodate in terms of space and power. The company plans to operate an entire campus of facilities as a unified system, though with distinct networks managing communication between accelerators, servers, and data centers. Cisco and Marvell are also building scale-across solutions, including high-capacity routers, switching chips, and optical links intended for geographically dispersed GPU clusters.
Not without caveats
Spreading AI infrastructure across multiple locations introduces serious technical obstacles. While scale-up networks achieve exceptional bandwidth and minimal latency across short ranges, data center-to-data center communication faces the physics of long-distance transmission. Even fiber-optic systems cannot overcome propagation delays, and congestion, packet loss, and synchronization costs can degrade real-world performance. Linking two large GPU clusters with high-speed connections does not automatically make them function as a single, tightly coupled system.
These issues matter most during AI training, when accelerators constantly share model parameters, gradients, and intermediate computations. Large-scale operations demand efficient collective communications, since delays affecting individual accelerators cascade to harm the entire cluster's output. The success of distributed computing therefore hinges on network topology, congestion control, communication protocols, and software design—not just raw bandwidth figures.
More sophisticated hardware and software needed
Tackling these obstacles requires increasingly advanced network adapters and switches. Current network interface controllers can execute Remote Direct Memory Access (RDMA) operations without continuous CPU overhead, while capabilities like HPE Slingshot hardware collectives and Nvidia SHARP shift certain communication tasks onto the network hardware itself. SmartNICs and data processing units reduce the load on main processors by managing selected network and infrastructure duties in specialized silicon. These advances can boost distributed application efficiency without necessarily requiring faster physical links.
Software proves equally critical to large-scale AI infrastructure performance. Communication libraries such as libfabric, MPI, and UCX shape how applications move data and exploit modern network adapter capabilities. GPU-aware communication and tools like GPUDirect RDMA bypass unnecessary host memory transfers, while refined collective operations cut communication costs across thousands of accelerators, the researchers noted.
The mounting complexity of AI infrastructure also changes how operators should measure network performance. While the sector continues rolling out faster interconnects, including Ethernet variants with nominal speeds of 800 Gb/s and 1.6 Tb/s, peak rates alone do not capture system capabilities because real application results depend on how well network hardware and software work together to move data, prevent congestion, and synchronize work across thousands of processors.
The evolution continues
The rise of scale-across interconnection represents a shift in how AI infrastructure gets designed. With accelerator capabilities climbing and AI clusters expanding beyond single facilities, communication efficiency now rivals processor speed in importance. Coming generations of AI systems will rely not just on quicker accelerators or CPUs, but on advanced network adapters, switches, and—crucially—software capable of uniting computational resources spread across regions without excessive communication overhead.