Hydra Host Unveils 144-GPU H200 Cluster for ExaAILabs at $5 Million Price Point
Hydra Host has detailed its role in constructing the Exacluster, a system featuring 18 Lenovo machines equipped with 144 Nvidia H200 GPUs and custom networking infrastructure designed for both training and inference workloads.
A Lot of Compute Power
The foundation of this system rests on 18 Lenovo nodes housing 144 Nvidia H200 GPUs alongside 20TB of HBM3E memory—distributed at eight units per system. This configuration delivers 570 FP8 PetaTOPS of compute performance suited for artificial intelligence applications. Among these nodes, 16 undergo configuration and optimization by HydraHost specifically for training operations, which demand substantial computational and memory resources, while the remaining two function as inference nodes. Hydra Host also deployed its Brokkr platform across the infrastructure for GPU provisioning, management, and remote rental capabilities.
Working alongside Computacenter, Hydra Host engineered a networking design customized for the cluster's requirements. The architecture incorporates 3.2Tbps InfiniBand handling east-west traffic and 400Gbps Ethernet managing north-south communication, with each server receiving dual 200Gbps connections and 400Gbps Dell Ethernet switches facilitating traffic flow. Computacenter's network specialists verified that all elements conformed to Nvidia's reference architecture for optimal interoperability.
We supplied the 18 Lenovo nodes with H200 GPUs (16 interconnected and two inference nodes), designed the networking architecture in collaboration with Computacenter, and facilitated colocation through Patmos
Andrea Holt, spokesperson for Hydra Host
Beyond GPU capabilities, the infrastructure demonstrates considerable general-purpose computing strength. Each server incorporates 192 96-core processors totaling 3,456 cores, paired with 36TB of DDR5 memory and 270TB of NVMe solid-state storage. Expandable storage bays allow for straightforward capacity increases. Patmos manages the colocation arrangement, supplying approximately 100kW of power and cooling infrastructure necessary for these resource-intensive systems.
Best Performance at Best Price
The Exacluster carries a $5 million price tag, translating to $277,777 per machine—a figure comparable to pricing for a single 8-way H200 baseboard rather than a complete server system. The mechanisms behind achieving this pricing warrant examination.
Hydra Host maintains a close partnership with Nvidia and exclusively offers Nvidia GPU services, with its Brokkr software optimized primarily for CUDA. Meanwhile, ExaAI benefits from backing by Nvidia, positioning the company to potentially access preferential pricing arrangements.
We are best in market at getting our customers the right GPU for their needs and at the best price
Ryan Horjus, Lead Sales Engineer at Hydra
Horjus elaborated on the project's support structure: "This cluster was supported by Nvidia from an architecture design and their Inception program. Hydra handled it for Exa, as we do for other companies." The company has developed expertise in constructing tailored solutions for emerging businesses while generating revenue from idle hardware capacity.
Hydra has helped startups get into their own clusters for better pricing through bulk purchasing. They can achieve ideal pricing through our network. They are also able to monetize the servers when not in use via the Brokkr management platform.
Ryan Horjus
Brokkr functions as both a GPU management and provisioning platform and a monetization system for GPU resources. It delivers datacenters and startups with an integrated software solution for deploying hardware to end users and processing payments, according to Ariel Deschapell, chief technology officer and co-founder of Hydra.
One of its key features is automated bare metal provisioning and lifecycle management. That means the platform does all the work of configuring and managing the base server OS and firmware, setting up drivers and other supporting software, and running tests on the GPUs and other components. That speeds up and standardizes the delivery process significantly, reducing idle time on servers and GPUs. It also makes it easy to resell unused servers later to other users on the Brokkr platform looking for bare metal GPUs, if capacity needs change.
Ariel Deschapell, chief technology officer and co-founder of Hydra
Source: Tom's Hardware