AI Server under GB200 Architecture

The NVIDIA GB200 architecture combines Grace CPUs and Blackwell GPUs into high-performance AI servers, enabling rack-scale AI workloads with unprecedented throughput and efficiency.

Overview of GB200 Architecture

The NVIDIA GB200 is a unified AI computing module that integrates one Grace CPU and two Blackwell GPUs into a single superchip, interconnected via NVLink-C2C for high-bandwidth, coherent memory access at 900 GB/s. This design allows the CPU and GPUs to share memory efficiently, reducing PCIe overhead and enabling large-scale AI and HPC workloads . Each GB200 superchip uses liquid cooling for the most power-intensive components, while other components are air-cooled to maintain optimal performance .

Rack-Scale Deployment

GB200 modules are deployed in rack-scale systems such as the DGX SuperPOD and Dell PowerEdge XE8712:

• DGX SuperPOD: Built from Scalable Units (SU), each containing 8 DGX GB200 systems. A single DGX GB200 rack includes 18 compute trays, each with two GB200 superchips, totaling 72 GPUs per rack. The system leverages NVSwitch and NVLink networks for high-speed interconnects and supports large AI/ML workloads with hybrid liquid-air cooling .

• Dell PowerEdge XE8712: Designed for high-density AI inference, supporting 2 to 36 servers per IR7000 rack, with up to 144 Blackwell GPUs per rack. Integrated management and connectivity, including iDRAC10 and Integrated Rack Controller (IRC), provide telemetry, leak detection, and rack-level cooling coordination. This platform is optimized for LLM inference workloads and multitenant AI traffic .

NVL72 Configuration

The GB200 NVL72 configuration stacks 36 GB200 superchips in a single liquid-cooled rack, resulting in:

• 72 B200 GPUs
• 36 Grace CPUs
• 13.4 TB of unified GPU memory
• 1.44 exaflops FP4 compute
• NVSwitch fabric connecting all GPUs with 130 TB/s all-to-all bandwidth This setup allows training and inference of trillion-parameter models entirely within one rack, with energy efficiency up to 25x higher than previous H100 air-cooled systems .

Key Advantages

• High throughput and low latency for AI training and inference
• Scalable architecture for enterprise, research, and cloud deployments
• Unified memory domain across CPU and GPU for large model workloads
• Advanced cooling solutions to manage high-density compute
• Optimized for LLMs and HPC workloads, supporting both training and inference at rack scale

Applications

GB200-based servers are widely used for:

• Large language model training (e.g., OSSGPT120B, Llama 3.3 70B, DeepSeekR1 671B)
• High-performance AI inference in cloud and enterprise environments
• AI research and development in scalable DGX SuperPOD clusters
• Multi-tenant AI workloads requiring high-density GPU compute In summary, AI servers under the GB200 architecture provide a modular, high-performance, and energy-efficient platform for modern AI workloads, combining Grace CPUs, Blackwell GPUs, NVLink interconnects, and advanced cooling to deliver unprecedented compute density and scalability.