Announcing NVIDIA DGX GH200: The First 100 Terabyte GPU Memory System
At COMPUTEX 2023, NVIDIA announced NVIDIA DGX GH200, which marks another breakthrough in GPU-accelerated computing to power the most demanding giant AI...
Upgrading Multi-GPU Interconnectivity with the Third-Generation NVIDIA NVSwitch
Increasing demands in AI and high-performance computing (HPC) are driving a need for faster, more scalable interconnects with high-speed communication between...
Fueling High-Performance Computing with Full-Stack Innovation
High-performance computing (HPC) has become the essential instrument of scientific discovery. Whether it is discovering new, life-saving drugs, battling...
Optimizing Data Movement in GPU Applications with the NVIDIA Magnum IO Developer Environment
Magnum IO is the collection of IO technologies from NVIDIA and Mellanox that make up the IO subsystem of the modern data center and enable applications at...
Improving GPU Application Performance with NVIDIA CUDA 11.2 Device Link Time Optimization
CUDA 11.2 features the powerful link time optimization (LTO) feature for device code in GPU-accelerated applications. Device LTO brings the performance...
Accelerating NVSHMEM 2.0 Team-Based Collectives Using NCCL
NVSHMEM 2.0 is introducing a new API for performing collective operations based on the Team Management feature of the OpenSHMEM 1.5 specification. A team is a...
Optimizing Data Transfer Using Lossless Compression with NVIDIA nvcomp
One of the most interesting applications of compression is optimizing communications in GPU applications. GPUs are getting faster every year. For some apps,...
Fast, Flexible Allocation for NVIDIA CUDA with RAPIDS Memory Manager
When I joined the RAPIDS team in 2018, NVIDIA CUDA device memory allocation was a performance problem. RAPIDS cuDF allocates and deallocates memory at high...
Accelerating IO in the Modern Data Center: Network IO
This is the second post in the Accelerating IO series, which describes the architecture, components, and benefits of Magnum IO, the IO subsystem of the modern...
Figure 1. In the NVSHMEM memory model, each process (PE) has private memory, as well as symmetric memory that forms a partition of the partitioned global...
Lawrence Livermore Unveils Sierra, World's Third Fastest Supercomputer
The U.S. Department of Energy and the Lawrence Livermore National Laboratory (LLNL) last week announced the unveiling of Sierra, one of the world’s fastest...
NVIDIA Collective Communications Library (NCCL)Â provides optimized implementation of inter-GPU communication operations, such as allreduce and variants....
NVIDIA CEO Jensen Huang described the NVIDIA® DGX-2™ server as "the world's largest GPU" at its launch during GPU Technology Conference earlier this...
GPUs have been PCIe devices for many generations in client systems, and more recently in servers. The rapid growth in deep learning workloads has driven the...
[caption id="attachment_7685" align="alignright" width="300"] Figure 1: NVIDIA DGX-1.[/caption] One year ago today, NVIDIA announced the NVIDIA® DGX-1™,...
How NVLink Will Enable Faster, Easier Multi-GPU Computing
Accelerated systems have become the new standard for high performance computing (HPC) as GPUs continue to raise the bar for both performance and energy...
