Posts by Sylvain Jeaugey
Technical Walkthrough
Feb 28, 2022
Doubling all2all Performance with NVIDIA Collective Communication Library 2.12
Collective communications are a performance-critical ingredient of modern distributed AI training workloads such as recommender systems and natural language...
8 MIN READ
Technical Walkthrough
Feb 04, 2019
Massively Scale Your Deep Learning Training with NCCL 2.4
Imagine using tens of thousands of GPUs to train your neural network. Using multiple GPUs to train neural networks has become quite common with all deep...
8 MIN READ
Technical Walkthrough
Sep 26, 2018
Scaling Deep Learning Training with NCCL
NVIDIA Collective Communications Library (NCCL) provides optimized implementation of inter-GPU communication operations, such as allreduce and variants....
6 MIN READ