Posts by Paresh Kharya
Technical Walkthrough
Oct 11, 2021
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World’s Largest and Most Powerful Generative Language Model
MT-NLG has 3x the number of parameters compared to the existing largest model of this type and demonstrates unmatched accuracy in a broad set of natural language tasks.
13 MIN READ
News
Jul 13, 2015
Introducing the NVIDIA OpenACC Toolkit
Programmability is crucial to accelerated computing, and NVIDIA's CUDA Toolkit has been critical to the success of GPU computing. Over 3 million CUDA Toolkits…
4 MIN READ