Framework of workflow for NLP.
News 0 Nov 09, 2021

NVIDIA Announces Riva Speech AI and Large Language Modeling Software For Enterprise

At GTC, NVIDIA unveiled breakthroughs making it simpler for enterprise and research organizations to build state-of-the-art, customizable conversational AI. 3 MIN READ
Technical Walkthrough 0 Oct 11, 2021

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World’s Largest and Most Powerful Generative Language Model

MT-NLG has 3x the number of parameters compared to the existing largest model of this type and demonstrates unmatched accuracy in a broad set of natural… 13 MIN READ
Technical Walkthrough 0 Jun 08, 2021

Accelerating Conversational AI Research with New Cutting-Edge Neural Networks and Features from NeMo 1.0

The 1.0 update brings significant architectural, code quality, and documentation improvements as well as a plethora of new state-of-the-art neural networks and… 9 MIN READ
Technical Walkthrough 0 Apr 12, 2021

Scaling Language Model Training to a Trillion Parameters Using Megatron

Natural Language Processing (NLP) has seen rapid progress in recent years as computation at scale has become more available and datasets have become larger. 17 MIN READ
Technical Walkthrough 0 Oct 06, 2020

Adding External Knowledge and Controllability to Language Models with Megatron-CNTRL

Large language models such as Megatron and GPT-3 are transforming AI. We are excited about applications that can take advantage of these models to create better… 8 MIN READ
Technical Walkthrough 0 May 14, 2020

State-of-the-Art Language Modeling Using Megatron on the NVIDIA A100 GPU

Recent work has demonstrated that larger language models dramatically advance the state of the art in natural language processing (NLP) applications such as… 9 MIN READ