Technical Blog
Tag: Megatron
Subscribe
Technical Walkthrough
May 09, 2022
Generating Synthetic Data with Transformers: A Solution for Enterprise Data Challenges
Data privacy and availability remain an issue for enterprises. Delve into how synthetic tabular data generated by NeMo addresses these challenges.
8 MIN READ
News
Mar 09, 2022
Insider’s Guide to GTC: Computer Vision, NLP, Recommenders, and Robotics
Great sessions on custom computer vision models, expressive TTS, localized NLP, scalable recommenders, and commercial and healthcare robotics apps.
6 MIN READ
News
Nov 09, 2021
NVIDIA Announces Riva Speech AI and Large Language Modeling Software For Enterprise
At GTC, NVIDIA unveiled breakthroughs making it simpler for enterprise and research organizations to build state-of-the-art, customizable conversational AI.
3 MIN READ
Technical Walkthrough
Oct 11, 2021
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World’s Largest and Most Powerful Generative Language Model
MT-NLG has 3x the number of parameters compared to the existing largest model of this type and demonstrates unmatched accuracy in a broad set of natural language tasks.
13 MIN READ
Technical Walkthrough
Jun 08, 2021
Accelerating Conversational AI Research with New Cutting-Edge Neural Networks and Features from NeMo 1.0
The 1.0 update brings significant architectural, code quality, and documentation improvements as well as a plethora of new state-of-the-art neural networks and pretrained checkpoints in several languages.
9 MIN READ
Technical Walkthrough
Apr 12, 2021
Scaling Language Model Training to a Trillion Parameters Using Megatron
Natural Language Processing (NLP) has seen rapid progress in recent years as computation at scale has become more available and datasets have become larger.
17 MIN READ