DEVELOPER BLOG

Tag: pipeline parallelism

AI / Deep Learning

Scaling Language Model Training to a Trillion Parameters Using Megatron

Natural Language Processing (NLP) has seen rapid progress in recent years as computation at scale has become more available and datasets have become larger. 17 MIN READ