Bryan Catanzaro

Bryan Catanzaro is vice president of Applied Deep Learning Research at NVIDIA, where he leads a team finding new ways to use AI to improve projects ranging from language understanding to computer graphics and chip design. Bryan's research at NVIDIA led to the creation of CUDNN, and more recently, he helped lead the team that invented DLSS 2.0. Prior to NVIDIA, he worked at Baidu to create next-generation systems for training and deploying end-to-end, deep learning-based speech recognition. Bryan received his PhD in Electrical Engineering and Computer Sciences from the University of California, Berkeley.
Avatar photo

Posts by Bryan Catanzaro

Technical Walkthrough 2

Scaling Language Model Training to a Trillion Parameters Using Megatron

Natural Language Processing (NLP) has seen rapid progress in recent years as computation at scale has become more available and datasets have become larger. At... 17 MIN READ
Technical Walkthrough 0

Adding External Knowledge and Controllability to Language Models with Megatron-CNTRL

Large language models such as Megatron and GPT-3 are transforming AI. We are excited about applications that can take advantage of these models to create better... 8 MIN READ
Technical Walkthrough 0

State-of-the-Art Language Modeling Using Megatron on the NVIDIA A100 GPU

Recent work has demonstrated that larger language models dramatically advance the state of the art in natural language processing (NLP) applications such as... 9 MIN READ
Technical Walkthrough 0

Deep Speech: Accurate Speech Recognition with GPU-Accelerated Deep Learning

Speech recognition is an established technology, but it tends to fail when we need it the most, such as in noisy or crowded environments, or when the speaker is... 9 MIN READ
Python Logo
Technical Walkthrough 0

Copperhead: Data Parallel Python

Programming environments like C and Fortran allow complete and unrestricted access to computing hardware, but often require programmers to understand the... 12 MIN READ