Deep Learning Demystified Webinar | Thursday, 1 December, 2022   Register Free


In recent years, multiple neural network architectures have emerged, designed to solve specific problems such as object detection, language translation, and recommendation engines. These architectures are further adapted to handle different data sizes, formats, and resolutions when applied to multiple domains in medical imaging, autonomous driving, financial services and others.

Academic and industry researchers and data scientists rely on the flexibility of the NVIDIA platform to prototype, explore, train and deploy a wide variety of deep neural networks architectures using GPU-accelerated deep learning frameworks such as MXNet, Pytorch, TensorFlow, and inference optimizers such as TensorRT.

Designed specifically for deep learning, Tensor Cores on Volta and Turing GPUs, deliver significantly higher training and inference performance compared to full precision (FP32) training. Each Tensor Core provides matrix multiply in half precision (FP16), and accumulating results in full precision (FP32). This key capability enables Volta to deliver 3X performance speedups in training and inference over the previous generation. All samples are optimized to take advantage of Tensor Cores and have been tested for accuracy and convergence. You can access these reference implementations through NVIDIA NGC and GitHub.

NVIDIA NGC Containers

Tensor Cores optimized training code-samples that ship with NVIDIA optimized PyTorch, MXNet and TensorFlow containers.

Get NGC Container

NVIDIA NGC Model Scripts

Tensor Cores optimized training code-samples. Learn how they are implemented, train with your own data or integrate into your applications.

Get Model Scripts

Model Scripts by Application Areas

Click on the application area to jump directly to that section:

Computer Vision

Computer vision deals with algorithms and techniques for computers to understand the world around us using image and video data or in other words, teaching machines to automate the tasks performed by human visual systems. Common computer vision tasks include image classification, object detection in images and videos, image segmentation, and image restoration. In recent years, deep learning has revolutionized the field of computer vision with algorithms that deliver super-human accuracy on the above tasks. Below is a list of popular deep neural network models used in computer vision and their open-source implementation.

EfficientNet(B0.B4) for PyTorch

Website>   GitHub>

EfficientNet(B0.B4) for Tensorflow2

Website>   GitHub>

nnUNet

Website> 

ResNet-50 for TensorFlow

Website>   GitHub>

ResNet-50 for MXNet

Website>   GitHub>

ResNet-50 for PyTorch

Website>   GitHub>

ResNeXt101-32x4d for PyTorch

Website>   GitHub>

SE-ResNeXt101-32x4d for PyTorch

Website>   GitHub>

ResNext101-32x4d for TensorFlow

Website>   GitHub>

SE-ResNext101-32x4d for TensorFlow

Website>   GitHub>

SSD for TensorFlow

Website>   GitHub>

SSD for PyTorch

Website>   GitHub>

Mask R-CNN for PyTorch

Website>   GitHub>

Mask R-CNN for TensorFlow2

Website>   GitHub>

UNET-Industrial for TensorFlow

Website>   GitHub>

UNet Medical for TensorFlow

Website>   GitHub>

UNet Medical for TensorFlow 2

Website>   GitHub>

VNet for TensorFlow

Website>   GitHub>

3D-UNet Medical Image Segmentation for TensorFlow

Website>   GitHub>

nnU-Net for PyTorch

GitHub>

Natural Language Processing

Natural-language processing (NLP) deals with algorithms and techniques for computers to understand, interpret, manipulate and converse in human languages. NLP algorithms can work with audio and text data and transform them into audio or text outputs. Common NLP tasks include sentiment analysis, speech recognition, speech synthesis, language translation, and natural-language generation. Deep learning algorithms enable end-to-end training of NLP models without the need to hand-engineer features from raw input data. Below is a list of popular deep neural network models used in natural language processing their open source implementations.

BERT for Tensorflow2

Website>   GitHub>

GNMT for TensorFlow

Website>   GitHub>

GNMT for PyTorch

Website>   GitHub>

BERT for TensorFlow

Website>   GitHub>

Electra/TF2

Website>   GitHub>

Transformer for PyTorch

Website>   GitHub>

Fastertransformer

   GitHub>

Transformer-XL for PyTorch

Website>   GitHub>

BioBERT for Tensorflow

Website>   GitHub>

BERT for PyTorch

Website>   GitHub>

Transformer-XL For TensorFlow

Website>   GitHub>

Recommender Systems

Recommender systems or recommendation engines are algorithms that offer ratings or suggestions for a particular product or item, from other possibilities, based on user behavior attributes. Common recommender system applications include recommendations for movies, music, news, books, search queries and other products. Below are examples for popular deep neural network models used for recommender systems.

NCF for PyTorch

Website>   GitHub>

NCF for TensorFlow

Website>   GitHub>

VAE-CF for TensorFlow

Website>   GitHub>

DLRM for Pytorch

Website>   GitHub>

DLRM for Tensorflow2

Website>   GitHub>

Wide & Deep for TensorFlow

Website>   GitHub>

Wide and Deep / Tensorflow2

Website>   GitHub>

Text to Speech

Tacotron and WaveGlow for PyTorch

Website>   GitHub>

FastPitch 1.0 for PyTorch

Website>   GitHub>
 

Automatic Speech Recognition

Jasper for PyTorch

Website>   GitHub>

Kaldi

Website>   GitHub>
 

Performance Guide

NVIDIA GPUs accelerate diverse application areas, from vision to speech and from recommender systems to generative adversarial networks (GANs).

They also support every deep learning framework across multiple network types, including convolutional neural networks (CNNs), recurrent neural networks (RNNs) and more.

See how optimized NGC containers and NVIDIA’s complete solution stack power your deep learning research.