Deep Learning Demystified Webinar | Thursday, 1 December, 2022   Register Free


In recent years, multiple neural network architectures have emerged, designed to solve specific problems such as object detection, language translation, and recommendation engines. These architectures are further adapted to handle different data sizes, formats, and resolutions when applied to multiple domains in medical imaging, autonomous driving, financial services and others.

Academic and industry researchers and data scientists rely on the flexibility of the NVIDIA platform to prototype, explore, train and deploy a wide variety of deep neural networks architectures using GPU-accelerated deep learning frameworks such as MXNet, Pytorch, TensorFlow, and inference optimizers such as TensorRT.

Designed specifically for deep learning, Tensor Cores on Volta and Turing GPUs, deliver significantly higher training and inference performance compared to full precision (FP32) training. Each Tensor Core provides matrix multiply in half precision (FP16), and accumulating results in full precision (FP32). This key capability enables Volta to deliver 3X performance speedups in training and inference over the previous generation. All samples are optimized to take advantage of Tensor Cores and have been tested for accuracy and convergence. You can access these reference implementations through NVIDIA NGC and GitHub.

NVIDIA NGC Containers

Tensor Cores optimized training code-samples that ship with NVIDIA optimized PyTorch, MXNet and TensorFlow containers.

Get NGC Container

NVIDIA NGC Model Scripts

Tensor Cores optimized training code-samples. Learn how they are implemented, train with your own data or integrate into your applications.

Get Model Scripts

Model Scripts by Application Areas

Click on the application area to jump directly to that section:

Computer Vision

Computer vision deals with algorithms and techniques for computers to understand the world around us using image and video data or in other words, teaching machines to automate the tasks performed by human visual systems. Common computer vision tasks include image classification, object detection in images and videos, image segmentation, and image restoration. In recent years, deep learning has revolutionized the field of computer vision with algorithms that deliver super-human accuracy on the above tasks. Below is a list of popular deep neural network models used in computer vision and their open-source implementation.




EfficientNet(B0.B4) for PyTorch


Website>   GitHub>


EfficientNet(B0.B4) for Tensorflow2


Website>   GitHub>


nnUNet


Website> 


ResNet-50 for TensorFlow


Website>   GitHub>


ResNet-50 for MXNet


Website>   GitHub>


ResNet-50 for PyTorch


Website>   GitHub>


ResNeXt101-32x4d for PyTorch


Website>   GitHub>


SE-ResNeXt101-32x4d for PyTorch


Website>   GitHub>


ResNext101-32x4d for TensorFlow


Website>   GitHub>


SE-ResNext101-32x4d for TensorFlow


Website>   GitHub>



SSD for TensorFlow


Website>   GitHub>


SSD for PyTorch


Website>   GitHub>



Mask R-CNN for PyTorch


Website>   GitHub>


Mask R-CNN for TensorFlow2


Website>   GitHub>



UNET-Industrial for TensorFlow


Website>   GitHub>


UNet Medical for TensorFlow


Website>   GitHub>


UNet Medical for TensorFlow 2


Website>   GitHub>


VNet for TensorFlow


Website>   GitHub>


3D-UNet Medical Image Segmentation for TensorFlow


Website>   GitHub>


nnU-Net for PyTorch


GitHub>

Natural Language Processing

Natural-language processing (NLP) deals with algorithms and techniques for computers to understand, interpret, manipulate and converse in human languages. NLP algorithms can work with audio and text data and transform them into audio or text outputs. Common NLP tasks include sentiment analysis, speech recognition, speech synthesis, language translation, and natural-language generation. Deep learning algorithms enable end-to-end training of NLP models without the need to hand-engineer features from raw input data. Below is a list of popular deep neural network models used in natural language processing their open source implementations.




BERT for Tensorflow2


Website>   GitHub>


GNMT for TensorFlow


Website>   GitHub>


GNMT for PyTorch


Website>   GitHub>



BERT for TensorFlow


Website>   GitHub>


Electra/TF2


Website>   GitHub>



Transformer for PyTorch


Website>   GitHub>


Fastertransformer


   GitHub>


Transformer-XL for PyTorch


Website>   GitHub>


BioBERT for Tensorflow


Website>   GitHub>


BERT for PyTorch


Website>   GitHub>


Transformer-XL For TensorFlow


Website>   GitHub>

Recommender Systems

Recommender systems or recommendation engines are algorithms that offer ratings or suggestions for a particular product or item, from other possibilities, based on user behavior attributes. Common recommender system applications include recommendations for movies, music, news, books, search queries and other products. Below are examples for popular deep neural network models used for recommender systems.




NCF for PyTorch


Website>   GitHub>


NCF for TensorFlow


Website>   GitHub>


VAE-CF for TensorFlow


Website>   GitHub>


DLRM for Pytorch


Website>   GitHub>


DLRM for Tensorflow2


Website>   GitHub>


Wide & Deep for TensorFlow


Website>   GitHub>


Wide and Deep / Tensorflow2


Website>   GitHub>

Text to Speech




Tacotron and WaveGlow for PyTorch


Website>   GitHub>


FastPitch 1.0 for PyTorch


Website>   GitHub>
 

Automatic Speech Recognition




Jasper for PyTorch


Website>   GitHub>


Kaldi


Website>   GitHub>
 

Performance Guide

NVIDIA GPUs accelerate diverse application areas, from vision to speech and from recommender systems to generative adversarial networks (GANs).

They also support every deep learning framework across multiple network types, including convolutional neural networks (CNNs), recurrent neural networks (RNNs) and more.

See how optimized NGC containers and NVIDIA’s complete solution stack power your deep learning research.