TensorRT Getting Started
TensorRT 7.2: What’s New
TensorRT 7.2 is packed with new optimizations that accelerate video-based workloads such as web conferencing and content streaming.
Highlights:
- Optimizations for high-quality video effects such as live virtual background, delivering 30X performance vs CPUs
- New Optimizations in RNNs, speeds up applications such as Fraud & Anomaly detection by 2x
- Fully Connected Layer Optimizations, resulting upto 250% faster inference for Recommenders and MLPs
TensorRT 7.2 is available now.
Introductory Resources
Learn how to apply TensorRT optimizations and deploy a PyTorch model to GPUs.
Build a sample TensorRT application that detects common objects in images from scratch.
Download pre-trained models optimized for TensorRT to get started quickly.
Additional TensorRT Resources
Conversational AI
- Real-Time Natural Language Understanding with BERT Using TensorRT (Blog)
- Automatic Speech Recognition with TensorRT (Notebook)
- Accelerating Real-Time Text-to-Speech with you TensorRT (Blog)
- NLU with BERT (Notebook)
- Real Time Text-to-Speech (Sample)
- Neural Machine Translation (NMT) Using A Sequence To Sequence (seq2seq) Model (Sample Code)
- Building An RNN Network Layer By Layer (Sample Code)
Image and Video
- Real-time object detection on GPUs in 10 mins (Blog)
- How to perform inference for common applications (Webinar)
- Creating object detection pipelines on GPUs (Blog)
- Object detection with SSD network (Python Code Sample)
- Object detection with SSD, Faster R-CNN networks (C++ Code Samples)
Recommendation Systems
- Accelerating Wide and Deep with TensorRT (Blog)
- Movie Recommendation Using Neural Collaborative Filter (NCF) (Sample Code)
- Deep Recommender (Sample Code)
- Intro to Recommenders in TensorRT (Video)
NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.