Transfer Learning Toolkit

Boost your AI development by 10X, without a huge investment in AI expertise. Speedup training and
quickly create highly accurate and performant, domain-specific AI models.

Get Started

Creating an AI/ML model from scratch to solve a business problem is capital intensive and time consuming. Transfer learning is a popular technique that can be used to extract learned features from an existing neural network model to a new one. The NVIDIA Transfer Learning Toolkit (TLT) is the AI toolkit that abstracts away the AI/DL framework complexity and leverages high quality pre-trained models to enable you to build production quality models faster with only a fraction of data required.

A toolkit for anyone building AI apps and services, TLT helps reduce costs associated with large scale data collection, labeling, and eliminates the burden of training AI/ML models from scratch.

With TLT, you can use NVIDIA’s production quality pre-trained models and deploy as is or apply minimal fine-tuning for various computer vision and conversational AI use-cases. TLT is a core component of NVIDIA's Train, Adapt and Optimize (TAO) platform, a UI-based, guided workflow for creating AI

embedded-diagram-tlt3.0-launch-software-stack-diagram-1571862-01.png

Easier & Faster Training

Add state of the art AI to your application with minimal coding. No AI frameworks expertise needed.

Highly Accurate AI

Remove barriers and unlock higher network accuracy by using purpose-built pre-trained models

Greater Throughput

Reduce deployment costs significantly and achieve high inference throughput.

Quickly Create Customized, Production Ready Models For Computer Vision & Conversational AI

Avoid the time consuming process of creating and optimizing models from scratch or using unoptimized open source models and spend more time enhancing your application. TLT speeds up engineering efforts by over 10x using the NVIDIA production quality pre-trained models to achieve high throughput and accuracy in a shorter duration of time. These AI models are free and readily available for download from the NGC catalog.

Computer Vision Pre-Trained Models

You can jumpstart your AI project by using NVIDIA pretrained models already built for a variety of these industry use-cases, speeding up your Proof of Concept (PoC) to production process. The AI models can be readily used for common computer vision use-cases such as counting and detecting people in crowded spaces, detecting and classifying vehicles, license plate detection and recognition at a toll booth, parking management, heart rate monitoring for patients at a healthcare facility and more.

Facial Landmarks Estimation

PeopleNet

Pose Estimation

License Plate Detection and Recognition

People Detection

Detect person, bags and face in crowded spaces such as transport hubs, improve customer experiences, analyze pedestrian foot traffic and more.

License Plate Detection & Recognition

Detect and identifies vehicle license plates for various applications including parking enforcement, automated toll booth, traffic monitoring, etc.

Vehicle Detection & Classification

Detect type of vehicle or make/model of cars for smart city applications

Pose Estimation

Identify key joints on person's body.

Gaze Estimation

Estimates where a person is looking at with 3d line of sight.

Facial Landmark

Detect key landmarks on the face and track them for shape prediction, localizing the face in the image etc.

Heart Rate Estimation

Estimates heart rate using Computer vision for applications in healthcare and patient monitoring.

Human Gestures and Emotion

Computer vision tasks for detecting various hand gestures and emotion.

Segmentation

Identify each instance of multiple objects in a frame at the pixel level.

Text Recognition

Recognizes text from an image.

Object Detection

Detect one or multiple objects in a frame and place bounding boxes around the object.

Image Classification

Easily classify images into designated classes based on the image features. Supported network architectures: ResNet, GoogLeNet, EfficientNet, VGG, DarkNet, MobileNet and CSPDarkNet.

Achieve State-of-the-art Accuracy Using Model Architectures

With TLT, you have the full flexibility with either bringing your own data to fine-tune the model for a specific use-case using 100+ permutations of neural network architectures such as ResNet, VGG, FasterRCNN, RetinaNet, YOLOv3/v4 etc or use one of NVIDIA’s multi-purpose production quality models for common AI tasks instead of going through the hassle of training from scratch.

	Image Classification	Object Detection							Segmentation
		DetectNet_V2	FasterRCNN	SSD	YOLOV3	YOLOV4	RetinaNet	DSSD	MaskRCNN	UNET
ResNet 10/18/34/50/101	✔	✔	✔	✔	✔	✔	✔	✔	✔	✔
VGG16/19	✔	✔	✔	✔	✔	✔	✔	✔		✔
GoogLeNet	✔	✔	✔	✔	✔	✔	✔	✔
MobileNet V1/V2	✔	✔	✔	✔	✔	✔	✔	✔
SqueezeNet	✔	✔		✔	✔	✔	✔	✔
DarkNet 19/53	✔	✔	✔	✔	✔	✔	✔	✔
CSPDarkNet 19/53	✔					✔
EfficientNet B0/B1	✔		✔	✔			✔	✔

TLT adapts popular network architectures and backbones to your data, allowing you to train, fine tune, prune and export highly optimized and accurate AI models for high throughput inference.

Deploy State-of-the-Art AI Models

Faster Inference Using Model Pruning & Quantization-Aware Training

Companies building AI solutions are in need of highly accurate AI models that can efficiently make predictions while achieving faster inference within tight memory constraints. Unpruned AI models, in many computer vision use-cases, are not optimized for low power devices. If you are solving a problem with a limited dataset, transfer learning along with select pruning improves channel density for high throughput inference.

Learn More

Typically AI models when executed in lower precision are more compute-efficient. INT8 precision AI models are significantly faster than running inference in floating point, quantizing FP32/16 weights to INT8 post-training can reduce model accuracy due to quantization errors in some cases. With the Quantization-Aware Training (QAT) feature of TLT, quantization of weights in the training step helps produce comparable accuracy as FP16/FP32 models versus post-training quantization. With QAT in TLT, developers can achieve upto 2X inference speedup using INT8 precision while maintaining accuracy comparable to FP16.

Learn More

				Nano	TX2 NX	Xavier NX			AGX Xavier			T4	A100
Model Arch	Inference Resolution	Precision	Model Accuracy	GPU (FPS)*	GPU (FPS)*	GPU (FPS)	DLA1 (FPS)	DLA2 (FPS)	GPU (FPS)	DLA1 (FPS)	DLA2 (FPS)	GPU (FPS)	GPU (FPS)
PeopleNet-ResNet34	960x544x3	INT8	84% mAP	11	31	182	58	58	314	75	75	1043	6001
TrafficCamNet	960x544x3	INT8	84% mAP	19	51	264	105	105	478	140	140	1703	9520
LPD	640x480x3	INT8	98% mAP	66	178	770	194	194	1370	256	256	5921	21931
Facial Landmark	80x80x1	FP16	6.1 pixel error	125	319	747	-	-	1451	-	-	4735	23117
GazeNet	224x224x1 224x224x1 224x224x1 25x25x1	FP16	6.5 RMSE	98	280	923	-	-	1627	-	-	5219	26534
People Semantic Segmentation	960x544x3	INT8	92% MIOU	1.4	6	17	9	9	28	12	12	103	519
2D Body Pose Estimation	288x384x3	INT8	56% mAP	5	12	97	-	-	166	-	-	563	2686

Unlock peak inference performance with NVIDIA pre-trained models across NVIDIA platforms - Jetson Nano, TX2 NX, AGX Xavier, T4 and Ampere A100 GPUs. For more details on batch size and other models, check the detailed performance datasheet.

Note: * FP16 inference on Jetson Nano and TX2 NX

Powerful End-to-End Vision AI Pipeline Using DeepStream SDK

Build end-to-end services and solutions for transforming pixels and sensor data to actionable insights using DeepStream SDK and Transfer Learning Toolkit. The production ready AI models produced by TLT can be easily integrated with NVIDIA DeepStream SDK and TensorRT for high throughput inference and enabling you to unlock greater performance for a variety of applications including smart cities and hospitals, industrial inspection, logistics, traffic monitoring, retail analytics etc.

Learn More

Unlock highest stream density and deploy at scale using DeepStream SDK

Conversational AI Pre-Trained Models

TLT for Conversational AI includes support for Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) use-cases. You can now easily design personalized real-time call center experiences, smart kiosks, high quality services for- intent recognition, entity recognition, sentiment analysis and more using readily available pre-trained models from NGC.

Speech Recognition (ASR)

Automatic speech recognition (ASR) takes human voice as input and converts it into readable text.

Natural Language Processing (NLP)

Natural language understanding (NLU) takes text as input, understands context and intent, and uses it to generate an intelligent response.

Deploy State-of-the-Art Conversational AI Models

Powerful End-to-End AI Pipeline Using JARVIS

Jarvis is a fully accelerated application framework for developers building and deploying multimodal conversational AI services using state-of-the-art deep learning models. Developers at enterprises can easily fine-tune state-of-art-models on their data using Transfer Learning Toolkit to achieve higher accuracy of their specific context. Using optimized pretrained models and transfer learning you can train and deploy applications using just 1/10th data versus manual non-transfer learning based approaches.

Learn More

TLT ConvAI Workflow Diagram 1920x800.jpg

Train and deploy end-to-end conversational AI pipeline using Pretrained Models, TLT and Jarvis

CONVERSATIONAL AI PRE-TRAINED MODELS
COMPUTER VISION PRE-TRAINED MODELS